2024-07-24T21:23:48.417 INFO:root:teuthology version: 0.0.1.dev373+ga8aed60 2024-07-24T21:23:48.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-24T21:23:48.530 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828 branch: main description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7816828' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 7bc81acc946e36bfc3ec8313b59200005ddf3f73 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 86385b1daf5235400bbcd34a02515d168bde88e2 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 86385b1daf5235400bbcd34a02515d168bde88e2 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1368 sha1: 86385b1daf5235400bbcd34a02515d168bde88e2 sleep_before_teardown: 0 subset: 738/1024 suite: fs suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBErt0WBXzXUEB9PvMo047BNCjpvpdvb6RwPbnq0YdIOgDWvHTLIk9dGv0aUkqRMTlerXg8VDktDoS19N7LKqzkE= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP94yeQkR/5J8EMWn//eiPmOMYqWeeW7q4aax6DHvpBjRa+0bfmf1FwwSvJPTb2/xZwdT5Sr3XdbU9l4kWJpzWU= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDykTHE3G+urStD64PgVZykDee77oygzW1kupzHQ3NeAEuEUc9cEy9QQ9TYTKQrYrjkORa8wp5Iy9Sb6/7IzMk8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: a8aed60fc62d4ff39d2216b360f018eba7518cfe timestamp: 2024-07-24_14:14:53 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-07-24T21:23:48.530 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa; will attempt to use it 2024-07-24T21:23:48.531 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks 2024-07-24T21:23:48.531 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-24T21:23:48.533 INFO:teuthology.task.internal:Checking packages... 2024-07-24T21:23:48.553 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '86385b1daf5235400bbcd34a02515d168bde88e2' 2024-07-24T21:23:48.553 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-24T21:23:48.553 INFO:teuthology.packaging:ref: None 2024-07-24T21:23:48.553 INFO:teuthology.packaging:tag: None 2024-07-24T21:23:48.554 INFO:teuthology.packaging:branch: main 2024-07-24T21:23:48.554 INFO:teuthology.packaging:sha1: 86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:23:48.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-07-24T21:23:48.805 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5272.g6e8aab47 2024-07-24T21:23:48.806 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-24T21:23:48.812 INFO:teuthology.task.internal:no buildpackages task found 2024-07-24T21:23:48.812 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-24T21:23:48.861 INFO:teuthology.task.internal:Saving configuration 2024-07-24T21:23:48.892 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-24T21:23:48.899 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-24T21:23:48.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-24 21:18:35.731245', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBErt0WBXzXUEB9PvMo047BNCjpvpdvb6RwPbnq0YdIOgDWvHTLIk9dGv0aUkqRMTlerXg8VDktDoS19N7LKqzkE='} 2024-07-24T21:23:48.948 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-24 21:18:35.735006', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP94yeQkR/5J8EMWn//eiPmOMYqWeeW7q4aax6DHvpBjRa+0bfmf1FwwSvJPTb2/xZwdT5Sr3XdbU9l4kWJpzWU='} 2024-07-24T21:23:48.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-24 21:18:35.733253', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDykTHE3G+urStD64PgVZykDee77oygzW1kupzHQ3NeAEuEUc9cEy9QQ9TYTKQrYrjkORa8wp5Iy9Sb6/7IzMk8='} 2024-07-24T21:23:48.973 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-24T21:23:48.980 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-24T21:23:48.980 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-24T21:23:48.980 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-24T21:23:48.980 INFO:teuthology.run_tasks:Running task console_log... 2024-07-24T21:23:49.082 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff36e6a3010>, signals=[15]) 2024-07-24T21:23:49.082 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-24T21:23:49.088 INFO:teuthology.task.internal:Opening connections... 2024-07-24T21:23:49.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:23:49.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:23:49.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-07-24T21:23:49.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:23:49.263 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-07-24T21:23:49.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:23:49.338 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-24T21:23:49.344 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-07-24T21:23:49.363 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-07-24T21:23:49.363 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-07-24T21:23:49.421 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-07-24T21:23:49.421 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-07-24T21:23:49.421 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-07-24T21:23:49.421 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-07-24T21:23:49.421 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-24T21:23:49.422 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-24T21:23:49.423 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-24T21:23:49.423 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-07-24T21:23:49.448 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-07-24T21:23:49.466 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-07-24T21:23:49.467 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-24T21:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-24T21:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-24T21:23:49.524 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-07-24T21:23:49.547 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-07-24T21:23:49.565 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-07-24T21:23:49.565 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-07-24T21:23:49.623 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-07-24T21:23:49.623 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-07-24T21:23:49.623 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-07-24T21:23:49.623 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-07-24T21:23:49.623 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-24T21:23:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-24T21:23:49.625 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-07-24T21:23:49.650 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-24T21:23:49.661 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-24T21:23:49.695 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-24T21:23:49.695 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-07-24T21:23:49.698 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-07-24T21:23:49.700 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-07-24T21:23:49.716 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-24T21:23:49.766 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-24T21:23:49.766 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-07-24T21:23:49.769 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-07-24T21:23:49.771 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-07-24T21:23:49.884 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-24T21:23:49.963 INFO:teuthology.run_tasks:Running task kernel... 2024-07-24T21:23:49.987 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '7bc81acc946e36bfc3ec8313b59200005ddf3f73'} 2024-07-24T21:23:49.987 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-07-24T21:23:49.988 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-07-24T21:23:49.988 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-24T21:23:50.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:23:50.006 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-07-24T21:23:50.065 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-437.el9.x86_64 2024-07-24T21:23:50.065 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-437.el9.x86_64 2024-07-24T21:23:50.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2024-07-24T21:23:52.827 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.2 MB 00:01 2024-07-24T21:23:54.678 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-07-24T21:23:59.020 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-07-24T21:24:00.568 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 18 kB 00:00 2024-07-24T21:24:01.606 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-07-24T21:24:07.206 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-24T21:24:09.302 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-24T21:24:09.302 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-24T21:24:09.401 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-07-24T21:24:09.410 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 84 M 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 126 M 2024-07-24T21:24:09.411 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-07-24T21:24:09.714 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 8.0 MB/s | 1.5 MB 00:00 2024-07-24T21:24:10.089 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-core-5.14.0-480.el9.x86_64.rpm 30 MB/s | 17 MB 00:00 2024-07-24T21:24:10.497 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-modules-core-5.14.0-480.el9.x86_6 38 MB/s | 30 MB 00:00 2024-07-24T21:24:10.831 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 27 MB/s | 36 MB 00:01 2024-07-24T21:24:10.834 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-07-24T21:24:10.834 INFO:teuthology.orchestra.run.smithi060.stdout:Total 59 MB/s | 84 MB 00:01 2024-07-24T21:24:11.602 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-07-24T21:24:12.000 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-07-24T21:24:12.000 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-07-24T21:24:13.036 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-07-24T21:24:13.037 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-07-24T21:24:15.637 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-07-24T21:24:16.340 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-07-24T21:24:16.441 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-24T21:24:19.232 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-24T21:24:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-24T21:24:24.579 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-24T21:24:24.630 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:24:29.865 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:25:01.351 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:25:01.392 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:25:01.770 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:25:01.770 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-07-24T21:25:01.770 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-24T21:25:01.770 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-480.el9.x86_64 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:25:02.557 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-07-24T21:25:02.935 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-07-24T21:25:03.021 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-24T21:25:03.021 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-24T21:25:03.021 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-07-24T21:25:03.021 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-24T21:25:03.021 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-24T21:25:03.022 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-07-24T21:25:03.022 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-07-24T21:25:03.022 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-07-24T21:25:03.080 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-24T21:25:03.080 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-24T21:25:03.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-07-24T21:25:03.112 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-07-24T21:25:03.112 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-07-24T21:25:03.112 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 68.el9 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8264254 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-24T21:25:03.113 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:25:03.114 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-07-24T21:25:03.115 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-24T21:25:03.115 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-480.el9.x86_64 2024-07-24T21:25:03.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-24T21:25:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-07-24T21:25:05.037 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-24T21:25:05.081 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-07-24T21:25:05.083 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-07-24T21:25:05.100 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.eKcdIDtBU2 2024-07-24T21:25:05.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eKcdIDtBU2 2024-07-24T21:25:05.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.eKcdIDtBU2 2024-07-24T21:25:05.309 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.eKcdIDtBU2 is 6KB 2024-07-24T21:25:05.356 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.eKcdIDtBU2 2024-07-24T21:25:05.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-07-24T21:25:05.434 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-24T21:25:05.434 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-24T21:25:05.434 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-24T21:25:05.434 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-07-24T21:25:05.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-07-24T21:25:05.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-07-24T21:25:35.575 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-24T21:25:35.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:25:35.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:25:54.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-07-24T21:26:03.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:26:03.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:27:03.021 DEBUG:teuthology.orchestra.remote:timed out 2024-07-24T21:27:15.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:27:15.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:27:15.340 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-07-24T21:27:15.740 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:27:15.740 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-07-24T21:27:15.740 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-07-24T21:27:15.757 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-480.el9.x86_64 2024-07-24T21:27:15.757 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-07-24T21:27:15.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-24T21:27:15.757 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-24T21:27:16.759 DEBUG:teuthology.parallel:result is None 2024-07-24T21:27:16.759 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-24T21:27:16.767 INFO:teuthology.task.internal:Creating test directory... 2024-07-24T21:27:16.768 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-24T21:27:16.771 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-24T21:27:16.774 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-24T21:27:16.792 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-24T21:27:16.807 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-24T21:27:16.856 INFO:teuthology.task.internal:Creating archive directory... 2024-07-24T21:27:16.856 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-24T21:27:16.860 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-24T21:27:16.862 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-24T21:27:17.033 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-24T21:27:17.042 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-24T21:27:17.042 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-24T21:27:17.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:27:17.059 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-24T21:27:17.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:27:17.077 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-24T21:27:17.080 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-24T21:27:17.103 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-24T21:27:17.138 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.152 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.158 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.159 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.188 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.199 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-24T21:27:17.201 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-24T21:27:17.208 INFO:teuthology.task.internal:Configuring sudo... 2024-07-24T21:27:17.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-24T21:27:17.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-24T21:27:17.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-24T21:27:17.255 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-24T21:27:17.266 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-24T21:27:17.266 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-24T21:27:17.281 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-24T21:27:17.284 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-24T21:27:17.312 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:17.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:17.433 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:17.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:17.563 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:27:17.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-24T21:27:17.630 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:17.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:17.727 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:17.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:17.851 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:27:17.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-24T21:27:17.916 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:17.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-24T21:27:18.015 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:18.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-24T21:27:18.143 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:27:18.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-24T21:27:18.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-07-24T21:27:18.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-07-24T21:27:18.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-07-24T21:27:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T21:27:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T21:27:18.289 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T21:27:18.779 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-24T21:27:18.789 INFO:teuthology.task.internal:Starting timer... 2024-07-24T21:27:18.789 INFO:teuthology.run_tasks:Running task pcp... 2024-07-24T21:27:18.848 INFO:teuthology.run_tasks:Running task selinux... 2024-07-24T21:27:18.896 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-24T21:27:18.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-07-24T21:27:18.972 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-07-24T21:27:18.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-07-24T21:27:19.060 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-07-24T21:27:19.062 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-07-24T21:27:19.132 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-07-24T21:27:19.134 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-24T21:27:19.134 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-07-24T21:27:19.163 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-07-24T21:27:19.163 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-07-24T21:27:19.216 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-07-24T21:27:19.216 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-07-24T21:27:19.239 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-07-24T21:27:19.239 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-07-24T21:27:19.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T21:27:19.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:27:19.268 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T21:27:19.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:27:19.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T21:27:19.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:27:19.321 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-24T21:27:19.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-07-24T21:27:19.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-07-24T21:27:19.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-07-24T21:27:19.398 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-24T21:27:19.409 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-07-24T21:27:19.411 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-07-24T21:27:19.534 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-24T21:27:19.545 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-24T21:27:19.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-24T21:31:56.455 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-07-24T21:31:56.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:31:56.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:31:56.534 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-07-24T21:31:56.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-07-24T21:31:56.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-07-24T21:31:56.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:31:56.695 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-07-24T21:31:56.773 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-07-24T21:31:56.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-07-24T21:31:56.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-24T21:31:56.851 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-07-24T21:31:56.930 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-07-24T21:31:56.931 INFO:teuthology.run_tasks:Running task clock... 2024-07-24T21:31:56.941 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-24T21:31:56.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-24T21:31:56.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T21:31:56.945 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-24T21:31:56.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T21:31:56.947 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-24T21:31:56.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T21:31:56.974 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-24T21:31:56.978 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-24T21:31:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-24T21:31:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-24T21:31:57.000 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-24T21:31:57.010 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-07-24T21:31:57.014 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-24T21:31:57.020 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-07-24T21:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-07-24T21:31:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-24T21:31:57.035 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-07-24T21:31:57.037 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-07-24T21:31:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-24T21:31:57.047 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-07-24T21:31:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-24T21:31:57.061 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-24T21:31:57.066 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-24T21:31:57.074 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-24T21:31:57.099 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.102 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.127 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-07-24T21:31:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.133 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T21:31:57.134 INFO:teuthology.run_tasks:Running task install... 2024-07-24T21:31:57.142 DEBUG:teuthology.task.install:project ceph 2024-07-24T21:31:57.143 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2'}} 2024-07-24T21:31:57.143 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2'} 2024-07-24T21:31:57.143 INFO:teuthology.task.install:Using flavor: default 2024-07-24T21:31:57.148 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-24T21:31:57.148 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-24T21:31:57.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2', 'tag': None, 'wait_for_package': False} 2024-07-24T21:31:57.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:31:57.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2', 'tag': None, 'wait_for_package': False} 2024-07-24T21:31:57.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:31:57.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2', 'tag': None, 'wait_for_package': False} 2024-07-24T21:31:57.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:31:57.296 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/ 2024-07-24T21:31:57.296 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5197.g86385b1d 2024-07-24T21:31:57.302 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/ 2024-07-24T21:31:57.302 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5197.g86385b1d 2024-07-24T21:31:57.313 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/ 2024-07-24T21:31:57.314 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5197.g86385b1d 2024-07-24T21:31:57.406 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-24T21:31:57.406 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:31:57.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-24T21:31:57.411 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-24T21:31:57.412 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:31:57.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-24T21:31:57.421 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-24T21:31:57.421 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:31:57.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-24T21:31:57.435 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-24T21:31:57.435 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86385b1daf5235400bbcd34a02515d168bde88e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-24T21:31:57.436 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-24T21:31:57.437 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86385b1daf5235400bbcd34a02515d168bde88e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-24T21:31:57.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-24T21:31:57.445 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86385b1daf5235400bbcd34a02515d168bde88e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-24T21:31:57.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-24T21:31:57.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-24T21:31:57.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-24T21:31:57.575 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-24T21:31:57.580 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-24T21:31:57.582 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-24T21:31:57.644 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-07-24T21:31:57.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-07-24T21:31:57.806 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-07-24T21:31:57.806 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-07-24T21:31:57.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-07-24T21:31:57.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-07-24T21:31:57.988 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-07-24T21:31:58.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-24T21:31:58.029 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-07-24T21:31:58.053 INFO:teuthology.orchestra.run.smithi060.stdout:45 files removed 2024-07-24T21:31:58.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-24T21:31:58.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-24T21:31:58.604 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 254 kB/s | 83 kB 00:00 2024-07-24T21:31:58.635 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 260 kB/s | 83 kB 00:00 2024-07-24T21:31:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 258 kB/s | 83 kB 00:00 2024-07-24T21:31:58.938 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 82 kB/s | 24 kB 00:00 2024-07-24T21:31:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 80 kB/s | 24 kB 00:00 2024-07-24T21:31:59.071 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 76 kB/s | 24 kB 00:00 2024-07-24T21:31:59.264 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-07-24T21:31:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-07-24T21:31:59.422 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-07-24T21:31:59.749 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-07-24T21:31:59.922 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-07-24T21:32:00.142 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-07-24T21:32:01.509 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-07-24T21:32:01.699 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-07-24T21:32:02.179 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-07-24T21:32:05.910 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-07-24T21:32:06.049 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-07-24T21:32:06.597 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-07-24T21:32:07.450 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 18 kB 00:00 2024-07-24T21:32:08.278 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 18 kB 00:00 2024-07-24T21:32:08.379 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-07-24T21:32:08.407 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 18 kB 00:01 2024-07-24T21:32:09.583 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 19 MB/s | 22 MB 00:01 2024-07-24T21:32:09.669 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 20 MB/s | 22 MB 00:01 2024-07-24T21:32:13.827 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-24T21:32:15.041 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-07-24T21:32:15.230 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-07-24T21:32:16.057 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:16.057 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:16.068 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-24T21:32:16.069 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-24T21:32:16.070 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-24T21:32:16.071 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-24T21:32:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-24T21:32:16.075 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-24T21:32:16.076 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-24T21:32:16.139 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-07-24T21:32:16.163 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-07-24T21:32:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 6.6 k 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.4 M 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 903 k 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 145 k 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 1.5 M 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 149 k 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 14 M 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 7.4 M 2024-07-24T21:32:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 49 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 11 M 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 50 M 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 274 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 771 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 228 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 27 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 18 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 32 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 758 k 2024-07-24T21:32:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 127 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 161 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 321 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 303 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 99 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 85 k 2024-07-24T21:32:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 170 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.4 M 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 21 M 2024-07-24T21:32:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 34 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 2.4 M 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 284 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 4.7 M 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 17 M 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 16 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 25 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-24T21:32:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 163 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 510 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.5 M 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-24T21:32:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-24T21:32:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 45 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 138 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-24T21:32:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-24T21:32:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-24T21:32:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-24T21:32:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-24T21:32:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 12 Packages 2024-07-24T21:32:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:32:16.178 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 240 M 2024-07-24T21:32:16.178 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-07-24T21:32:17.194 INFO:teuthology.orchestra.run.smithi123.stdout:(1/160): ceph-19.0.0-5197.g86385b1d.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-07-24T21:32:17.301 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.302 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.314 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-24T21:32:17.315 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.315 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-24T21:32:17.316 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-24T21:32:17.318 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-24T21:32:17.322 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-24T21:32:17.322 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-24T21:32:17.386 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 6.6 k 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.4 M 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 903 k 2024-07-24T21:32:17.412 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 145 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 1.5 M 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 149 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 14 M 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 7.4 M 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 49 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 11 M 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 50 M 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 274 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 771 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 228 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 27 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 18 k 2024-07-24T21:32:17.413 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 32 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 758 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 127 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 161 k 2024-07-24T21:32:17.414 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 321 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 303 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 99 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 85 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 170 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-24T21:32:17.415 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.4 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 21 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 34 k 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 2.4 M 2024-07-24T21:32:17.416 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 284 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 4.7 M 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 17 M 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 16 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 25 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 163 k 2024-07-24T21:32:17.417 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 510 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.5 M 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-24T21:32:17.418 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 45 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 138 k 2024-07-24T21:32:17.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-24T21:32:17.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-24T21:32:17.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-24T21:32:17.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-24T21:32:17.423 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-07-24T21:32:17.424 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-07-24T21:32:17.425 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 12 Packages 2024-07-24T21:32:17.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:32:17.425 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 240 M 2024-07-24T21:32:17.425 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-07-24T21:32:17.495 INFO:teuthology.orchestra.run.smithi123.stdout:(2/160): ceph-base-19.0.0-5197.g86385b1d.el9.x8 12 MB/s | 5.4 MB 00:00 2024-07-24T21:32:17.528 INFO:teuthology.orchestra.run.smithi123.stdout:(3/160): ceph-fuse-19.0.0-5197.g86385b1d.el9.x8 2.6 MB/s | 903 kB 00:00 2024-07-24T21:32:17.553 INFO:teuthology.orchestra.run.smithi123.stdout:(4/160): ceph-immutable-object-cache-19.0.0-519 2.4 MB/s | 145 kB 00:00 2024-07-24T21:32:17.620 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.621 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.633 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-24T21:32:17.633 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-24T21:32:17.634 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-24T21:32:17.635 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-24T21:32:17.637 INFO:teuthology.orchestra.run.smithi123.stdout:(5/160): ceph-mgr-19.0.0-5197.g86385b1d.el9.x86 18 MB/s | 1.5 MB 00:00 2024-07-24T21:32:17.637 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-24T21:32:17.640 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-24T21:32:17.640 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-24T21:32:17.708 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 6.6 k 2024-07-24T21:32:17.725 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.4 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 903 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 145 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 1.5 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 149 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 14 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 7.4 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 49 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 11 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 50 M 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 274 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 771 k 2024-07-24T21:32:17.726 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 228 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 27 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 18 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 32 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 758 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 127 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-24T21:32:17.727 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 161 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 321 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 303 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 99 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 85 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 170 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-24T21:32:17.728 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.4 M 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 3.2 M 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-24T21:32:17.729 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 21 M 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 34 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 2.4 M 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 284 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 4.7 M 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 17 M 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5197.g86385b1d.el9 ceph-noarch 16 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 25 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-24T21:32:17.730 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 163 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 510 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 5.5 M 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-24T21:32:17.731 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-24T21:32:17.732 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-24T21:32:17.733 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-24T21:32:17.733 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-24T21:32:17.733 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-24T21:32:17.734 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-24T21:32:17.734 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 45 k 2024-07-24T21:32:17.734 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.0.0-5197.g86385b1d.el9 ceph 138 k 2024-07-24T21:32:17.734 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-24T21:32:17.735 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-24T21:32:17.736 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-24T21:32:17.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-24T21:32:17.738 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:Install 148 Packages 2024-07-24T21:32:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 12 Packages 2024-07-24T21:32:17.740 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:32:17.740 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 240 M 2024-07-24T21:32:17.740 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-07-24T21:32:17.870 INFO:teuthology.orchestra.run.smithi123.stdout:(6/160): ceph-mon-19.0.0-5197.g86385b1d.el9.x86 20 MB/s | 4.7 MB 00:00 2024-07-24T21:32:17.937 INFO:teuthology.orchestra.run.smithi123.stdout:(7/160): ceph-mds-19.0.0-5197.g86385b1d.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-07-24T21:32:18.307 INFO:teuthology.orchestra.run.smithi060.stdout:(1/160): ceph-19.0.0-5197.g86385b1d.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-07-24T21:32:18.359 INFO:teuthology.orchestra.run.smithi086.stdout:(1/160): ceph-19.0.0-5197.g86385b1d.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-07-24T21:32:18.404 INFO:teuthology.orchestra.run.smithi123.stdout:(8/160): ceph-common-19.0.0-5197.g86385b1d.el9. 16 MB/s | 21 MB 00:01 2024-07-24T21:32:18.488 INFO:teuthology.orchestra.run.smithi123.stdout:(9/160): ceph-selinux-19.0.0-5197.g86385b1d.el9 303 kB/s | 25 kB 00:00 2024-07-24T21:32:18.665 INFO:teuthology.orchestra.run.smithi060.stdout:(2/160): ceph-fuse-19.0.0-5197.g86385b1d.el9.x8 2.5 MB/s | 903 kB 00:00 2024-07-24T21:32:18.741 INFO:teuthology.orchestra.run.smithi060.stdout:(3/160): ceph-base-19.0.0-5197.g86385b1d.el9.x8 7.5 MB/s | 5.4 MB 00:00 2024-07-24T21:32:18.766 INFO:teuthology.orchestra.run.smithi060.stdout:(4/160): ceph-immutable-object-cache-19.0.0-519 1.4 MB/s | 145 kB 00:00 2024-07-24T21:32:18.888 INFO:teuthology.orchestra.run.smithi123.stdout:(10/160): ceph-osd-19.0.0-5197.g86385b1d.el9.x8 17 MB/s | 17 MB 00:01 2024-07-24T21:32:18.972 INFO:teuthology.orchestra.run.smithi123.stdout:(11/160): cephfs-mirror-19.0.0-5197.g86385b1d.e 2.7 MB/s | 228 kB 00:00 2024-07-24T21:32:19.024 INFO:teuthology.orchestra.run.smithi060.stdout:(5/160): ceph-mds-19.0.0-5197.g86385b1d.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-07-24T21:32:19.026 INFO:teuthology.orchestra.run.smithi086.stdout:(2/160): ceph-fuse-19.0.0-5197.g86385b1d.el9.x8 1.3 MB/s | 903 kB 00:00 2024-07-24T21:32:19.064 INFO:teuthology.orchestra.run.smithi123.stdout:(12/160): cephfs-shell-19.0.0-5197.g86385b1d.el 294 kB/s | 27 kB 00:00 2024-07-24T21:32:19.193 INFO:teuthology.orchestra.run.smithi086.stdout:(3/160): ceph-immutable-object-cache-19.0.0-519 870 kB/s | 145 kB 00:00 2024-07-24T21:32:19.239 INFO:teuthology.orchestra.run.smithi123.stdout:(13/160): ceph-radosgw-19.0.0-5197.g86385b1d.el 8.4 MB/s | 11 MB 00:01 2024-07-24T21:32:19.256 INFO:teuthology.orchestra.run.smithi123.stdout:(14/160): libcephfs-devel-19.0.0-5197.g86385b1d 165 kB/s | 32 kB 00:00 2024-07-24T21:32:19.348 INFO:teuthology.orchestra.run.smithi123.stdout:(15/160): libcephfs2-19.0.0-5197.g86385b1d.el9. 6.8 MB/s | 758 kB 00:00 2024-07-24T21:32:19.358 INFO:teuthology.orchestra.run.smithi060.stdout:(6/160): ceph-mgr-19.0.0-5197.g86385b1d.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-07-24T21:32:19.374 INFO:teuthology.orchestra.run.smithi123.stdout:(16/160): libcephsqlite-19.0.0-5197.g86385b1d.e 1.4 MB/s | 163 kB 00:00 2024-07-24T21:32:19.424 INFO:teuthology.orchestra.run.smithi123.stdout:(17/160): librados-devel-19.0.0-5197.g86385b1d. 1.6 MB/s | 127 kB 00:00 2024-07-24T21:32:19.474 INFO:teuthology.orchestra.run.smithi123.stdout:(18/160): libradosstriper1-19.0.0-5197.g86385b1 5.0 MB/s | 510 kB 00:00 2024-07-24T21:32:19.550 INFO:teuthology.orchestra.run.smithi060.stdout:(7/160): ceph-mon-19.0.0-5197.g86385b1d.el9.x86 9.0 MB/s | 4.7 MB 00:00 2024-07-24T21:32:19.566 INFO:teuthology.orchestra.run.smithi123.stdout:(19/160): python3-ceph-argparse-19.0.0-5197.g86 491 kB/s | 45 kB 00:00 2024-07-24T21:32:19.666 INFO:teuthology.orchestra.run.smithi123.stdout:(20/160): python3-ceph-common-19.0.0-5197.g8638 1.4 MB/s | 138 kB 00:00 2024-07-24T21:32:19.766 INFO:teuthology.orchestra.run.smithi123.stdout:(21/160): python3-cephfs-19.0.0-5197.g86385b1d. 1.6 MB/s | 161 kB 00:00 2024-07-24T21:32:20.092 INFO:teuthology.orchestra.run.smithi123.stdout:(22/160): librgw2-19.0.0-5197.g86385b1d.el9.x86 8.2 MB/s | 5.5 MB 00:00 2024-07-24T21:32:20.175 INFO:teuthology.orchestra.run.smithi123.stdout:(23/160): python3-rbd-19.0.0-5197.g86385b1d.el9 3.6 MB/s | 303 kB 00:00 2024-07-24T21:32:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:(4/160): ceph-mds-19.0.0-5197.g86385b1d.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-07-24T21:32:20.234 INFO:teuthology.orchestra.run.smithi123.stdout:(24/160): python3-rgw-19.0.0-5197.g86385b1d.el9 1.7 MB/s | 99 kB 00:00 2024-07-24T21:32:20.301 INFO:teuthology.orchestra.run.smithi123.stdout:(25/160): rbd-fuse-19.0.0-5197.g86385b1d.el9.x8 1.3 MB/s | 85 kB 00:00 2024-07-24T21:32:20.326 INFO:teuthology.orchestra.run.smithi123.stdout:(26/160): python3-rados-19.0.0-5197.g86385b1d.e 573 kB/s | 321 kB 00:00 2024-07-24T21:32:20.350 INFO:teuthology.orchestra.run.smithi060.stdout:(8/160): ceph-common-19.0.0-5197.g86385b1d.el9. 9.2 MB/s | 21 MB 00:02 2024-07-24T21:32:20.401 INFO:teuthology.orchestra.run.smithi123.stdout:(27/160): rbd-nbd-19.0.0-5197.g86385b1d.el9.x86 2.2 MB/s | 170 kB 00:00 2024-07-24T21:32:20.434 INFO:teuthology.orchestra.run.smithi060.stdout:(9/160): ceph-selinux-19.0.0-5197.g86385b1d.el9 303 kB/s | 25 kB 00:00 2024-07-24T21:32:20.485 INFO:teuthology.orchestra.run.smithi086.stdout:(5/160): ceph-mgr-19.0.0-5197.g86385b1d.el9.x86 5.1 MB/s | 1.5 MB 00:00 2024-07-24T21:32:20.502 INFO:teuthology.orchestra.run.smithi123.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5197.g 343 kB/s | 34 kB 00:00 2024-07-24T21:32:20.560 INFO:teuthology.orchestra.run.smithi086.stdout:(6/160): ceph-base-19.0.0-5197.g86385b1d.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-07-24T21:32:20.594 INFO:teuthology.orchestra.run.smithi123.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5197.g86385b1 1.6 MB/s | 149 kB 00:00 2024-07-24T21:32:20.819 INFO:teuthology.orchestra.run.smithi123.stdout:(30/160): rbd-mirror-19.0.0-5197.g86385b1d.el9. 6.1 MB/s | 3.2 MB 00:00 2024-07-24T21:32:20.834 INFO:teuthology.orchestra.run.smithi060.stdout:(10/160): ceph-radosgw-19.0.0-5197.g86385b1d.el 8.5 MB/s | 11 MB 00:01 2024-07-24T21:32:20.934 INFO:teuthology.orchestra.run.smithi060.stdout:(11/160): cephfs-mirror-19.0.0-5197.g86385b1d.e 2.2 MB/s | 228 kB 00:00 2024-07-24T21:32:20.993 INFO:teuthology.orchestra.run.smithi060.stdout:(12/160): cephfs-shell-19.0.0-5197.g86385b1d.el 464 kB/s | 27 kB 00:00 2024-07-24T21:32:21.060 INFO:teuthology.orchestra.run.smithi060.stdout:(13/160): libcephfs-devel-19.0.0-5197.g86385b1d 478 kB/s | 32 kB 00:00 2024-07-24T21:32:21.160 INFO:teuthology.orchestra.run.smithi060.stdout:(14/160): libcephfs2-19.0.0-5197.g86385b1d.el9. 7.4 MB/s | 758 kB 00:00 2024-07-24T21:32:21.185 INFO:teuthology.orchestra.run.smithi086.stdout:(7/160): ceph-mon-19.0.0-5197.g86385b1d.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-07-24T21:32:21.219 INFO:teuthology.orchestra.run.smithi060.stdout:(15/160): libcephsqlite-19.0.0-5197.g86385b1d.e 2.7 MB/s | 163 kB 00:00 2024-07-24T21:32:21.278 INFO:teuthology.orchestra.run.smithi060.stdout:(16/160): librados-devel-19.0.0-5197.g86385b1d. 2.1 MB/s | 127 kB 00:00 2024-07-24T21:32:21.353 INFO:teuthology.orchestra.run.smithi123.stdout:(31/160): ceph-test-19.0.0-5197.g86385b1d.el9.x 17 MB/s | 50 MB 00:02 2024-07-24T21:32:21.353 INFO:teuthology.orchestra.run.smithi060.stdout:(17/160): libradosstriper1-19.0.0-5197.g86385b1 6.6 MB/s | 510 kB 00:00 2024-07-24T21:32:21.444 INFO:teuthology.orchestra.run.smithi123.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5197.g86 3.0 MB/s | 284 kB 00:00 2024-07-24T21:32:21.528 INFO:teuthology.orchestra.run.smithi123.stdout:(33/160): ceph-mgr-rook-19.0.0-5197.g86385b1d.e 594 kB/s | 49 kB 00:00 2024-07-24T21:32:21.611 INFO:teuthology.orchestra.run.smithi123.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5197.g8 197 kB/s | 16 kB 00:00 2024-07-24T21:32:21.704 INFO:teuthology.orchestra.run.smithi123.stdout:(35/160): ceph-volume-19.0.0-5197.g86385b1d.el9 2.9 MB/s | 274 kB 00:00 2024-07-24T21:32:21.854 INFO:teuthology.orchestra.run.smithi123.stdout:(36/160): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-07-24T21:32:21.896 INFO:teuthology.orchestra.run.smithi123.stdout:(37/160): cephadm-19.0.0-5197.g86385b1d.el9.noa 3.9 MB/s | 771 kB 00:00 2024-07-24T21:32:21.954 INFO:teuthology.orchestra.run.smithi123.stdout:(38/160): cephfs-top-19.0.0-5197.g86385b1d.el9. 182 kB/s | 18 kB 00:00 2024-07-24T21:32:21.961 INFO:teuthology.orchestra.run.smithi060.stdout:(18/160): librgw2-19.0.0-5197.g86385b1d.el9.x86 9.0 MB/s | 5.5 MB 00:00 2024-07-24T21:32:22.020 INFO:teuthology.orchestra.run.smithi060.stdout:(19/160): python3-ceph-argparse-19.0.0-5197.g86 775 kB/s | 45 kB 00:00 2024-07-24T21:32:22.036 INFO:teuthology.orchestra.run.smithi086.stdout:(8/160): ceph-radosgw-19.0.0-5197.g86385b1d.el9 13 MB/s | 11 MB 00:00 2024-07-24T21:32:22.046 INFO:teuthology.orchestra.run.smithi123.stdout:(39/160): libgfortran-11.4.1-3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2024-07-24T21:32:22.070 INFO:teuthology.orchestra.run.smithi060.stdout:(20/160): python3-ceph-common-19.0.0-5197.g8638 2.7 MB/s | 138 kB 00:00 2024-07-24T21:32:22.071 INFO:teuthology.orchestra.run.smithi123.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-07-24T21:32:22.077 INFO:teuthology.orchestra.run.smithi086.stdout:(9/160): ceph-selinux-19.0.0-5197.g86385b1d.el9 607 kB/s | 25 kB 00:00 2024-07-24T21:32:22.097 INFO:teuthology.orchestra.run.smithi123.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-07-24T21:32:22.120 INFO:teuthology.orchestra.run.smithi060.stdout:(21/160): python3-cephfs-19.0.0-5197.g86385b1d. 3.2 MB/s | 161 kB 00:00 2024-07-24T21:32:22.122 INFO:teuthology.orchestra.run.smithi123.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-07-24T21:32:22.164 INFO:teuthology.orchestra.run.smithi123.stdout:(43/160): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-07-24T21:32:22.190 INFO:teuthology.orchestra.run.smithi123.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-07-24T21:32:22.196 INFO:teuthology.orchestra.run.smithi060.stdout:(22/160): python3-rados-19.0.0-5197.g86385b1d.e 4.2 MB/s | 321 kB 00:00 2024-07-24T21:32:22.215 INFO:teuthology.orchestra.run.smithi123.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-07-24T21:32:22.240 INFO:teuthology.orchestra.run.smithi123.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-24T21:32:22.266 INFO:teuthology.orchestra.run.smithi123.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-07-24T21:32:22.279 INFO:teuthology.orchestra.run.smithi060.stdout:(23/160): python3-rbd-19.0.0-5197.g86385b1d.el9 3.6 MB/s | 303 kB 00:00 2024-07-24T21:32:22.291 INFO:teuthology.orchestra.run.smithi123.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-07-24T21:32:22.333 INFO:teuthology.orchestra.run.smithi123.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-07-24T21:32:22.355 INFO:teuthology.orchestra.run.smithi060.stdout:(24/160): python3-rgw-19.0.0-5197.g86385b1d.el9 1.3 MB/s | 99 kB 00:00 2024-07-24T21:32:22.358 INFO:teuthology.orchestra.run.smithi123.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-07-24T21:32:22.383 INFO:teuthology.orchestra.run.smithi123.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-07-24T21:32:22.408 INFO:teuthology.orchestra.run.smithi123.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-07-24T21:32:22.430 INFO:teuthology.orchestra.run.smithi060.stdout:(25/160): rbd-fuse-19.0.0-5197.g86385b1d.el9.x8 1.1 MB/s | 85 kB 00:00 2024-07-24T21:32:22.434 INFO:teuthology.orchestra.run.smithi123.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-07-24T21:32:22.467 INFO:teuthology.orchestra.run.smithi123.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 583 kB/s | 18 kB 00:00 2024-07-24T21:32:22.518 INFO:teuthology.orchestra.run.smithi123.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-07-24T21:32:22.543 INFO:teuthology.orchestra.run.smithi123.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 11 kB 00:00 2024-07-24T21:32:22.568 INFO:teuthology.orchestra.run.smithi123.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-07-24T21:32:22.594 INFO:teuthology.orchestra.run.smithi123.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-07-24T21:32:22.619 INFO:teuthology.orchestra.run.smithi123.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-07-24T21:32:22.644 INFO:teuthology.orchestra.run.smithi123.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 12 MB/s | 581 kB 00:00 2024-07-24T21:32:22.669 INFO:teuthology.orchestra.run.smithi123.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 518 kB/s | 25 kB 00:00 2024-07-24T21:32:22.695 INFO:teuthology.orchestra.run.smithi123.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-07-24T21:32:22.720 INFO:teuthology.orchestra.run.smithi123.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-07-24T21:32:22.745 INFO:teuthology.orchestra.run.smithi123.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-07-24T21:32:22.771 INFO:teuthology.orchestra.run.smithi123.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-07-24T21:32:22.805 INFO:teuthology.orchestra.run.smithi060.stdout:(26/160): rbd-mirror-19.0.0-5197.g86385b1d.el9. 8.4 MB/s | 3.2 MB 00:00 2024-07-24T21:32:22.889 INFO:teuthology.orchestra.run.smithi060.stdout:(27/160): rbd-nbd-19.0.0-5197.g86385b1d.el9.x86 2.0 MB/s | 170 kB 00:00 2024-07-24T21:32:22.913 INFO:teuthology.orchestra.run.smithi123.stdout:(66/160): ceph-mgr-dashboard-19.0.0-5197.g86385 6.0 MB/s | 14 MB 00:02 2024-07-24T21:32:22.973 INFO:teuthology.orchestra.run.smithi060.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5197.g 409 kB/s | 34 kB 00:00 2024-07-24T21:32:22.996 INFO:teuthology.orchestra.run.smithi123.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-07-24T21:32:23.063 INFO:teuthology.orchestra.run.smithi123.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-07-24T21:32:23.066 INFO:teuthology.orchestra.run.smithi060.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5197.g86385b1 1.6 MB/s | 149 kB 00:00 2024-07-24T21:32:23.105 INFO:teuthology.orchestra.run.smithi123.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-07-24T21:32:23.130 INFO:teuthology.orchestra.run.smithi123.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 6.8 MB/s | 911 kB 00:00 2024-07-24T21:32:23.155 INFO:teuthology.orchestra.run.smithi123.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-07-24T21:32:23.189 INFO:teuthology.orchestra.run.smithi123.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-07-24T21:32:23.223 INFO:teuthology.orchestra.run.smithi123.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-07-24T21:32:23.323 INFO:teuthology.orchestra.run.smithi123.stdout:(74/160): python3-babel-2.9.1-2.el9.noarch.rpm 45 MB/s | 6.0 MB 00:00 2024-07-24T21:32:23.348 INFO:teuthology.orchestra.run.smithi123.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 499 kB/s | 61 kB 00:00 2024-07-24T21:32:23.415 INFO:teuthology.orchestra.run.smithi123.stdout:(76/160): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-07-24T21:32:23.440 INFO:teuthology.orchestra.run.smithi123.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-07-24T21:32:23.466 INFO:teuthology.orchestra.run.smithi123.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-07-24T21:32:23.483 INFO:teuthology.orchestra.run.smithi123.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 621 kB/s | 26 kB 00:00 2024-07-24T21:32:23.500 INFO:teuthology.orchestra.run.smithi123.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 330 kB/s | 19 kB 00:00 2024-07-24T21:32:23.525 INFO:teuthology.orchestra.run.smithi123.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-07-24T21:32:23.542 INFO:teuthology.orchestra.run.smithi123.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 841 kB/s | 35 kB 00:00 2024-07-24T21:32:23.575 INFO:teuthology.orchestra.run.smithi123.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-07-24T21:32:23.601 INFO:teuthology.orchestra.run.smithi123.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-07-24T21:32:23.626 INFO:teuthology.orchestra.run.smithi123.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-07-24T21:32:23.693 INFO:teuthology.orchestra.run.smithi123.stdout:(86/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-07-24T21:32:23.718 INFO:teuthology.orchestra.run.smithi123.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 370 kB/s | 42 kB 00:00 2024-07-24T21:32:23.743 INFO:teuthology.orchestra.run.smithi123.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-07-24T21:32:23.768 INFO:teuthology.orchestra.run.smithi123.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-07-24T21:32:23.794 INFO:teuthology.orchestra.run.smithi123.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-07-24T21:32:23.819 INFO:teuthology.orchestra.run.smithi123.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-07-24T21:32:23.844 INFO:teuthology.orchestra.run.smithi123.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-07-24T21:32:23.870 INFO:teuthology.orchestra.run.smithi123.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-07-24T21:32:23.895 INFO:teuthology.orchestra.run.smithi123.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 899 kB/s | 44 kB 00:00 2024-07-24T21:32:23.928 INFO:teuthology.orchestra.run.smithi123.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.9 MB/s | 221 kB 00:00 2024-07-24T21:32:23.954 INFO:teuthology.orchestra.run.smithi123.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 642 kB/s | 36 kB 00:00 2024-07-24T21:32:23.979 INFO:teuthology.orchestra.run.smithi123.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 973 kB/s | 47 kB 00:00 2024-07-24T21:32:24.021 INFO:teuthology.orchestra.run.smithi123.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 4.6 MB/s | 304 kB 00:00 2024-07-24T21:32:24.046 INFO:teuthology.orchestra.run.smithi123.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 903 kB/s | 58 kB 00:00 2024-07-24T21:32:24.080 INFO:teuthology.orchestra.run.smithi123.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.7 MB/s | 96 kB 00:00 2024-07-24T21:32:24.239 INFO:teuthology.orchestra.run.smithi123.stdout:(101/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-07-24T21:32:24.264 INFO:teuthology.orchestra.run.smithi123.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 297 kB/s | 64 kB 00:00 2024-07-24T21:32:24.287 INFO:teuthology.orchestra.run.smithi086.stdout:(10/160): ceph-test-19.0.0-5197.g86385b1d.el9.x 23 MB/s | 50 MB 00:02 2024-07-24T21:32:24.289 INFO:teuthology.orchestra.run.smithi123.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2024-07-24T21:32:24.314 INFO:teuthology.orchestra.run.smithi123.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-07-24T21:32:24.336 INFO:teuthology.orchestra.run.smithi086.stdout:(11/160): cephfs-mirror-19.0.0-5197.g86385b1d.e 4.5 MB/s | 228 kB 00:00 2024-07-24T21:32:24.364 INFO:teuthology.orchestra.run.smithi123.stdout:(105/160): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-07-24T21:32:24.378 INFO:teuthology.orchestra.run.smithi086.stdout:(12/160): cephfs-shell-19.0.0-5197.g86385b1d.el 653 kB/s | 27 kB 00:00 2024-07-24T21:32:24.412 INFO:teuthology.orchestra.run.smithi086.stdout:(13/160): libcephfs-devel-19.0.0-5197.g86385b1d 953 kB/s | 32 kB 00:00 2024-07-24T21:32:24.479 INFO:teuthology.orchestra.run.smithi086.stdout:(14/160): libcephfs2-19.0.0-5197.g86385b1d.el9. 11 MB/s | 758 kB 00:00 2024-07-24T21:32:24.481 INFO:teuthology.orchestra.run.smithi123.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 216 kB/s | 19 kB 00:00 2024-07-24T21:32:24.507 INFO:teuthology.orchestra.run.smithi123.stdout:(107/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-07-24T21:32:24.532 INFO:teuthology.orchestra.run.smithi123.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-24T21:32:24.538 INFO:teuthology.orchestra.run.smithi086.stdout:(15/160): libcephsqlite-19.0.0-5197.g86385b1d.e 2.7 MB/s | 163 kB 00:00 2024-07-24T21:32:24.557 INFO:teuthology.orchestra.run.smithi123.stdout:(109/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-07-24T21:32:24.583 INFO:teuthology.orchestra.run.smithi123.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-07-24T21:32:24.588 INFO:teuthology.orchestra.run.smithi086.stdout:(16/160): librados-devel-19.0.0-5197.g86385b1d. 2.5 MB/s | 127 kB 00:00 2024-07-24T21:32:24.608 INFO:teuthology.orchestra.run.smithi123.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-07-24T21:32:24.633 INFO:teuthology.orchestra.run.smithi123.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-24T21:32:24.663 INFO:teuthology.orchestra.run.smithi086.stdout:(17/160): libradosstriper1-19.0.0-5197.g86385b1 6.7 MB/s | 510 kB 00:00 2024-07-24T21:32:24.675 INFO:teuthology.orchestra.run.smithi123.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-07-24T21:32:24.692 INFO:teuthology.orchestra.run.smithi123.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-07-24T21:32:24.707 INFO:teuthology.orchestra.run.smithi060.stdout:(30/160): ceph-test-19.0.0-5197.g86385b1d.el9.x 12 MB/s | 50 MB 00:04 2024-07-24T21:32:24.709 INFO:teuthology.orchestra.run.smithi123.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-07-24T21:32:24.743 INFO:teuthology.orchestra.run.smithi123.stdout:(116/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-07-24T21:32:24.768 INFO:teuthology.orchestra.run.smithi123.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 240 kB/s | 14 kB 00:00 2024-07-24T21:32:24.793 INFO:teuthology.orchestra.run.smithi123.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-07-24T21:32:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-07-24T21:32:24.844 INFO:teuthology.orchestra.run.smithi123.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-07-24T21:32:24.865 INFO:teuthology.orchestra.run.smithi060.stdout:(31/160): ceph-mgr-dashboard-19.0.0-5197.g86385 7.8 MB/s | 14 MB 00:01 2024-07-24T21:32:24.869 INFO:teuthology.orchestra.run.smithi123.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-07-24T21:32:24.894 INFO:teuthology.orchestra.run.smithi123.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-07-24T21:32:24.920 INFO:teuthology.orchestra.run.smithi123.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-07-24T21:32:24.930 INFO:teuthology.orchestra.run.smithi086.stdout:(18/160): librgw2-19.0.0-5197.g86385b1d.el9.x86 21 MB/s | 5.5 MB 00:00 2024-07-24T21:32:24.932 INFO:teuthology.orchestra.run.smithi060.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5197.g86 4.2 MB/s | 284 kB 00:00 2024-07-24T21:32:24.945 INFO:teuthology.orchestra.run.smithi123.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-07-24T21:32:24.989 INFO:teuthology.orchestra.run.smithi086.stdout:(19/160): python3-ceph-argparse-19.0.0-5197.g86 779 kB/s | 45 kB 00:00 2024-07-24T21:32:25.008 INFO:teuthology.orchestra.run.smithi060.stdout:(33/160): ceph-mgr-rook-19.0.0-5197.g86385b1d.e 657 kB/s | 49 kB 00:00 2024-07-24T21:32:25.012 INFO:teuthology.orchestra.run.smithi123.stdout:(125/160): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-07-24T21:32:25.037 INFO:teuthology.orchestra.run.smithi123.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-07-24T21:32:25.056 INFO:teuthology.orchestra.run.smithi086.stdout:(20/160): python3-ceph-common-19.0.0-5197.g8638 2.0 MB/s | 138 kB 00:00 2024-07-24T21:32:25.062 INFO:teuthology.orchestra.run.smithi123.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-07-24T21:32:25.079 INFO:teuthology.orchestra.run.smithi123.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-07-24T21:32:25.083 INFO:teuthology.orchestra.run.smithi060.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5197.g8 220 kB/s | 16 kB 00:00 2024-07-24T21:32:25.096 INFO:teuthology.orchestra.run.smithi123.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-07-24T21:32:25.122 INFO:teuthology.orchestra.run.smithi123.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-24T21:32:25.122 INFO:teuthology.orchestra.run.smithi086.stdout:(21/160): python3-cephfs-19.0.0-5197.g86385b1d. 2.4 MB/s | 161 kB 00:00 2024-07-24T21:32:25.138 INFO:teuthology.orchestra.run.smithi123.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-07-24T21:32:25.167 INFO:teuthology.orchestra.run.smithi060.stdout:(35/160): ceph-volume-19.0.0-5197.g86385b1d.el9 3.2 MB/s | 274 kB 00:00 2024-07-24T21:32:25.172 INFO:teuthology.orchestra.run.smithi123.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-07-24T21:32:25.197 INFO:teuthology.orchestra.run.smithi123.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-07-24T21:32:25.206 INFO:teuthology.orchestra.run.smithi086.stdout:(22/160): python3-rados-19.0.0-5197.g86385b1d.e 3.8 MB/s | 321 kB 00:00 2024-07-24T21:32:25.248 INFO:teuthology.orchestra.run.smithi123.stdout:(134/160): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-07-24T21:32:25.273 INFO:teuthology.orchestra.run.smithi123.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-07-24T21:32:25.299 INFO:teuthology.orchestra.run.smithi123.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 559 kB/s | 28 kB 00:00 2024-07-24T21:32:25.306 INFO:teuthology.orchestra.run.smithi086.stdout:(23/160): python3-rbd-19.0.0-5197.g86385b1d.el9 3.0 MB/s | 303 kB 00:00 2024-07-24T21:32:25.324 INFO:teuthology.orchestra.run.smithi123.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-07-24T21:32:25.333 INFO:teuthology.orchestra.run.smithi060.stdout:(36/160): cephadm-19.0.0-5197.g86385b1d.el9.noa 4.5 MB/s | 771 kB 00:00 2024-07-24T21:32:25.349 INFO:teuthology.orchestra.run.smithi123.stdout:(138/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 594 kB/s | 90 kB 00:00 2024-07-24T21:32:25.374 INFO:teuthology.orchestra.run.smithi123.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-07-24T21:32:25.382 INFO:teuthology.orchestra.run.smithi086.stdout:(24/160): python3-rgw-19.0.0-5197.g86385b1d.el9 1.3 MB/s | 99 kB 00:00 2024-07-24T21:32:25.400 INFO:teuthology.orchestra.run.smithi123.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-07-24T21:32:25.425 INFO:teuthology.orchestra.run.smithi123.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-24T21:32:25.440 INFO:teuthology.orchestra.run.smithi086.stdout:(25/160): rbd-fuse-19.0.0-5197.g86385b1d.el9.x8 1.4 MB/s | 85 kB 00:00 2024-07-24T21:32:25.450 INFO:teuthology.orchestra.run.smithi123.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-24T21:32:25.476 INFO:teuthology.orchestra.run.smithi123.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-24T21:32:25.484 INFO:teuthology.orchestra.run.smithi060.stdout:(37/160): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-07-24T21:32:25.501 INFO:teuthology.orchestra.run.smithi123.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-24T21:32:25.518 INFO:teuthology.orchestra.run.smithi060.stdout:(38/160): cephfs-top-19.0.0-5197.g86385b1d.el9. 99 kB/s | 18 kB 00:00 2024-07-24T21:32:25.526 INFO:teuthology.orchestra.run.smithi123.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-07-24T21:32:25.543 INFO:teuthology.orchestra.run.smithi123.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-07-24T21:32:25.568 INFO:teuthology.orchestra.run.smithi123.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-07-24T21:32:25.607 INFO:teuthology.orchestra.run.smithi086.stdout:(26/160): rbd-mirror-19.0.0-5197.g86385b1d.el9. 19 MB/s | 3.2 MB 00:00 2024-07-24T21:32:25.610 INFO:teuthology.orchestra.run.smithi123.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-07-24T21:32:25.643 INFO:teuthology.orchestra.run.smithi060.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-07-24T21:32:25.683 INFO:teuthology.orchestra.run.smithi086.stdout:(27/160): rbd-nbd-19.0.0-5197.g86385b1d.el9.x86 2.2 MB/s | 170 kB 00:00 2024-07-24T21:32:25.694 INFO:teuthology.orchestra.run.smithi123.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-07-24T21:32:25.701 INFO:teuthology.orchestra.run.smithi060.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-07-24T21:32:25.727 INFO:teuthology.orchestra.run.smithi060.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-24T21:32:25.752 INFO:teuthology.orchestra.run.smithi060.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-07-24T21:32:25.769 INFO:teuthology.orchestra.run.smithi123.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 24 MB/s | 1.8 MB 00:00 2024-07-24T21:32:25.794 INFO:teuthology.orchestra.run.smithi060.stdout:(43/160): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-07-24T21:32:25.803 INFO:teuthology.orchestra.run.smithi123.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 1.2 MB/s | 41 kB 00:00 2024-07-24T21:32:25.819 INFO:teuthology.orchestra.run.smithi060.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-07-24T21:32:25.836 INFO:teuthology.orchestra.run.smithi123.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 7.3 MB/s | 247 kB 00:00 2024-07-24T21:32:25.845 INFO:teuthology.orchestra.run.smithi060.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-07-24T21:32:25.866 INFO:teuthology.orchestra.run.smithi086.stdout:(28/160): ceph-osd-19.0.0-5197.g86385b1d.el9.x8 3.2 MB/s | 17 MB 00:05 2024-07-24T21:32:25.870 INFO:teuthology.orchestra.run.smithi060.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-24T21:32:25.870 INFO:teuthology.orchestra.run.smithi123.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-07-24T21:32:25.892 INFO:teuthology.orchestra.run.smithi086.stdout:(29/160): ceph-grafana-dashboards-19.0.0-5197.g 164 kB/s | 34 kB 00:00 2024-07-24T21:32:25.895 INFO:teuthology.orchestra.run.smithi060.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-07-24T21:32:25.903 INFO:teuthology.orchestra.run.smithi123.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 6.1 MB/s | 207 kB 00:00 2024-07-24T21:32:25.921 INFO:teuthology.orchestra.run.smithi060.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-07-24T21:32:25.945 INFO:teuthology.orchestra.run.smithi123.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-07-24T21:32:25.954 INFO:teuthology.orchestra.run.smithi060.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-07-24T21:32:25.967 INFO:teuthology.orchestra.run.smithi086.stdout:(30/160): ceph-mgr-cephadm-19.0.0-5197.g86385b1 1.5 MB/s | 149 kB 00:00 2024-07-24T21:32:25.971 INFO:teuthology.orchestra.run.smithi060.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 2.2 MB/s | 107 kB 00:00 2024-07-24T21:32:25.996 INFO:teuthology.orchestra.run.smithi060.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-07-24T21:32:26.012 INFO:teuthology.orchestra.run.smithi123.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-07-24T21:32:26.021 INFO:teuthology.orchestra.run.smithi060.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-07-24T21:32:26.047 INFO:teuthology.orchestra.run.smithi060.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-07-24T21:32:26.104 INFO:teuthology.orchestra.run.smithi123.stdout:(157/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-07-24T21:32:26.122 INFO:teuthology.orchestra.run.smithi060.stdout:(54/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-07-24T21:32:26.147 INFO:teuthology.orchestra.run.smithi060.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 18 kB 00:00 2024-07-24T21:32:26.171 INFO:teuthology.orchestra.run.smithi123.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-07-24T21:32:26.190 INFO:teuthology.orchestra.run.smithi060.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-07-24T21:32:26.215 INFO:teuthology.orchestra.run.smithi060.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-07-24T21:32:26.240 INFO:teuthology.orchestra.run.smithi060.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 911 kB/s | 45 kB 00:00 2024-07-24T21:32:26.266 INFO:teuthology.orchestra.run.smithi060.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-07-24T21:32:26.299 INFO:teuthology.orchestra.run.smithi060.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 9.9 MB/s | 581 kB 00:00 2024-07-24T21:32:26.324 INFO:teuthology.orchestra.run.smithi060.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 444 kB/s | 25 kB 00:00 2024-07-24T21:32:26.350 INFO:teuthology.orchestra.run.smithi060.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-07-24T21:32:26.375 INFO:teuthology.orchestra.run.smithi060.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-07-24T21:32:26.400 INFO:teuthology.orchestra.run.smithi060.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-07-24T21:32:26.405 INFO:teuthology.orchestra.run.smithi123.stdout:(159/160): librados2-19.0.0-5197.g86385b1d.el9. 4.0 MB/s | 3.4 MB 00:00 2024-07-24T21:32:26.426 INFO:teuthology.orchestra.run.smithi060.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-07-24T21:32:26.501 INFO:teuthology.orchestra.run.smithi060.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-07-24T21:32:26.585 INFO:teuthology.orchestra.run.smithi060.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-07-24T21:32:26.627 INFO:teuthology.orchestra.run.smithi060.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-07-24T21:32:26.660 INFO:teuthology.orchestra.run.smithi060.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-07-24T21:32:26.702 INFO:teuthology.orchestra.run.smithi060.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-07-24T21:32:26.797 INFO:teuthology.orchestra.run.smithi123.stdout:(160/160): librbd1-19.0.0-5197.g86385b1d.el9.x8 2.6 MB/s | 3.2 MB 00:01 2024-07-24T21:32:26.801 INFO:teuthology.orchestra.run.smithi086.stdout:(31/160): ceph-mgr-dashboard-19.0.0-5197.g86385 15 MB/s | 14 MB 00:00 2024-07-24T21:32:26.804 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-07-24T21:32:26.804 INFO:teuthology.orchestra.run.smithi123.stdout:Total 23 MB/s | 240 MB 00:10 2024-07-24T21:32:26.827 INFO:teuthology.orchestra.run.smithi060.stdout:(71/160): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-07-24T21:32:26.869 INFO:teuthology.orchestra.run.smithi060.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-07-24T21:32:26.895 INFO:teuthology.orchestra.run.smithi060.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-07-24T21:32:27.001 INFO:teuthology.orchestra.run.smithi086.stdout:(32/160): ceph-common-19.0.0-5197.g86385b1d.el9 2.4 MB/s | 21 MB 00:08 2024-07-24T21:32:27.026 INFO:teuthology.orchestra.run.smithi086.stdout:(33/160): ceph-mgr-modules-core-19.0.0-5197.g86 1.2 MB/s | 284 kB 00:00 2024-07-24T21:32:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:(34/160): ceph-mgr-rook-19.0.0-5197.g86385b1d.e 988 kB/s | 49 kB 00:00 2024-07-24T21:32:27.070 INFO:teuthology.orchestra.run.smithi060.stdout:(74/160): ceph-osd-19.0.0-5197.g86385b1d.el9.x8 2.2 MB/s | 17 MB 00:07 2024-07-24T21:32:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:(35/160): ceph-prometheus-alerts-19.0.0-5197.g8 328 kB/s | 16 kB 00:00 2024-07-24T21:32:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:(36/160): ceph-volume-19.0.0-5197.g86385b1d.el9 3.6 MB/s | 274 kB 00:00 2024-07-24T21:32:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:(75/160): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-07-24T21:32:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:(37/160): cephadm-19.0.0-5197.g86385b1d.el9.noa 8.2 MB/s | 771 kB 00:00 2024-07-24T21:32:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:(76/160): python3-colorama-0.4.6-3.el9.noarch.r 212 kB/s | 61 kB 00:00 2024-07-24T21:32:27.194 INFO:teuthology.orchestra.run.smithi086.stdout:(38/160): cephfs-top-19.0.0-5197.g86385b1d.el9. 272 kB/s | 18 kB 00:00 2024-07-24T21:32:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-07-24T21:32:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-07-24T21:32:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 343 kB/s | 26 kB 00:00 2024-07-24T21:32:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-07-24T21:32:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-07-24T21:32:27.319 INFO:teuthology.orchestra.run.smithi086.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-07-24T21:32:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-07-24T21:32:27.353 INFO:teuthology.orchestra.run.smithi086.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-07-24T21:32:27.378 INFO:teuthology.orchestra.run.smithi086.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-07-24T21:32:27.404 INFO:teuthology.orchestra.run.smithi086.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-07-24T21:32:27.422 INFO:teuthology.orchestra.run.smithi060.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-07-24T21:32:27.445 INFO:teuthology.orchestra.run.smithi086.stdout:(43/160): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-07-24T21:32:27.456 INFO:teuthology.orchestra.run.smithi060.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-07-24T21:32:27.471 INFO:teuthology.orchestra.run.smithi086.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-07-24T21:32:27.481 INFO:teuthology.orchestra.run.smithi060.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-07-24T21:32:27.496 INFO:teuthology.orchestra.run.smithi086.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-07-24T21:32:27.506 INFO:teuthology.orchestra.run.smithi060.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 932 kB/s | 77 kB 00:00 2024-07-24T21:32:27.522 INFO:teuthology.orchestra.run.smithi086.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-24T21:32:27.532 INFO:teuthology.orchestra.run.smithi060.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 851 kB/s | 42 kB 00:00 2024-07-24T21:32:27.547 INFO:teuthology.orchestra.run.smithi086.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-07-24T21:32:27.557 INFO:teuthology.orchestra.run.smithi060.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-07-24T21:32:27.572 INFO:teuthology.orchestra.run.smithi086.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-07-24T21:32:27.582 INFO:teuthology.orchestra.run.smithi060.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-07-24T21:32:27.606 INFO:teuthology.orchestra.run.smithi086.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-07-24T21:32:27.608 INFO:teuthology.orchestra.run.smithi060.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-07-24T21:32:27.631 INFO:teuthology.orchestra.run.smithi086.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-07-24T21:32:27.633 INFO:teuthology.orchestra.run.smithi060.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 683 kB/s | 51 kB 00:00 2024-07-24T21:32:27.656 INFO:teuthology.orchestra.run.smithi086.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-07-24T21:32:27.658 INFO:teuthology.orchestra.run.smithi060.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 720 kB/s | 54 kB 00:00 2024-07-24T21:32:27.682 INFO:teuthology.orchestra.run.smithi086.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-07-24T21:32:27.692 INFO:teuthology.orchestra.run.smithi060.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-07-24T21:32:27.707 INFO:teuthology.orchestra.run.smithi086.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 663 kB/s | 33 kB 00:00 2024-07-24T21:32:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 579 kB/s | 18 kB 00:00 2024-07-24T21:32:27.750 INFO:teuthology.orchestra.run.smithi060.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 498 kB/s | 44 kB 00:00 2024-07-24T21:32:27.791 INFO:teuthology.orchestra.run.smithi086.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-07-24T21:32:27.816 INFO:teuthology.orchestra.run.smithi086.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 154 kB/s | 11 kB 00:00 2024-07-24T21:32:27.841 INFO:teuthology.orchestra.run.smithi086.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-07-24T21:32:27.867 INFO:teuthology.orchestra.run.smithi086.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-07-24T21:32:27.900 INFO:teuthology.orchestra.run.smithi086.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-07-24T21:32:27.934 INFO:teuthology.orchestra.run.smithi086.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 8.6 MB/s | 581 kB 00:00 2024-07-24T21:32:27.935 INFO:teuthology.orchestra.run.smithi060.stdout:(95/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-07-24T21:32:27.959 INFO:teuthology.orchestra.run.smithi060.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 836 kB/s | 221 kB 00:00 2024-07-24T21:32:27.960 INFO:teuthology.orchestra.run.smithi086.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-07-24T21:32:27.984 INFO:teuthology.orchestra.run.smithi060.stdout:(97/160): qatlib-service-24.02.0-1.el9.x86_64.r 155 kB/s | 36 kB 00:00 2024-07-24T21:32:27.985 INFO:teuthology.orchestra.run.smithi086.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-07-24T21:32:28.010 INFO:teuthology.orchestra.run.smithi060.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 636 kB/s | 47 kB 00:00 2024-07-24T21:32:28.010 INFO:teuthology.orchestra.run.smithi086.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-07-24T21:32:28.023 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-07-24T21:32:28.035 INFO:teuthology.orchestra.run.smithi060.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-07-24T21:32:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-07-24T21:32:28.060 INFO:teuthology.orchestra.run.smithi060.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-07-24T21:32:28.061 INFO:teuthology.orchestra.run.smithi086.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-07-24T21:32:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-07-24T21:32:28.111 INFO:teuthology.orchestra.run.smithi060.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-07-24T21:32:28.144 INFO:teuthology.orchestra.run.smithi060.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-07-24T21:32:28.161 INFO:teuthology.orchestra.run.smithi086.stdout:(66/160): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-07-24T21:32:28.169 INFO:teuthology.orchestra.run.smithi060.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-07-24T21:32:28.212 INFO:teuthology.orchestra.run.smithi060.stdout:(105/160): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-07-24T21:32:28.212 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-07-24T21:32:28.212 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-07-24T21:32:28.286 INFO:teuthology.orchestra.run.smithi086.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-07-24T21:32:28.337 INFO:teuthology.orchestra.run.smithi060.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 283 kB/s | 19 kB 00:00 2024-07-24T21:32:28.353 INFO:teuthology.orchestra.run.smithi086.stdout:(68/160): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-07-24T21:32:28.370 INFO:teuthology.orchestra.run.smithi060.stdout:(107/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-07-24T21:32:28.395 INFO:teuthology.orchestra.run.smithi086.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-07-24T21:32:28.395 INFO:teuthology.orchestra.run.smithi060.stdout:(108/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-07-24T21:32:28.420 INFO:teuthology.orchestra.run.smithi086.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 6.8 MB/s | 911 kB 00:00 2024-07-24T21:32:28.421 INFO:teuthology.orchestra.run.smithi060.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 515 kB/s | 26 kB 00:00 2024-07-24T21:32:28.445 INFO:teuthology.orchestra.run.smithi086.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-07-24T21:32:28.446 INFO:teuthology.orchestra.run.smithi060.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-07-24T21:32:28.463 INFO:teuthology.orchestra.run.smithi060.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-07-24T21:32:28.488 INFO:teuthology.orchestra.run.smithi086.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-07-24T21:32:28.497 INFO:teuthology.orchestra.run.smithi060.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-24T21:32:28.513 INFO:teuthology.orchestra.run.smithi086.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-07-24T21:32:28.539 INFO:teuthology.orchestra.run.smithi060.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-07-24T21:32:28.564 INFO:teuthology.orchestra.run.smithi060.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-24T21:32:28.581 INFO:teuthology.orchestra.run.smithi060.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-07-24T21:32:28.607 INFO:teuthology.orchestra.run.smithi060.stdout:(116/160): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-07-24T21:32:28.623 INFO:teuthology.orchestra.run.smithi060.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 338 kB/s | 14 kB 00:00 2024-07-24T21:32:28.649 INFO:teuthology.orchestra.run.smithi060.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-07-24T21:32:28.654 INFO:teuthology.orchestra.run.smithi086.stdout:(74/160): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-07-24T21:32:28.674 INFO:teuthology.orchestra.run.smithi060.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-07-24T21:32:28.680 INFO:teuthology.orchestra.run.smithi086.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 373 kB/s | 61 kB 00:00 2024-07-24T21:32:28.700 INFO:teuthology.orchestra.run.smithi060.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-07-24T21:32:28.733 INFO:teuthology.orchestra.run.smithi060.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 3.6 MB/s | 216 kB 00:00 2024-07-24T21:32:28.755 INFO:teuthology.orchestra.run.smithi086.stdout:(76/160): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-07-24T21:32:28.758 INFO:teuthology.orchestra.run.smithi060.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-07-24T21:32:28.780 INFO:teuthology.orchestra.run.smithi086.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-07-24T21:32:28.784 INFO:teuthology.orchestra.run.smithi060.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-07-24T21:32:28.806 INFO:teuthology.orchestra.run.smithi086.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-07-24T21:32:28.809 INFO:teuthology.orchestra.run.smithi060.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-07-24T21:32:28.831 INFO:teuthology.orchestra.run.smithi086.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-07-24T21:32:28.834 INFO:teuthology.orchestra.run.smithi060.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-07-24T21:32:28.848 INFO:teuthology.orchestra.run.smithi086.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 289 kB/s | 19 kB 00:00 2024-07-24T21:32:28.860 INFO:teuthology.orchestra.run.smithi060.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-07-24T21:32:28.873 INFO:teuthology.orchestra.run.smithi086.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-07-24T21:32:28.885 INFO:teuthology.orchestra.run.smithi060.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-07-24T21:32:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 699 kB/s | 35 kB 00:00 2024-07-24T21:32:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 250 kB 00:00 2024-07-24T21:32:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 4.5 MB/s | 222 kB 00:00 2024-07-24T21:32:28.952 INFO:teuthology.orchestra.run.smithi060.stdout:(128/160): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-07-24T21:32:28.977 INFO:teuthology.orchestra.run.smithi060.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-24T21:32:29.003 INFO:teuthology.orchestra.run.smithi060.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-24T21:32:29.016 INFO:teuthology.orchestra.run.smithi086.stdout:(85/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-07-24T21:32:29.028 INFO:teuthology.orchestra.run.smithi060.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-24T21:32:29.041 INFO:teuthology.orchestra.run.smithi086.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 682 kB/s | 77 kB 00:00 2024-07-24T21:32:29.058 INFO:teuthology.orchestra.run.smithi086.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 398 kB/s | 42 kB 00:00 2024-07-24T21:32:29.061 INFO:teuthology.orchestra.run.smithi060.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-07-24T21:32:29.087 INFO:teuthology.orchestra.run.smithi060.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-07-24T21:32:29.112 INFO:teuthology.orchestra.run.smithi060.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-07-24T21:32:29.154 INFO:teuthology.orchestra.run.smithi060.stdout:(135/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-07-24T21:32:29.166 INFO:teuthology.orchestra.run.smithi086.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-07-24T21:32:29.179 INFO:teuthology.orchestra.run.smithi060.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-07-24T21:32:29.196 INFO:teuthology.orchestra.run.smithi060.stdout:(137/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-07-24T21:32:29.222 INFO:teuthology.orchestra.run.smithi060.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-07-24T21:32:29.247 INFO:teuthology.orchestra.run.smithi060.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-07-24T21:32:29.267 INFO:teuthology.orchestra.run.smithi086.stdout:(89/160): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-07-24T21:32:29.272 INFO:teuthology.orchestra.run.smithi060.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-07-24T21:32:29.297 INFO:teuthology.orchestra.run.smithi060.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-24T21:32:29.322 INFO:teuthology.orchestra.run.smithi060.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-07-24T21:32:29.334 INFO:teuthology.orchestra.run.smithi086.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.0 MB/s | 279 kB 00:00 2024-07-24T21:32:29.357 INFO:teuthology.orchestra.run.smithi060.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-07-24T21:32:29.359 INFO:teuthology.orchestra.run.smithi086.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 266 kB/s | 51 kB 00:00 2024-07-24T21:32:29.382 INFO:teuthology.orchestra.run.smithi060.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-07-24T21:32:29.393 INFO:teuthology.orchestra.run.smithi086.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-07-24T21:32:29.407 INFO:teuthology.orchestra.run.smithi060.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-07-24T21:32:29.433 INFO:teuthology.orchestra.run.smithi060.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-24T21:32:29.466 INFO:teuthology.orchestra.run.smithi060.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-07-24T21:32:29.516 INFO:teuthology.orchestra.run.smithi060.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-07-24T21:32:29.600 INFO:teuthology.orchestra.run.smithi060.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-07-24T21:32:29.601 INFO:teuthology.orchestra.run.smithi086.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 200 kB/s | 42 kB 00:00 2024-07-24T21:32:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 190 kB/s | 44 kB 00:00 2024-07-24T21:32:29.659 INFO:teuthology.orchestra.run.smithi060.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 31 MB/s | 1.8 MB 00:00 2024-07-24T21:32:29.684 INFO:teuthology.orchestra.run.smithi060.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 1.6 MB/s | 41 kB 00:00 2024-07-24T21:32:29.717 INFO:teuthology.orchestra.run.smithi060.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 7.2 MB/s | 247 kB 00:00 2024-07-24T21:32:29.751 INFO:teuthology.orchestra.run.smithi060.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-07-24T21:32:29.752 INFO:teuthology.orchestra.run.smithi086.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-07-24T21:32:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 242 kB/s | 36 kB 00:00 2024-07-24T21:32:29.785 INFO:teuthology.orchestra.run.smithi060.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 6.0 MB/s | 207 kB 00:00 2024-07-24T21:32:29.818 INFO:teuthology.orchestra.run.smithi060.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 7.7 MB/s | 260 kB 00:00 2024-07-24T21:32:29.869 INFO:teuthology.orchestra.run.smithi060.stdout:(156/160): librbd1-19.0.0-5197.g86385b1d.el9.x8 8.0 MB/s | 3.2 MB 00:00 2024-07-24T21:32:29.911 INFO:teuthology.orchestra.run.smithi060.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-07-24T21:32:29.961 INFO:teuthology.orchestra.run.smithi086.stdout:(97/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-07-24T21:32:29.961 INFO:teuthology.orchestra.run.smithi060.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 27 MB/s | 2.4 MB 00:00 2024-07-24T21:32:29.986 INFO:teuthology.orchestra.run.smithi086.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 203 kB/s | 47 kB 00:00 2024-07-24T21:32:29.994 INFO:teuthology.orchestra.run.smithi060.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-07-24T21:32:30.011 INFO:teuthology.orchestra.run.smithi086.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-07-24T21:32:30.038 INFO:teuthology.orchestra.run.smithi086.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-07-24T21:32:30.062 INFO:teuthology.orchestra.run.smithi086.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.3 MB/s | 96 kB 00:00 2024-07-24T21:32:30.087 INFO:teuthology.orchestra.run.smithi086.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-07-24T21:32:30.100 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-07-24T21:32:30.100 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-07-24T21:32:30.112 INFO:teuthology.orchestra.run.smithi086.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-07-24T21:32:30.137 INFO:teuthology.orchestra.run.smithi086.stdout:(104/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-07-24T21:32:30.171 INFO:teuthology.orchestra.run.smithi086.stdout:(105/160): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-07-24T21:32:30.261 INFO:teuthology.orchestra.run.smithi060.stdout:(160/160): librados2-19.0.0-5197.g86385b1d.el9. 4.1 MB/s | 3.4 MB 00:00 2024-07-24T21:32:30.270 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-07-24T21:32:30.270 INFO:teuthology.orchestra.run.smithi060.stdout:Total 19 MB/s | 240 MB 00:12 2024-07-24T21:32:30.288 INFO:teuthology.orchestra.run.smithi086.stdout:(106/160): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-07-24T21:32:30.313 INFO:teuthology.orchestra.run.smithi086.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-07-24T21:32:30.338 INFO:teuthology.orchestra.run.smithi086.stdout:(108/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-07-24T21:32:30.364 INFO:teuthology.orchestra.run.smithi086.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-07-24T21:32:30.389 INFO:teuthology.orchestra.run.smithi086.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-07-24T21:32:30.414 INFO:teuthology.orchestra.run.smithi086.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-07-24T21:32:30.441 INFO:teuthology.orchestra.run.smithi086.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-24T21:32:30.473 INFO:teuthology.orchestra.run.smithi086.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-07-24T21:32:30.498 INFO:teuthology.orchestra.run.smithi086.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-24T21:32:30.524 INFO:teuthology.orchestra.run.smithi086.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-24T21:32:30.549 INFO:teuthology.orchestra.run.smithi086.stdout:(116/160): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-07-24T21:32:30.574 INFO:teuthology.orchestra.run.smithi086.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-07-24T21:32:30.599 INFO:teuthology.orchestra.run.smithi086.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-07-24T21:32:30.625 INFO:teuthology.orchestra.run.smithi086.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-07-24T21:32:30.658 INFO:teuthology.orchestra.run.smithi086.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-07-24T21:32:30.684 INFO:teuthology.orchestra.run.smithi086.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 3.6 MB/s | 216 kB 00:00 2024-07-24T21:32:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-24T21:32:30.743 INFO:teuthology.orchestra.run.smithi086.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 317 kB/s | 11 kB 00:00 2024-07-24T21:32:30.768 INFO:teuthology.orchestra.run.smithi086.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-07-24T21:32:30.793 INFO:teuthology.orchestra.run.smithi086.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-07-24T21:32:30.860 INFO:teuthology.orchestra.run.smithi086.stdout:(126/160): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-07-24T21:32:30.886 INFO:teuthology.orchestra.run.smithi086.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-07-24T21:32:30.903 INFO:teuthology.orchestra.run.smithi086.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-07-24T21:32:30.929 INFO:teuthology.orchestra.run.smithi086.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-24T21:32:30.953 INFO:teuthology.orchestra.run.smithi086.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-24T21:32:30.970 INFO:teuthology.orchestra.run.smithi086.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-07-24T21:32:31.012 INFO:teuthology.orchestra.run.smithi086.stdout:(132/160): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-07-24T21:32:31.046 INFO:teuthology.orchestra.run.smithi086.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-07-24T21:32:31.071 INFO:teuthology.orchestra.run.smithi086.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-07-24T21:32:31.096 INFO:teuthology.orchestra.run.smithi086.stdout:(135/160): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-07-24T21:32:31.122 INFO:teuthology.orchestra.run.smithi086.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-07-24T21:32:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-07-24T21:32:31.172 INFO:teuthology.orchestra.run.smithi086.stdout:(138/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 713 kB/s | 90 kB 00:00 2024-07-24T21:32:31.197 INFO:teuthology.orchestra.run.smithi086.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-07-24T21:32:31.223 INFO:teuthology.orchestra.run.smithi086.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-07-24T21:32:31.248 INFO:teuthology.orchestra.run.smithi086.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-24T21:32:31.273 INFO:teuthology.orchestra.run.smithi086.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-24T21:32:31.298 INFO:teuthology.orchestra.run.smithi086.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-24T21:32:31.324 INFO:teuthology.orchestra.run.smithi086.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-24T21:32:31.349 INFO:teuthology.orchestra.run.smithi086.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-07-24T21:32:31.374 INFO:teuthology.orchestra.run.smithi086.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-24T21:32:31.399 INFO:teuthology.orchestra.run.smithi086.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-07-24T21:32:31.442 INFO:teuthology.orchestra.run.smithi086.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-07-24T21:32:31.508 INFO:teuthology.orchestra.run.smithi086.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 35 MB/s | 2.3 MB 00:00 2024-07-24T21:32:31.567 INFO:teuthology.orchestra.run.smithi086.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 31 MB/s | 1.8 MB 00:00 2024-07-24T21:32:31.581 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-07-24T21:32:31.600 INFO:teuthology.orchestra.run.smithi086.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 1.2 MB/s | 41 kB 00:00 2024-07-24T21:32:31.635 INFO:teuthology.orchestra.run.smithi086.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 7.2 MB/s | 247 kB 00:00 2024-07-24T21:32:31.668 INFO:teuthology.orchestra.run.smithi086.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-07-24T21:32:31.718 INFO:teuthology.orchestra.run.smithi086.stdout:(154/160): librados2-19.0.0-5197.g86385b1d.el9. 9.9 MB/s | 3.4 MB 00:00 2024-07-24T21:32:31.744 INFO:teuthology.orchestra.run.smithi086.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.7 MB/s | 207 kB 00:00 2024-07-24T21:32:31.777 INFO:teuthology.orchestra.run.smithi086.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 4.4 MB/s | 260 kB 00:00 2024-07-24T21:32:31.809 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-07-24T21:32:31.809 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-07-24T21:32:31.811 INFO:teuthology.orchestra.run.smithi086.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-07-24T21:32:31.861 INFO:teuthology.orchestra.run.smithi086.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-07-24T21:32:31.895 INFO:teuthology.orchestra.run.smithi086.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-07-24T21:32:32.070 INFO:teuthology.orchestra.run.smithi086.stdout:(160/160): librbd1-19.0.0-5197.g86385b1d.el9.x8 4.8 MB/s | 3.2 MB 00:00 2024-07-24T21:32:32.079 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-07-24T21:32:32.079 INFO:teuthology.orchestra.run.smithi086.stdout:Total 16 MB/s | 240 MB 00:14 2024-07-24T21:32:33.298 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-07-24T21:32:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-07-24T21:32:33.489 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-07-24T21:32:34.139 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-07-24T21:32:34.139 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-07-24T21:32:35.370 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-07-24T21:32:35.370 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-07-24T21:32:36.492 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-07-24T21:32:36.602 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-24T21:32:36.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-24T21:32:36.694 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-24T21:32:37.009 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-24T21:32:37.111 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:37.233 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:37.262 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:37.353 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:37.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:32:37.415 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/172 2024-07-24T21:32:37.441 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/172 2024-07-24T21:32:37.474 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/172 2024-07-24T21:32:37.567 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 11/172 2024-07-24T21:32:37.717 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 12/172 2024-07-24T21:32:37.766 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-24T21:32:37.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:38.009 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:38.196 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:38.292 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-24T21:32:38.646 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:38.698 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:38.797 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:38.858 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:39.052 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-24T21:32:39.196 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-24T21:32:39.230 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-24T21:32:39.312 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-24T21:32:39.356 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-24T21:32:39.388 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-24T21:32:39.432 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-24T21:32:39.464 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-24T21:32:39.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-24T21:32:39.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-24T21:32:39.614 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-24T21:32:39.654 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-24T21:32:39.722 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-24T21:32:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-24T21:32:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 35/172 2024-07-24T21:32:39.884 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 36/172 2024-07-24T21:32:39.913 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 37/172 2024-07-24T21:32:39.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-24T21:32:40.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-24T21:32:40.129 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-24T21:32:40.267 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-24T21:32:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-24T21:32:40.836 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-24T21:32:40.918 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-24T21:32:40.947 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-24T21:32:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-07-24T21:32:40.983 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-24T21:32:41.028 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-24T21:32:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-24T21:32:41.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/172 2024-07-24T21:32:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-24T21:32:41.111 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 49/172 2024-07-24T21:32:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-24T21:32:41.254 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-24T21:32:41.315 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-24T21:32:41.406 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-24T21:32:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-24T21:32:41.494 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-24T21:32:41.544 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-24T21:32:41.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-24T21:32:41.610 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:41.619 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-24T21:32:41.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-24T21:32:41.702 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-24T21:32:41.742 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:41.745 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-24T21:32:41.788 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-24T21:32:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:41.897 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-07-24T21:32:41.924 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-24T21:32:41.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-24T21:32:41.974 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-24T21:32:41.976 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:42.005 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-24T21:32:42.012 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:32:42.013 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-24T21:32:42.037 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/172 2024-07-24T21:32:42.065 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/172 2024-07-24T21:32:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-24T21:32:42.102 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-24T21:32:42.122 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/172 2024-07-24T21:32:42.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-24T21:32:42.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-24T21:32:42.204 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 11/172 2024-07-24T21:32:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-24T21:32:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-24T21:32:42.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-24T21:32:42.373 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 12/172 2024-07-24T21:32:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-24T21:32:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-24T21:32:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-24T21:32:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-24T21:32:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-24T21:32:42.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-24T21:32:42.532 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:42.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-24T21:32:42.578 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:42.602 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:42.646 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 5/172 2024-07-24T21:32:42.665 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:42.675 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:42.695 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:42.720 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:42.750 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:32:42.786 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:32:42.795 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-24T21:32:42.812 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/172 2024-07-24T21:32:42.830 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/172 2024-07-24T21:32:42.863 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/172 2024-07-24T21:32:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:42.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-24T21:32:42.946 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 11/172 2024-07-24T21:32:42.959 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-24T21:32:42.993 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-24T21:32:43.023 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:43.035 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-24T21:32:43.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-24T21:32:43.095 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 12/172 2024-07-24T21:32:43.138 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-24T21:32:43.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-24T21:32:43.166 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:43.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-24T21:32:43.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-24T21:32:43.256 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-24T21:32:43.273 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:32:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:43.342 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-24T21:32:43.365 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 15/172 2024-07-24T21:32:43.387 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:43.396 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:32:43.406 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-24T21:32:43.446 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:43.499 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:43.553 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:43.640 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-24T21:32:43.667 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-24T21:32:43.882 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:43.970 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:43.992 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-24T21:32:44.014 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-24T21:32:44.021 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-24T21:32:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-24T21:32:44.108 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-24T21:32:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-24T21:32:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-24T21:32:44.181 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-24T21:32:44.186 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-24T21:32:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-24T21:32:44.267 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-24T21:32:44.283 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-24T21:32:44.299 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-24T21:32:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-24T21:32:44.332 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-24T21:32:44.377 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-24T21:32:44.397 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-24T21:32:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-24T21:32:44.451 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-24T21:32:44.464 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-24T21:32:44.480 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-24T21:32:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-24T21:32:44.506 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-24T21:32:44.524 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-24T21:32:44.535 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-24T21:32:44.570 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-24T21:32:44.603 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-24T21:32:44.634 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 35/172 2024-07-24T21:32:44.644 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-24T21:32:44.679 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 36/172 2024-07-24T21:32:44.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-24T21:32:44.689 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-24T21:32:44.706 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 37/172 2024-07-24T21:32:44.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-24T21:32:44.729 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-24T21:32:44.773 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-24T21:32:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-24T21:32:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-24T21:32:44.805 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-24T21:32:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-24T21:32:44.833 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-24T21:32:44.903 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-24T21:32:44.909 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-24T21:32:44.956 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-24T21:32:44.986 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-24T21:32:45.021 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-24T21:32:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-24T21:32:45.097 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-24T21:32:45.118 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-24T21:32:45.132 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 35/172 2024-07-24T21:32:45.175 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 36/172 2024-07-24T21:32:45.212 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 37/172 2024-07-24T21:32:45.280 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-24T21:32:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-24T21:32:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-24T21:32:45.421 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-24T21:32:45.534 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-24T21:32:45.542 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-24T21:32:45.596 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-24T21:32:45.681 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-24T21:32:45.765 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-24T21:32:45.799 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-24T21:32:45.836 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-24T21:32:45.880 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-24T21:32:45.919 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/172 2024-07-24T21:32:45.965 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 49/172 2024-07-24T21:32:46.115 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-24T21:32:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-24T21:32:46.192 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-24T21:32:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-24T21:32:46.221 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-24T21:32:46.249 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-24T21:32:46.288 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-24T21:32:46.294 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-24T21:32:46.332 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/172 2024-07-24T21:32:46.350 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-24T21:32:46.378 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 49/172 2024-07-24T21:32:46.424 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-24T21:32:46.460 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-24T21:32:46.497 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-24T21:32:46.528 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-24T21:32:46.544 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-24T21:32:46.589 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-24T21:32:46.681 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-24T21:32:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-24T21:32:46.752 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-24T21:32:46.798 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-24T21:32:46.817 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-24T21:32:46.831 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-24T21:32:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-24T21:32:46.859 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-24T21:32:46.885 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-24T21:32:46.892 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-24T21:32:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-24T21:32:46.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-24T21:32:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-24T21:32:46.968 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-24T21:32:47.011 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-24T21:32:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-24T21:32:47.119 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-24T21:32:47.159 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-24T21:32:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-24T21:32:47.201 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-24T21:32:47.213 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-24T21:32:47.230 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-24T21:32:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:47.272 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-24T21:32:47.305 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-24T21:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-24T21:32:47.370 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-24T21:32:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 105/172 2024-07-24T21:32:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-24T21:32:47.405 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-24T21:32:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-24T21:32:47.421 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-24T21:32:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-24T21:32:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-24T21:32:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-24T21:32:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-24T21:32:47.523 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-24T21:32:47.569 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-24T21:32:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-24T21:32:47.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-24T21:32:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-24T21:32:47.726 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-24T21:32:47.728 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-24T21:32:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-24T21:32:47.761 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-24T21:32:47.790 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-24T21:32:47.800 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-24T21:32:47.829 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-24T21:32:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-24T21:32:47.861 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-24T21:32:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-24T21:32:47.891 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-24T21:32:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-24T21:32:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-24T21:32:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-24T21:32:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:47.979 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-24T21:32:48.011 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-24T21:32:48.013 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:48.013 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-24T21:32:48.043 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 109/172 2024-07-24T21:32:48.104 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-24T21:32:48.153 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-24T21:32:48.183 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-24T21:32:48.219 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-24T21:32:48.261 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-24T21:32:48.293 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-24T21:32:48.294 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-24T21:32:48.352 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:32:48.403 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-24T21:32:48.458 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-24T21:32:48.497 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-24T21:32:48.602 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-24T21:32:48.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:32:49.133 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-24T21:32:49.173 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-24T21:32:49.207 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-24T21:32:49.311 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-24T21:32:49.363 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-24T21:32:49.377 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-24T21:32:49.408 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-24T21:32:49.428 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-24T21:32:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-24T21:32:49.498 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-24T21:32:49.534 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-24T21:32:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-24T21:32:49.634 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-24T21:32:49.679 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-24T21:32:49.710 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-24T21:32:49.732 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-24T21:32:49.767 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-24T21:32:49.806 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-24T21:32:49.813 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-24T21:32:49.982 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-24T21:32:50.012 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-24T21:32:50.090 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-24T21:32:50.410 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:50.456 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:50.515 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-24T21:32:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-24T21:32:50.731 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-24T21:32:50.843 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-24T21:32:51.003 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:51.058 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:51.154 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:51.154 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-24T21:32:51.154 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-24T21:32:51.154 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:32:51.190 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:32:52.056 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-24T21:32:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-24T21:32:52.160 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-24T21:32:52.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-24T21:32:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-24T21:32:52.261 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-24T21:32:52.289 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-24T21:32:52.322 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-24T21:32:52.512 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-24T21:32:52.556 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:52.668 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-24T21:32:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 105/172 2024-07-24T21:32:52.702 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-24T21:32:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 104/172 2024-07-24T21:32:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 105/172 2024-07-24T21:32:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-24T21:32:53.194 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-24T21:32:53.226 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:53.280 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:53.311 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 109/172 2024-07-24T21:32:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-24T21:32:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 108/172 2024-07-24T21:32:53.523 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 109/172 2024-07-24T21:32:55.561 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:55.607 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:56.157 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:56.199 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:56.285 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:56.285 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-24T21:32:56.286 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-24T21:32:56.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:32:56.317 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:32:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 110/172 2024-07-24T21:32:56.740 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 111/172 2024-07-24T21:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-24T21:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-24T21:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-07-24T21:33:02.693 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-07-24T21:33:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:02.868 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:02.884 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:02.884 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:02.884 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-24T21:33:02.884 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:04.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:04.423 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:04.670 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:04.670 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:04.670 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-24T21:33:04.670 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:04.670 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:04.671 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:05.107 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:05.130 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:05.130 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:05.130 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-24T21:33:05.130 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:05.130 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:05.131 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:05.237 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-24T21:33:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-24T21:33:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-24T21:33:05.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-24T21:33:05.591 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-24T21:33:05.668 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-24T21:33:05.756 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-24T21:33:05.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-24T21:33:05.940 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-24T21:33:05.973 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-24T21:33:06.000 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-24T21:33:06.406 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:06.432 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:06.464 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 129/172 2024-07-24T21:33:06.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 130/172 2024-07-24T21:33:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-07-24T21:33:07.813 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:07.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:07.994 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:07.994 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:07.994 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-24T21:33:07.994 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:09.294 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 112/172 2024-07-24T21:33:09.294 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-07-24T21:33:09.294 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-07-24T21:33:09.294 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-07-24T21:33:09.294 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-07-24T21:33:09.295 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-07-24T21:33:09.295 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-07-24T21:33:09.295 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-07-24T21:33:09.295 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:09.474 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:09.481 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 113/172 2024-07-24T21:33:09.491 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:09.491 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-24T21:33:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:09.508 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:09.704 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:09.750 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:09.760 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:09.772 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:09.772 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:09.772 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-24T21:33:09.773 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:09.773 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:09.773 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:10.209 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:10.232 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:10.366 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-24T21:33:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-24T21:33:10.484 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-24T21:33:10.669 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-24T21:33:10.718 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-24T21:33:10.720 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:10.752 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-24T21:33:10.864 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:10.884 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-24T21:33:11.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 134/172 2024-07-24T21:33:11.031 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-24T21:33:11.044 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:11.054 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:11.067 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-24T21:33:11.072 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 114/172 2024-07-24T21:33:11.072 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:11.073 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-24T21:33:11.073 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:11.073 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-24T21:33:11.073 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:11.077 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:11.077 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:11.077 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-24T21:33:11.077 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:11.077 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:11.078 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:11.108 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-24T21:33:11.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:11.132 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:11.135 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-24T21:33:11.232 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 137/172 2024-07-24T21:33:11.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 138/172 2024-07-24T21:33:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 115/172 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-24T21:33:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:11.526 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:11.550 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:11.583 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 129/172 2024-07-24T21:33:11.646 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 130/172 2024-07-24T21:33:11.682 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:11.796 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:11.821 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 116/172 2024-07-24T21:33:11.822 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:11.822 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-24T21:33:11.822 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:11.822 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-24T21:33:11.822 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-24T21:33:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-24T21:33:12.058 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-24T21:33:12.230 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-24T21:33:12.254 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-24T21:33:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:12.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:12.307 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-24T21:33:12.341 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:12.363 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:12.363 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-24T21:33:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:12.387 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-24T21:33:12.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:12.431 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-24T21:33:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:12.432 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:12.432 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:12.486 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-24T21:33:12.636 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-24T21:33:12.681 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-24T21:33:12.713 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-24T21:33:12.740 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-24T21:33:12.743 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:12.766 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:13.181 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 128/172 2024-07-24T21:33:13.213 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 129/172 2024-07-24T21:33:13.280 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 130/172 2024-07-24T21:33:13.303 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:15.882 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:15.904 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:16.016 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 134/172 2024-07-24T21:33:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:16.229 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:16.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:16.284 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:16.383 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 137/172 2024-07-24T21:33:16.429 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 138/172 2024-07-24T21:33:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 131/172 2024-07-24T21:33:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:17.392 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-24T21:33:17.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:17.446 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 144/172 2024-07-24T21:33:17.448 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:17.448 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:17.448 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-24T21:33:17.449 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:17.449 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:17.449 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:17.487 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 145/172 2024-07-24T21:33:17.493 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:17.515 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:17.515 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:17.515 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-24T21:33:17.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:17.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 146/172 2024-07-24T21:33:17.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-24T21:33:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:17.583 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:17.583 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-24T21:33:17.583 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:17.583 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:17.587 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-24T21:33:17.733 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 132/172 2024-07-24T21:33:17.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-24T21:33:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 151/172 2024-07-24T21:33:17.860 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:17.890 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-24T21:33:17.894 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:17.919 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-24T21:33:17.982 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 133/172 2024-07-24T21:33:18.156 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 134/172 2024-07-24T21:33:18.187 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 135/172 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-24T21:33:18.213 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:18.263 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:18.275 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 136/172 2024-07-24T21:33:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 137/172 2024-07-24T21:33:18.433 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 138/172 2024-07-24T21:33:18.574 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:18.617 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:18.636 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:19.441 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-24T21:33:19.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 156/172 2024-07-24T21:33:19.473 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:19.498 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 140/172 2024-07-24T21:33:19.498 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:19.499 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-24T21:33:19.499 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:19.499 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-24T21:33:19.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:19.541 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:19.566 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5197.g86385 141/172 2024-07-24T21:33:19.566 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:19.566 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-24T21:33:19.566 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:19.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-24T21:33:19.617 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:19.622 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-24T21:33:19.641 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 142/172 2024-07-24T21:33:19.642 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:19.642 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-24T21:33:19.642 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:19.642 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-24T21:33:19.642 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:19.704 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-24T21:33:19.743 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-24T21:33:19.769 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-24T21:33:19.794 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:19.820 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-24T21:33:19.820 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:19.904 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:19.937 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-24T21:33:19.937 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 143/172 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-24T21:33:20.001 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:20.022 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:20.047 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-24T21:33:20.052 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:20.052 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:20.241 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:20.241 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:20.317 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:20.342 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:20.368 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-24T21:33:20.440 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 144/172 2024-07-24T21:33:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 145/172 2024-07-24T21:33:22.704 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 146/172 2024-07-24T21:33:22.736 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-24T21:33:22.764 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-24T21:33:22.945 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:22.992 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-24T21:33:23.024 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 151/172 2024-07-24T21:33:23.064 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-24T21:33:23.094 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:23.135 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-24T21:33:23.757 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:23.792 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:23.812 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:24.634 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 156/172 2024-07-24T21:33:24.747 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-24T21:33:24.805 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-24T21:33:24.885 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-24T21:33:24.925 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-24T21:33:24.950 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-24T21:33:24.976 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:25.003 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-24T21:33:25.003 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:25.013 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 144/172 2024-07-24T21:33:25.055 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 145/172 2024-07-24T21:33:25.088 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 146/172 2024-07-24T21:33:25.120 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-24T21:33:25.121 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-24T21:33:25.171 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-24T21:33:25.205 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:25.230 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-24T21:33:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:25.370 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:25.417 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-24T21:33:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:25.450 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 151/172 2024-07-24T21:33:25.491 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-24T21:33:25.508 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:25.532 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:25.557 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-24T21:33:25.558 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-24T21:33:25.623 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:26.204 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:26.247 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:26.269 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 1/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 3/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 4/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5197.g86385 5/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 8/172 2024-07-24T21:33:26.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 9/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 10/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 11/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 12/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 13/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 15/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 16/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 17/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 18/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 19/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 20/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 21/172 2024-07-24T21:33:26.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 22/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 23/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 24/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 25/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 26/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 27/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 28/172 2024-07-24T21:33:26.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 29/172 2024-07-24T21:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 30/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 31/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 32/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 33/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 34/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 35/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 36/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 37/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 38/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 39/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-24T21:33:26.296 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-24T21:33:26.297 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-24T21:33:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-24T21:33:26.299 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-24T21:33:26.300 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-24T21:33:26.301 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-24T21:33:26.302 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-24T21:33:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-24T21:33:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-24T21:33:26.305 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 151/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-24T21:33:26.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:26.307 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-24T21:33:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 156/172 2024-07-24T21:33:27.285 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-24T21:33:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-24T21:33:27.426 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-24T21:33:27.463 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-24T21:33:27.488 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-24T21:33:27.514 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:27.540 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-24T21:33:27.540 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:27.625 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-24T21:33:27.666 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-24T21:33:27.666 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:27.767 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-24T21:33:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-24T21:33:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:27.986 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-24T21:33:27.986 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:28.062 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-24T21:33:28.094 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:28.129 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-24T21:33:28.215 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:28.969 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:28.969 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:28.969 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-07-24T21:33:28.969 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-24T21:33:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:28.971 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-24T21:33:28.972 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.973 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.973 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.973 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.973 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.973 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.974 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.974 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.974 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.974 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.974 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.975 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.976 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.976 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.976 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.976 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.976 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.977 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.977 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:28.977 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-24T21:33:28.977 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-07-24T21:33:28.978 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-24T21:33:28.978 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-24T21:33:28.978 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-24T21:33:28.978 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-24T21:33:28.978 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.979 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-24T21:33:28.980 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-24T21:33:28.981 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-24T21:33:28.982 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-24T21:33:28.982 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-24T21:33:28.982 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-07-24T21:33:28.982 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-24T21:33:28.982 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-24T21:33:28.983 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-24T21:33:28.983 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-24T21:33:28.983 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:28.983 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-24T21:33:28.983 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-24T21:33:28.984 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.985 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-24T21:33:28.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-24T21:33:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-24T21:33:28.988 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-24T21:33:28.989 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-24T21:33:28.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-24T21:33:28.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-07-24T21:33:28.992 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:33:28.993 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-07-24T21:33:29.391 DEBUG:teuthology.parallel:result is None 2024-07-24T21:33:31.522 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:31.522 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 1/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 3/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 4/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5197.g86385 5/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 8/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 9/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 10/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 11/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 12/172 2024-07-24T21:33:31.523 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 13/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 15/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 16/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 17/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 18/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 19/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 20/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 21/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 22/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 23/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 24/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 25/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 26/172 2024-07-24T21:33:31.524 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 27/172 2024-07-24T21:33:31.525 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 28/172 2024-07-24T21:33:31.525 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 29/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 30/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 31/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 32/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 33/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 34/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 35/172 2024-07-24T21:33:31.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 36/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 37/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 38/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 39/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-24T21:33:31.538 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-24T21:33:31.539 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-24T21:33:31.540 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-24T21:33:31.541 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-24T21:33:31.542 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-24T21:33:31.543 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-24T21:33:31.544 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-24T21:33:31.545 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 151/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-24T21:33:31.546 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-24T21:33:31.547 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-24T21:33:31.548 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-24T21:33:31.548 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-24T21:33:31.548 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-24T21:33:31.548 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:31.548 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:34.214 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-07-24T21:33:34.215 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.216 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:34.217 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.218 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-24T21:33:34.219 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-24T21:33:34.220 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-24T21:33:34.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-24T21:33:34.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:34.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-24T21:33:34.224 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-24T21:33:34.225 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-24T21:33:34.226 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-24T21:33:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-24T21:33:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-24T21:33:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-24T21:33:34.227 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-07-24T21:33:34.227 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-07-24T21:33:34.641 DEBUG:teuthology.parallel:result is None 2024-07-24T21:33:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 1/172 2024-07-24T21:33:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2/172 2024-07-24T21:33:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 3/172 2024-07-24T21:33:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 4/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5197.g86385 5/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 6/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 7/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 8/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 9/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 10/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 11/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 12/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 13/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 14/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_ 15/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 16/172 2024-07-24T21:33:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 17/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_6 18/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86 19/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 20/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el 21/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.0.0-5197.g86385b1d.el9. 22/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_6 23/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 24/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 25/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 26/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 27/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 28/172 2024-07-24T21:33:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 29/172 2024-07-24T21:33:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d. 30/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noa 31/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.n 32/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5197.g863 33/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el 34/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 35/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.e 36/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 37/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 38/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 39/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-24T21:33:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-24T21:33:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-24T21:33:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-24T21:33:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-24T21:33:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-24T21:33:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-24T21:33:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-24T21:33:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-24T21:33:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 149/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 151/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-24T21:33:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-24T21:33:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-24T21:33:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-24T21:33:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-24T21:33:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-24T21:33:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-24T21:33:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-24T21:33:37.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-24T21:33:37.626 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:37.626 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-07-24T21:33:37.626 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.627 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.628 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.629 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.0.0-5197.g86385b1d.el9.noarch 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-24T21:33:37.630 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-24T21:33:37.631 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-24T21:33:37.632 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.14-1.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-24T21:33:37.633 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-24T21:33:37.634 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-24T21:33:37.635 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-24T21:33:37.636 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-24T21:33:37.637 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-24T21:33:37.638 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.0.0-5197.g86385b1d.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-24T21:33:37.639 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-24T21:33:37.640 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-24T21:33:37.640 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-24T21:33:37.640 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:33:37.640 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-07-24T21:33:38.052 DEBUG:teuthology.parallel:result is None 2024-07-24T21:33:38.052 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:38.199 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-24T21:33:38.221 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.221 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.222 INFO:teuthology.task.install:The correct ceph version 19.0.0-5197.g86385b1d is installed. 2024-07-24T21:33:38.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:38.365 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-24T21:33:38.391 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.391 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.391 INFO:teuthology.task.install:The correct ceph version 19.0.0-5197.g86385b1d is installed. 2024-07-24T21:33:38.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:38.540 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-24T21:33:38.565 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.565 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5197.g86385b1d.el9 2024-07-24T21:33:38.565 INFO:teuthology.task.install:The correct ceph version 19.0.0-5197.g86385b1d is installed. 2024-07-24T21:33:38.567 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-24T21:33:38.567 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:33:38.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-24T21:33:38.597 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:33:38.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-24T21:33:38.627 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:33:38.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-24T21:33:38.657 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-24T21:33:38.657 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:33:38.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-07-24T21:33:38.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-24T21:33:38.751 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:33:38.751 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-07-24T21:33:38.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-24T21:33:38.846 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:33:38.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-07-24T21:33:38.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-24T21:33:38.939 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-24T21:33:38.940 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:33:38.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-24T21:33:38.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-24T21:33:39.033 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:33:39.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-24T21:33:39.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-24T21:33:39.127 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:33:39.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-24T21:33:39.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-24T21:33:39.219 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-24T21:33:39.220 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:33:39.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-07-24T21:33:39.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-24T21:33:39.312 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:33:39.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-07-24T21:33:39.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-24T21:33:39.403 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:33:39.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-07-24T21:33:39.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-24T21:33:39.494 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-24T21:33:39.612 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2', 'subvols': {'create': 2}} 2024-07-24T21:33:39.612 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:39.613 INFO:tasks.cephadm:Cluster fsid is 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:33:39.613 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-24T21:33:39.613 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.b': '172.21.15.86', 'mon.c': '172.21.15.123'} 2024-07-24T21:33:39.613 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-07-24T21:33:39.613 INFO:tasks.cephadm:First mgr is x 2024-07-24T21:33:39.613 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-24T21:33:39.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-07-24T21:33:39.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-07-24T21:33:39.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-07-24T21:33:39.693 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-24T21:33:39.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:39.833 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '86385b1daf5235400bbcd34a02515d168bde88e2', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55672/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5197-g86385b1d', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5197.g86385b1d'}, 'url': 'https://4.chacra.ceph.com/r/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-22 21:35:01.055497', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-07-24T21:33:39.971 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref main, sha1 86385b1daf5235400bbcd34a02515d168bde88e2 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:33:39.973 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/x86_64/flavors/default/cephadm 2024-07-24T21:33:39.973 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/x86_64/flavors/default/cephadm 2024-07-24T21:33:39.973 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:40.348 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 789815 Jul 24 21:33 /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:40.349 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:40.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 789815 Jul 24 21:33 /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:40.723 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/main/86385b1daf5235400bbcd34a02515d168bde88e2/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:41.113 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 789815 Jul 24 21:33 /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:41.114 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:41.135 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:41.153 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-24T21:33:41.187 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 on all hosts... 2024-07-24T21:33:41.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 pull 2024-07-24T21:33:41.190 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 pull 2024-07-24T21:33:41.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 pull 2024-07-24T21:33:42.096 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2... 2024-07-24T21:33:42.162 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2... 2024-07-24T21:33:42.180 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2... 2024-07-24T21:34:11.556 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:34:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev)", 2024-07-24T21:34:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0", 2024-07-24T21:34:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-07-24T21:34:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb" 2024-07-24T21:34:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T21:34:11.557 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev)", 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0", 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb" 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-07-24T21:34:11.591 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev)", 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0", 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb" 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-07-24T21:34:15.664 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-07-24T21:34:15.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-07-24T21:34:15.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-07-24T21:34:15.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-07-24T21:34:15.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-07-24T21:34:15.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-07-24T21:34:15.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-07-24T21:34:15.861 INFO:tasks.cephadm:Writing seed config... 2024-07-24T21:34:15.862 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-07-24T21:34:15.863 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-24T21:34:15.864 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-24T21:34:15.865 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:34:15.865 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-24T21:34:15.884 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 646042b4-4a04-11ef-bcad-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-07-24T21:34:15.884 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service 2024-07-24T21:34:15.926 DEBUG:teuthology.orchestra.run.smithi060:mgr.x> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x.service 2024-07-24T21:34:15.969 INFO:tasks.cephadm:Bootstrapping... 2024-07-24T21:34:15.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 -v bootstrap --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-24T21:34:16.189 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-07-24T21:34:16.189 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', '-v', 'bootstrap', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-07-24T21:34:16.189 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-24T21:34:16.189 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-07-24T21:34:16.216 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.1.2 2024-07-24T21:34:16.216 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-07-24T21:34:16.217 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-07-24T21:34:16.224 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-24T21:34:16.224 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-24T21:34:16.231 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-24T21:34:16.231 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:34:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-07-24T21:34:16.244 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-07-24T21:34:16.244 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-07-24T21:34:16.244 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-07-24T21:34:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.1.2 2024-07-24T21:34:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 5.1.2 is present 2024-07-24T21:34:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-07-24T21:34:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-07-24T21:34:16.275 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-24T21:34:16.275 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-24T21:34:16.281 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-24T21:34:16.281 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:34:16.288 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-07-24T21:34:16.295 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-07-24T21:34:16.295 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-07-24T21:34:16.295 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-07-24T21:34:16.296 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:16.296 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 140331014710080 on /run/cephadm/646042b4-4a04-11ef-bcad-c7b262605968.lock 2024-07-24T21:34:16.296 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 140331014710080 acquired on /run/cephadm/646042b4-4a04-11ef-bcad-c7b262605968.lock 2024-07-24T21:34:16.296 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-07-24T21:34:16.296 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-07-24T21:34:16.297 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-07-24T21:34:16.299 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-07-24T21:34:16.299 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-07-24T21:34:16.302 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-24T21:34:16.302 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-07-24T21:34:16.302 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-07-24T21:34:16.304 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-24T21:34:16.304 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-24T21:34:16.304 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-24T21:34:16.304 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-07-24T21:34:16.305 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-07-24T21:34:16.305 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-24T21:34:16.305 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-07-24T21:34:16.305 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-07-24T21:34:16.306 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-24T21:34:16.306 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-24T21:34:16.306 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2... 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2... 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:ff6978fe3241616ea65cf2dbfa33f964b4d58d6a8a22dca92f46bf44c1c2c5c1 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:a0a261c93a1a3b1a5634697d0569ba314563628d3546bf9326abaa78605cead1 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 2024-07-24T21:34:16.800 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-24T21:34:17.889 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:34:17.889 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:34:17.889 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-07-24T21:34:18.728 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-07-24T21:34:18.728 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-07-24T21:34:19.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQBbc6FmLaafABAAkdOi+hq9AeNZGHCVJhARFg== 2024-07-24T21:34:20.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQBbc6FmjwixOhAAxzEJ4bb/ZuI59NZ/+i4R9g== 2024-07-24T21:34:28.694 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQBec6FmS8mfDxAABHJXpbwtLIy4i7JBZgckfQ== 2024-07-24T21:34:28.695 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-07-24T21:34:33.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-24T21:34:33.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:34:33.642 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-07-24T21:34:35.180 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-07-24T21:34:35.401 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-24T21:34:35.548 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-24T21:34:35.779 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target → /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target. 2024-07-24T21:34:35.780 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target → /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target. 2024-07-24T21:34:36.007 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a 2024-07-24T21:34:36.007 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service: Unit ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service not loaded. 2024-07-24T21:34:36.180 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target.wants/ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service → /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968@.service. 2024-07-24T21:34:36.739 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-24T21:34:36.739 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-07-24T21:34:36.746 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-24T21:34:36.746 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:34:36.746 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-07-24T21:34:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-07-24T21:34:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-07-24T21:34:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-07-24T21:34:37.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:37 smithi060 ceph-mon[32632]: from='client.? 172.21.15.60:0/2666175492' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-24T21:34:37.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-07-24T21:34:37.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:37.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-24T21:34:37.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:37.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-07-24T21:34:37.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.562135s) 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-07-24T21:34:37.780 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-07-24T21:34:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:38 smithi060 ceph-mon[32632]: from='client.? 172.21.15.60:0/2520261388' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-24T21:34:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:38 smithi060 ceph-mon[32632]: from='client.? 172.21.15.60:0/2520261388' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-24T21:34:40.162 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-24T21:34:40.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-24T21:34:40.164 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-07-24T21:34:40.928 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:40 smithi060 ceph-mon[32632]: from='client.? 172.21.15.60:0/2562529718' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:34:41.140 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-07-24T21:34:41.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 systemd[1]: Stopping Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:34:41.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a[32628]: 2024-07-24T21:34:41.405+0000 7f305e6c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T21:34:41.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a[32628]: 2024-07-24T21:34:41.405+0000 7f305e6c8640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-24T21:34:41.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 podman[32876]: 2024-07-24 21:34:41.590734842 +0000 UTC m=+0.282319667 container died 7e96c14e650eb315349c0a3a5d23db8032b9df48083f107c1f98464a0ff37e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-24T21:34:42.198 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 podman[32876]: 2024-07-24 21:34:41.972331736 +0000 UTC m=+0.663916562 container cleanup 7e96c14e650eb315349c0a3a5d23db8032b9df48083f107c1f98464a0ff37e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T21:34:42.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:41 smithi060 bash[32876]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a 2024-07-24T21:34:42.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 podman[32889]: 2024-07-24 21:34:42.198134069 +0000 UTC m=+0.598248252 container remove 7e96c14e650eb315349c0a3a5d23db8032b9df48083f107c1f98464a0ff37e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, RELEASE=main-86385b1, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-24T21:34:42.525 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-24T21:34:42.525 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 systemd[1]: Stopped Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:34:42.525 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 systemd[1]: Starting Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:34:42.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 podman[32981]: 2024-07-24 21:34:42.52422876 +0000 UTC m=+0.091113375 container create fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 podman[32981]: 2024-07-24 21:34:42.45067235 +0000 UTC m=+0.017556980 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 podman[32981]: 2024-07-24 21:34:42.710272776 +0000 UTC m=+0.277157401 container init fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240716) 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 podman[32981]: 2024-07-24 21:34:42.713624921 +0000 UTC m=+0.280509537 container start fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: set uid:gid to 167:167 (ceph:ceph) 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev), process ceph-mon, pid 2 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: pidfile_write: ignore empty --pid-file 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: load: jerasure load: lrc 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: RocksDB version: 7.9.2 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Git sha 0 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Compile date 2024-07-22 20:32:46 2024-07-24T21:34:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: DB SUMMARY 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: DB Session ID: L85LNTSJRKYL57PKBSYC 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: CURRENT file: CURRENT 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: IDENTITY file: IDENTITY 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95329 ; 2024-07-24T21:34:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.error_if_exists: 0 2024-07-24T21:34:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.create_if_missing: 0 2024-07-24T21:34:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.paranoid_checks: 1 2024-07-24T21:34:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-24T21:34:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.env: 0x5560f8e23840 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.fs: PosixFileSystem 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.info_log: 0x5560f9e6f4c0 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_file_opening_threads: 16 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.statistics: (nil) 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.use_fsync: 0 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_log_file_size: 0 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.keep_log_file_num: 1000 2024-07-24T21:34:42.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.recycle_log_file_num: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_fallocate: 1 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_mmap_reads: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_mmap_writes: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.use_direct_reads: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.create_missing_column_families: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.db_log_dir: 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.wal_dir: 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-24T21:34:42.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.advise_random_on_open: 1 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.db_write_buffer_size: 0 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.write_buffer_manager: 0x5560f9e72e60 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.rate_limiter: (nil) 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.wal_recovery_mode: 2 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enable_thread_tracking: 0 2024-07-24T21:34:42.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enable_pipelined_write: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.unordered_write: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.row_cache: None 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.wal_filter: None 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_ingest_behind: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.two_write_queues: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.manual_wal_flush: 0 2024-07-24T21:34:42.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.wal_compression: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.atomic_flush: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.log_readahead_size: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.best_efforts_recovery: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.allow_data_in_errors: 0 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.db_host_id: __hostname__ 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-24T21:34:42.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_background_jobs: 2 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_background_compactions: -1 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_subcompactions: 1 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_total_wal_size: 0 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_open_files: -1 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bytes_per_sync: 0 2024-07-24T21:34:42.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_readahead_size: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_background_flushes: -1 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Compression algorithms supported: 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kZSTD supported: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kXpressCompression supported: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kBZip2Compression supported: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kLZ4Compression supported: 1 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kZlibCompression supported: 1 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kLZ4HCCompression supported: 1 2024-07-24T21:34:42.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: kSnappyCompression supported: 1 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.merge_operator: 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_filter: None 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_filter_factory: None 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.sst_partitioner_factory: None 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5560f9e6f0e0) 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-07-24T21:34:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x5560f9e63610 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-07-24T21:34:42.797 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-07-24T21:34:42.798 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.write_buffer_size: 33554432 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_write_buffer_number: 2 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression: NoCompression 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression: Disabled 2024-07-24T21:34:42.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.prefix_extractor: nullptr 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.num_levels: 7 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-24T21:34:42.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.level: 32767 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.strategy: 0 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.enabled: false 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-24T21:34:42.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.target_file_size_base: 67108864 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-24T21:34:42.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.arena_block_size: 1048576 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.disable_auto_compactions: 0 2024-07-24T21:34:42.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.inplace_update_support: 0 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-24T21:34:42.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.bloom_locality: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.max_successive_merges: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.paranoid_file_checks: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.force_consistency_checks: 1 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.report_bg_io_stats: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.ttl: 2592000 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enable_blob_files: false 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.min_blob_size: 0 2024-07-24T21:34:42.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_file_size: 268435456 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.blob_file_starting_level: 0 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 998c8f22-77cc-435e-b98b-8d0578f04fa7 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856882741218, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-24T21:34:42.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-24T21:34:42.807 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-24T21:34:42.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856882742186, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91450, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 284, "table_properties": {"data_size": 89491, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11722, "raw_average_key_size": 48, "raw_value_size": 82909, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 243, "num_filter_entries": 243, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1721856882, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "998c8f22-77cc-435e-b98b-8d0578f04fa7", "db_session_id": "L85LNTSJRKYL57PKBSYC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-24T21:34:42.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856882742285, "job": 1, "event": "recovery_finished"} 2024-07-24T21:34:42.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-24T21:34:42.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5560f9e90e00 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: DB pointer 0x5560f9f88000 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-24T21:34:42.808 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 91.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 91.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-24T21:34:42.809 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 27.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 27.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x5560f9e63610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-07-24T21:34:42.810 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???) e1 preinit fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).mds e1 new map 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).mds e1 print_map 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: btime 2024-07-24T21:34:36:682639+0000 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-07-24T21:34:42.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 bash[32981]: fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d 2024-07-24T21:34:42.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 systemd[1]: Started Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:34:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-24T21:34:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: monmap epoch 1 2024-07-24T21:34:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: last_changed 2024-07-24T21:34:31.410206+0000 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: min_mon_release 19 (squid) 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: election_strategy: 1 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: fsmap 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: osdmap e1: 0 total, 0 up, 0 in 2024-07-24T21:34:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:42 smithi060 ceph-mon[33000]: mgrmap e1: no daemons active 2024-07-24T21:34:43.771 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-24T21:34:43.791 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-24T21:34:43.791 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-07-24T21:34:43.791 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-24T21:34:43.792 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-24T21:34:44.078 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x 2024-07-24T21:34:44.079 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x.service: Unit ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x.service not loaded. 2024-07-24T21:34:44.237 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968.target.wants/ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-646042b4-4a04-11ef-bcad-c7b262605968@.service. 2024-07-24T21:34:44.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:44 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3330979277' entity='client.admin' 2024-07-24T21:34:44.476 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 systemd[1]: Starting Ceph mgr.x for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:34:44.771 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 podman[33211]: 2024-07-24 21:34:44.377619934 +0000 UTC m=+0.017657776 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 2024-07-24T21:34:44.771 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 podman[33211]: 2024-07-24 21:34:44.476622559 +0000 UTC m=+0.116660381 container create 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.build-date=20240716, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-07-24T21:34:44.771 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 podman[33211]: 2024-07-24 21:34:44.687525061 +0000 UTC m=+0.327562895 container init 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, RELEASE=main-86385b1, maintainer=Guillaume Abrioux ) 2024-07-24T21:34:44.771 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 podman[33211]: 2024-07-24 21:34:44.691157826 +0000 UTC m=+0.331195654 container start 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-24T21:34:44.771 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 bash[33211]: 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 2024-07-24T21:34:44.788 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-24T21:34:44.788 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-07-24T21:34:44.797 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-24T21:34:44.797 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:34:44.798 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-07-24T21:34:44.798 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-07-24T21:34:44.808 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-24T21:34:44.809 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-07-24T21:34:44.818 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-07-24T21:34:45.055 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:44 smithi060 systemd[1]: Started Ceph mgr.x for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:34:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4247447477' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-24T21:34:45.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-24T21:34:45.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-24T21:34:45.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-24T21:34:45.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-24T21:34:45.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-24T21:34:45.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-24T21:34:45.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-07-24T21:34:36:682639+0000", 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-24T21:34:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-07-24T21:34:36.683288+0000", 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:34:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-07-24T21:34:48.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:48 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1667134546' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-24T21:34:49.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-24T21:34:49.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-24T21:34:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:49.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-07-24T21:34:36:682639+0000", 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:49.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-07-24T21:34:36.683288+0000", 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:34:49.099 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-07-24T21:34:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: Activating manager daemon x 2024-07-24T21:34:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: mgrmap e2: x(active, starting, since 0.00338313s) 2024-07-24T21:34:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: Manager daemon x is now available 2024-07-24T21:34:50.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:34:50.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:34:50.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' 2024-07-24T21:34:50.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' 2024-07-24T21:34:50.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:50 smithi060 ceph-mon[33000]: from='mgr.14100 172.21.15.60:0/4010844621' entity='mgr.x' 2024-07-24T21:34:52.341 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:52.341 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:34:52.341 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:34:52.341 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-07-24T21:34:52.342 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-24T21:34:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-07-24T21:34:36:682639+0000", 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-24T21:34:52.345 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-07-24T21:34:36.683288+0000", 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:34:52.346 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-07-24T21:34:52.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:52 smithi060 ceph-mon[33000]: mgrmap e3: x(active, since 1.00793s) 2024-07-24T21:34:52.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:52 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/758251800' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-24T21:34:53.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:53 smithi060 ceph-mon[33000]: mgrmap e4: x(active, since 2s) 2024-07-24T21:34:53.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:53 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/239298698' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-24T21:34:53.520 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-24T21:34:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-07-24T21:34:54.538 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setuser ceph since I am not root 2024-07-24T21:34:54.538 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:34:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setgroup ceph since I am not root 2024-07-24T21:34:54.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:54 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1618936092' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-24T21:34:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:55 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1618936092' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-24T21:34:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:55 smithi060 ceph-mon[33000]: mgrmap e5: x(active, since 4s) 2024-07-24T21:34:56.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:34:56.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-24T21:34:56.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-24T21:34:56.199 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-24T21:34:56.199 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-24T21:34:56.199 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:34:56.199 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-07-24T21:34:56.199 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-07-24T21:34:56.461 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2545428381' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-24T21:35:00.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: Active manager daemon x restarted 2024-07-24T21:35:00.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: Activating manager daemon x 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: osdmap e2: 0 total, 0 up, 0 in 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: mgrmap e6: x(active, starting, since 0.00324883s) 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: Manager daemon x is now available 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:35:00.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:34:59 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:35:00.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:00 smithi060 ceph-mon[33000]: Found migration_current of "None". Setting to last migration. 2024-07-24T21:35:00.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:00 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:00.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:00 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:00.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:00 smithi060 ceph-mon[33000]: mgrmap e7: x(active, since 1.00553s) 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-07-24T21:35:01.155 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-07-24T21:35:01.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-mon[33000]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-24T21:35:01.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-mon[33000]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-24T21:35:02.071 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:35:01.990+0000 7ff6699a5640 -1 log_channel(cephadm) log [ERR] : [24/Jul/2024:21:35:01] ENGINE Error in HTTPServer.serve 2024-07-24T21:35:02.071 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Traceback (most recent call last): 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._connections.run(self.expiration_interval) 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._run(expiration_interval) 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s = self.context.wrap_socket( 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:35:02.072 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: return self.sslsocket_class._create( 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self.do_handshake() 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._sslobj.do_handshake() 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:35:02.073 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: mgrmap e8: x(active, since 2s) 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:01] ENGINE Bus STARTING 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:02.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:01] ENGINE Serving on http://172.21.15.60:8765 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:01] ENGINE Serving on https://172.21.15.60:7150 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:01] ENGINE Bus STARTED 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:02 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:01] ENGINE Error in HTTPServer.serve 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:35:02.746 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-07-24T21:35:02.747 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:35:03.434 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-24T21:35:03.434 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-07-24T21:35:03.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:03 smithi060 ceph-mon[33000]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:04.388 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Generating public/private rsa key pair. 2024-07-24T21:35:04.388 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Your identification has been saved in /tmp/tmpypc0482r/key 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Your public key has been saved in /tmp/tmpypc0482r/key.pub 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: The key fingerprint is: 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: SHA256:N3Or1Y1Lfqf2fH6Z+1m03cUFBhXzq1WBhr+2KdhUIHM ceph-646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: The key's randomart image is: 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: +---[RSA 3072]----+ 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | ooBo | 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | o E + +.| 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | + + +| 2024-07-24T21:35:04.389 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | o .+| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | S + o .o+| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | . = =o+=| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | + +.*.B| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | . = =.=B| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: | . ..+B@| 2024-07-24T21:35:04.390 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: +----[SHA256]-----+ 2024-07-24T21:35:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:05 smithi060 ceph-mon[33000]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:05 smithi060 ceph-mon[33000]: Generating ssh key... 2024-07-24T21:35:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:05 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:05 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:05.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:35:05.856 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-24T21:35:05.856 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-07-24T21:35:05.856 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-07-24T21:35:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:06 smithi060 ceph-mon[33000]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:07.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:07 smithi060 ceph-mon[33000]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:08.413 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:08 smithi060 ceph-mon[33000]: Deploying cephadm binary to smithi060 2024-07-24T21:35:11.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-07-24T21:35:11.232 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-07-24T21:35:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:11 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:11 smithi060 ceph-mon[33000]: Added host smithi060 2024-07-24T21:35:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:11 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:13.238 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-24T21:35:13.238 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-07-24T21:35:13.290 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:13 smithi060 ceph-mon[33000]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:13.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:13 smithi060 ceph-mon[33000]: Saving service mon spec with placement count:5 2024-07-24T21:35:13.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:13 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:13.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:13 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:14.428 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-24T21:35:14.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:14 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:14.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:14 smithi060 ceph-mon[33000]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:14.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:14 smithi060 ceph-mon[33000]: Saving service mgr spec with placement count:2 2024-07-24T21:35:14.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:14 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:16.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:16 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3223520379' entity='client.admin' 2024-07-24T21:35:17.044 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-07-24T21:35:17.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:17 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1039997093' entity='client.admin' 2024-07-24T21:35:17.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:17 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:19.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:18 smithi060 ceph-mon[33000]: from='mgr.14118 172.21.15.60:0/1608165989' entity='mgr.x' 2024-07-24T21:35:19.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:18 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2578095719' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-24T21:35:19.071 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:18 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setuser ceph since I am not root 2024-07-24T21:35:19.071 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:18 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setgroup ceph since I am not root 2024-07-24T21:35:19.856 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:19 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2578095719' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-24T21:35:19.856 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:19 smithi060 ceph-mon[33000]: mgrmap e9: x(active, since 19s) 2024-07-24T21:35:20.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/534204556' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-24T21:35:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:35:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-24T21:35:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-24T21:35:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-24T21:35:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-24T21:35:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:35:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-07-24T21:35:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 9... 2024-07-24T21:35:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: Active manager daemon x restarted 2024-07-24T21:35:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: Activating manager daemon x 2024-07-24T21:35:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: osdmap e3: 0 total, 0 up, 0 in 2024-07-24T21:35:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: mgrmap e10: x(active, starting, since 0.00322853s) 2024-07-24T21:35:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: Manager daemon x is now available 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:35:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:35:26.136 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:35:25.826+0000 7fcca26d7640 -1 log_channel(cephadm) log [ERR] : [24/Jul/2024:21:35:25] ENGINE Error in HTTPServer.serve 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Traceback (most recent call last): 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._connections.run(self.expiration_interval) 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._run(expiration_interval) 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s = self.context.wrap_socket( 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:35:26.137 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: return self.sslsocket_class._create( 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self.do_handshake() 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._sslobj.do_handshake() 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:35:26.138 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:35:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:35:26.184 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-07-24T21:35:26.184 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-24T21:35:26.184 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-24T21:35:26.184 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-07-24T21:35:26.185 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 9 is available 2024-07-24T21:35:26.185 INFO:teuthology.orchestra.run.smithi060.stdout:Generating a dashboard self-signed certificate... 2024-07-24T21:35:26.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:26 smithi060 ceph-mon[33000]: mgrmap e11: x(active, since 1.00642s) 2024-07-24T21:35:26.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:26 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:26.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:26 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:25] ENGINE Bus STARTING 2024-07-24T21:35:26.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:26 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:25] ENGINE Serving on http://172.21.15.60:8765 2024-07-24T21:35:27.473 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-24T21:35:27.473 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-07-24T21:35:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:25] ENGINE Serving on https://172.21.15.60:7150 2024-07-24T21:35:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:25] ENGINE Bus STARTED 2024-07-24T21:35:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: [24/Jul/2024:21:35:25] ENGINE Error in HTTPServer.serve 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:35:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: mgrmap e12: x(active, since 2s) 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:27.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:28.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:28 smithi060 ceph-mon[33000]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:28.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:29 smithi060 ceph-mon[33000]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:29.586 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$RptXPPbciaCoAXrCnFl4Eeqivy3UnXf4bACOr.2bkRHEIXvhPXuEy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1721856928, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-24T21:35:29.587 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-07-24T21:35:30.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:30 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3399377357' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-24T21:35:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-07-24T21:35:30.729 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-24T21:35:30.729 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-07-24T21:35:30.737 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-24T21:35:30.737 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-07-24T21:35:30.737 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-07-24T21:35:30.737 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: Password: shqib3alym 2024-07-24T21:35:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:30.740 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config directory 2024-07-24T21:35:31.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:31 smithi060 ceph-mon[33000]: mgrmap e13: x(active, since 6s) 2024-07-24T21:35:31.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:31 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3579637139' entity='client.admin' 2024-07-24T21:35:31.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-24T21:35:31.985 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-24T21:35:31.985 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.985 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:31.986 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-07-24T21:35:32.031 INFO:tasks.cephadm:Fetching config... 2024-07-24T21:35:32.031 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:35:32.031 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-24T21:35:32.049 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-24T21:35:32.050 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:35:32.050 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-24T21:35:32.107 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-24T21:35:32.107 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:35:32.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-24T21:35:32.175 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-24T21:35:32.175 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:35:32.175 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-24T21:35:32.232 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-24T21:35:32.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-24T21:35:32.307 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:35:32.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-24T21:35:32.358 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:35:32.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-24T21:35:32.410 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIRbr1vYzyTSig/mfCQiqd6iOlL1p4BB0KwWmUAsGzSizVyZ6yfBEDN93eTIMFreAH1ObuOnTXHHPi6VoGS3MS9qrmCuQXidF5Jj1PxBCVFMxGqnDID/NrT15x3uuHe/5s7Dv4ZiXMUCAEZ27XweF0zuVRwuNEzd19rJ15614HJXr5j7gvqwFNboHphpDw+Kg1kw6kl4vMLLV8In3w3zhZw5BvJUBLvm6/sWY84uxaCWK09UKv1zT57Bq3HkSHb4YnkvN5kCIBsujhKTYAtlW1rXbJUidlpHnZGTxIQH+nPUPJUq1r9d96jFQOEs5SZriCLaesTFbTG66WMJz0zUoQwLBziLmGOY3LAfOR3PIqUjbG6Z+EoEMQ2PaTUh1UTMLUpPCAnhj53AUiMukTb/tyylMWTugdk1R7mdDzUXz/XsE+qPSga2toVHHyqQapHuEiAhnl4IrcZNsJ4Puvnu8pHNjk5GOJph+zLWJd4WU68EfN5Xtn9YjIw3bnwudJ8U= ceph-646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:35:32.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-24T21:35:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:35.193 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:35.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:35.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:35:35.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:35.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:35.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:35 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:36 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2146915489' entity='client.admin' 2024-07-24T21:35:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:36 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:36 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:36.819 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-24T21:35:36.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-24T21:35:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:39.600 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2024-07-24T21:35:39.600 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:35:39.600 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2024-07-24T21:35:39.620 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:35:39.620 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:35:39.677 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2024-07-24T21:35:39.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch host add smithi086 2024-07-24T21:35:39.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:35:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:35:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:35:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:35:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:39 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:42.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:41 smithi060 ceph-mon[33000]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:42 smithi060 ceph-mon[33000]: Deploying cephadm binary to smithi086 2024-07-24T21:35:45.432 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi086' with addr '172.21.15.86' 2024-07-24T21:35:45.432 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:45 smithi060 ceph-mon[33000]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:46.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch host ls --format=json 2024-07-24T21:35:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:46.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:46 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:46.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:46 smithi060 ceph-mon[33000]: Added host smithi086 2024-07-24T21:35:46.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:46 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:46.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:46 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:46.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:46 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:47.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:47 smithi060 ceph-mon[33000]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:47.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:47 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:48.046 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:48.047 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2024-07-24T21:35:48.632 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-07-24T21:35:48.632 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:35:48.632 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-07-24T21:35:48.650 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:35:48.650 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:35:48.707 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-07-24T21:35:48.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch host add smithi123 2024-07-24T21:35:48.967 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:49 smithi060 ceph-mon[33000]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:35:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:49 smithi060 ceph-mon[33000]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:35:51.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:50 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:51 smithi060 ceph-mon[33000]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:35:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:51 smithi060 ceph-mon[33000]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:35:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:51 smithi060 ceph-mon[33000]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:51 smithi060 ceph-mon[33000]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:35:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:51 smithi060 ceph-mon[33000]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:35:53.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:52 smithi060 ceph-mon[33000]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:35:54.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:53 smithi060 ceph-mon[33000]: Deploying cephadm binary to smithi123 2024-07-24T21:35:54.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:53 smithi060 ceph-mon[33000]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:54.721 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-07-24T21:35:55.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch host ls --format=json 2024-07-24T21:35:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: Added host smithi123 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:55.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:55 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:57.312 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:35:57.312 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-07-24T21:35:57.313 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:57 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:57.313 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:57 smithi060 ceph-mon[33000]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:57.920 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-24T21:35:57.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd crush tunables default 2024-07-24T21:35:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:35:58.296 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:58 smithi060 ceph-mon[33000]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:35:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:35:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:35:59.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:35:59 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:00.633 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-07-24T21:36:00.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:00 smithi060 ceph-mon[33000]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:00.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:00 smithi060 ceph-mon[33000]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:00.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:00 smithi060 ceph-mon[33000]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:00.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:00 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/912407725' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-24T21:36:01.238 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-07-24T21:36:01.238 INFO:tasks.cephadm:Adding mon.b on smithi086 2024-07-24T21:36:01.239 INFO:tasks.cephadm:Adding mon.c on smithi123 2024-07-24T21:36:01.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c' 2024-07-24T21:36:01.498 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:01 smithi060 ceph-mon[33000]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:01 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/912407725' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-24T21:36:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:01 smithi060 ceph-mon[33000]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:03.143 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2024-07-24T21:36:03.678 DEBUG:teuthology.orchestra.run.smithi086:mon.b> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.b.service 2024-07-24T21:36:03.681 DEBUG:teuthology.orchestra.run.smithi123:mon.c> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.c.service 2024-07-24T21:36:03.683 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-24T21:36:03.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-24T21:36:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:03 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:04.345 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:05.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:04 smithi060 ceph-mon[33000]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:36:05.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:04 smithi060 ceph-mon[33000]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2024-07-24T21:36:05.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:04 smithi060 ceph-mon[33000]: Deploying daemon mon.c on smithi123 2024-07-24T21:36:06.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:05 smithi060 ceph-mon[33000]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:36:07.434 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","modified":"2024-07-24T21:34:31.410206Z","created":"2024-07-24T21:34:31.410206Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-24T21:36:07.434 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-07-24T21:36:08.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:07 smithi060 ceph-mon[33000]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:08.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/2916973625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:08.599 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:08 smithi123 ceph-mon[37143]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-24T21:36:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:08 smithi123 bash[37124]: cd1607c2f83262b066fd06774c70a7b96ca4dcfd757ec9cda94aba90b5cd23ee 2024-07-24T21:36:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:08 smithi123 systemd[1]: Started Ceph mon.c for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:36:09.183 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-24T21:36:09.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-24T21:36:09.436 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:36:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: mon.a calling monitor election 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: mon.c calling monitor election 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: monmap epoch 2 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: last_changed 2024-07-24T21:36:08.570910+0000 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: min_mon_release 19 (squid) 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: election_strategy: 1 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: fsmap 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: mgrmap e13: x(active, since 49s) 2024-07-24T21:36:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: overall HEALTH_OK 2024-07-24T21:36:13.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:13.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:13.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:13.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:13.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:13 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: mon.a calling monitor election 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: mon.c calling monitor election 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: monmap epoch 2 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: last_changed 2024-07-24T21:36:08.570910+0000 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: min_mon_release 19 (squid) 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: election_strategy: 1 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:14.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: fsmap 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: mgrmap e13: x(active, since 49s) 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: overall HEALTH_OK 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:13 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:14.088 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:36:14.088 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","modified":"2024-07-24T21:36:08.570910Z","created":"2024-07-24T21:34:31.410206Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-24T21:36:14.088 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2024-07-24T21:36:14.653 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:14 smithi123 ceph-mon[37143]: Deploying daemon mon.b on smithi086 2024-07-24T21:36:14.653 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:14 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/2709357925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:14.653 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:14 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:15.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:14 smithi060 ceph-mon[33000]: Deploying daemon mon.b on smithi086 2024-07-24T21:36:15.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:14 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/2709357925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:15.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:14 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:15.680 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-24T21:36:15.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-24T21:36:15.938 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:36:15.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:15 smithi123 ceph-mon[37143]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:16.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:15 smithi060 ceph-mon[33000]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:16.926 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:16 smithi086 podman[37106]: 2024-07-24 21:36:16.744963366 +0000 UTC m=+0.017306531 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:36:17.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 podman[37106]: 2024-07-24 21:36:17.00505936 +0000 UTC m=+0.277402524 container init 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 podman[37106]: 2024-07-24 21:36:17.008090081 +0000 UTC m=+0.280433249 container start 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240716) 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: set uid:gid to 167:167 (ceph:ceph) 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev), process ceph-mon, pid 2 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pidfile_write: ignore empty --pid-file 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: load: jerasure load: lrc 2024-07-24T21:36:17.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: RocksDB version: 7.9.2 2024-07-24T21:36:17.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Git sha 0 2024-07-24T21:36:17.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Compile date 2024-07-22 20:32:46 2024-07-24T21:36:17.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: DB SUMMARY 2024-07-24T21:36:17.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: DB Session ID: 2N9L6Q0C4AKSIT0HK35N 2024-07-24T21:36:17.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: CURRENT file: CURRENT 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: IDENTITY file: IDENTITY 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.error_if_exists: 0 2024-07-24T21:36:17.298 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.create_if_missing: 0 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.paranoid_checks: 1 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.env: 0x55d0d10b5840 2024-07-24T21:36:17.299 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.fs: PosixFileSystem 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.info_log: 0x55d0d363f4c0 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_file_opening_threads: 16 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.statistics: (nil) 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.use_fsync: 0 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_log_file_size: 0 2024-07-24T21:36:17.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.keep_log_file_num: 1000 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.recycle_log_file_num: 0 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_fallocate: 1 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_mmap_reads: 0 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_mmap_writes: 0 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.use_direct_reads: 0 2024-07-24T21:36:17.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.create_missing_column_families: 0 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.db_log_dir: 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.wal_dir: 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-24T21:36:17.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.advise_random_on_open: 1 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.db_write_buffer_size: 0 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.write_buffer_manager: 0x55d0d3642e60 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.rate_limiter: (nil) 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.wal_recovery_mode: 2 2024-07-24T21:36:17.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enable_thread_tracking: 0 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enable_pipelined_write: 0 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.unordered_write: 0 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.row_cache: None 2024-07-24T21:36:17.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.wal_filter: None 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_ingest_behind: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.two_write_queues: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.manual_wal_flush: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.wal_compression: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.atomic_flush: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-24T21:36:17.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.log_readahead_size: 0 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.best_efforts_recovery: 0 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.allow_data_in_errors: 0 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.db_host_id: __hostname__ 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_background_jobs: 2 2024-07-24T21:36:17.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_background_compactions: -1 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_subcompactions: 1 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_total_wal_size: 0 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_open_files: -1 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bytes_per_sync: 0 2024-07-24T21:36:17.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_readahead_size: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_background_flushes: -1 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Compression algorithms supported: 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kZSTD supported: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kXpressCompression supported: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kBZip2Compression supported: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kLZ4Compression supported: 1 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kZlibCompression supported: 1 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kLZ4HCCompression supported: 1 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: kSnappyCompression supported: 1 2024-07-24T21:36:17.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.merge_operator: 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_filter: None 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_filter_factory: None 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.sst_partitioner_factory: None 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d0d363f0e0) 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-07-24T21:36:17.310 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_type: 0 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_index_type: 0 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_shortening: 1 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: checksum: 4 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: no_block_cache: 0 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache: 0x55d0d3633610 2024-07-24T21:36:17.311 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_options: 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: capacity : 536870912 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_shard_bits : 4 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: strict_capacity_limit : 0 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_compressed: (nil) 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: persistent_cache: (nil) 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size: 4096 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size_deviation: 10 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_restart_interval: 16 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_block_restart_interval: 1 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: metadata_block_size: 4096 2024-07-24T21:36:17.312 INFO:journalctl@ceph.mon.b.smithi086.stdout: partition_filters: 0 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: use_delta_encoding: 1 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: filter_policy: bloomfilter 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: whole_key_filtering: 1 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: verify_compression: 0 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: format_version: 5 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_index_compression: 1 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_align: 0 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: max_auto_readahead_size: 262144 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: prepopulate_block_cache: 0 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: initial_auto_readahead_size: 8192 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-07-24T21:36:17.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.write_buffer_size: 33554432 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_write_buffer_number: 2 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression: NoCompression 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression: Disabled 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.prefix_extractor: nullptr 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.num_levels: 7 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-24T21:36:17.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.level: 32767 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.strategy: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-24T21:36:17.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.enabled: false 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.target_file_size_base: 67108864 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-24T21:36:17.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.arena_block_size: 1048576 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-24T21:36:17.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.disable_auto_compactions: 0 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-24T21:36:17.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.inplace_update_support: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.bloom_locality: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.max_successive_merges: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.paranoid_file_checks: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.force_consistency_checks: 1 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.report_bg_io_stats: 0 2024-07-24T21:36:17.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.ttl: 2592000 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enable_blob_files: false 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.min_blob_size: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_file_size: 268435456 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.blob_file_starting_level: 0 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-24T21:36:17.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 91e795e8-9c15-4bcd-93e9-a8380b6a140a 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856977034807, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856977035255, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1721856977, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "91e795e8-9c15-4bcd-93e9-a8380b6a140a", "db_session_id": "2N9L6Q0C4AKSIT0HK35N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721856977035349, "job": 1, "event": "recovery_finished"} 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d0d3660e00 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: DB pointer 0x55d0d3758000 2024-07-24T21:36:17.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** DB Stats ** 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-24T21:36:17.322 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-24T21:36:17.323 INFO:journalctl@ceph.mon.b.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-24T21:36:17.324 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache BinnedLRUCache@0x55d0d3633610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-07-24T21:36:17.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: using public_addr v2:172.21.15.86:0/0 -> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: starting mon.b rank -1 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(???) e0 preinit fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).mds e1 new map 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).mds e1 print_map 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout: e1 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout: btime 2024-07-24T21:34:36:682639+0000 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-24T21:36:17.326 INFO:journalctl@ceph.mon.b.smithi086.stdout: legacy client fscid: -1 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout: No filesystems configured 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:36:17.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Deploying cephadm binary to smithi123 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Added host smithi123 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:17.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/912407725' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-24T21:36:17.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/912407725' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Deploying daemon mon.c on smithi123 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/2916973625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.a calling monitor election 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.c calling monitor election 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: monmap epoch 2 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: last_changed 2024-07-24T21:36:08.570910+0000 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: min_mon_release 19 (squid) 2024-07-24T21:36:17.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: election_strategy: 1 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: fsmap 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mgrmap e13: x(active, since 49s) 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: overall HEALTH_OK 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: Deploying daemon mon.b on smithi086 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/2709357925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 ceph-mon[37128]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 bash[37106]: 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 2024-07-24T21:36:17.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:17 smithi086 systemd[1]: Started Ceph mon.b for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: mon.a calling monitor election 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: mon.c calling monitor election 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: mon.b calling monitor election 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: monmap epoch 3 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: last_changed 2024-07-24T21:36:17.061434+0000 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: min_mon_release 19 (squid) 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: election_strategy: 1 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:22.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: fsmap 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: mgrmap e13: x(active, since 57s) 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: overall HEALTH_OK 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:22 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: mon.a calling monitor election 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: mon.c calling monitor election 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: mon.b calling monitor election 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: monmap epoch 3 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: last_changed 2024-07-24T21:36:17.061434+0000 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:22.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: min_mon_release 19 (squid) 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: election_strategy: 1 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: fsmap 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: mgrmap e13: x(active, since 57s) 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: overall HEALTH_OK 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.548 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:22 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: mon.a calling monitor election 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: mon.c calling monitor election 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: mon.b calling monitor election 2024-07-24T21:36:22.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: monmap epoch 3 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: last_changed 2024-07-24T21:36:17.061434+0000 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: created 2024-07-24T21:34:31.410206+0000 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: min_mon_release 19 (squid) 2024-07-24T21:36:22.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: election_strategy: 1 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: fsmap 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: osdmap e4: 0 total, 0 up, 0 in 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: mgrmap e13: x(active, since 57s) 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: overall HEALTH_OK 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:22.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:22 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:23 smithi123 ceph-mon[37143]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:23 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:23 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:23 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:23 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:23 smithi086 ceph-mon[37128]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:23 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:23 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:23 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:23 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:23 smithi060 ceph-mon[33000]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:23 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:23 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:23 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:23 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:23.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-07-24T21:36:23.647 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","modified":"2024-07-24T21:36:17.061434Z","created":"2024-07-24T21:34:31.410206Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-07-24T21:36:23.647 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 3 2024-07-24T21:36:24.220 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-24T21:36:24.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config generate-minimal-conf 2024-07-24T21:36:24.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/3173130777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:24.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:24.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:24.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.495 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:24 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/3173130777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:24.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:24.506 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.506 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:24 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/3173130777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:24.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:24 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.216 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: Reconfiguring mon.a (unknown last config time)... 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: Reconfiguring daemon mon.a on smithi060 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:25.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:25 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:25.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: Reconfiguring mon.a (unknown last config time)... 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: Reconfiguring daemon mon.a on smithi060 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:25.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:25 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: Reconfiguring mon.a (unknown last config time)... 2024-07-24T21:36:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: Reconfiguring daemon mon.a on smithi060 2024-07-24T21:36:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:25.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:25.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:25.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:25 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:26.345 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:26 smithi060 ceph-mon[33000]: Reconfiguring mon.b (monmap changed)... 2024-07-24T21:36:26.345 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:26 smithi060 ceph-mon[33000]: Reconfiguring daemon mon.b on smithi086 2024-07-24T21:36:26.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:26 smithi123 ceph-mon[37143]: Reconfiguring mon.b (monmap changed)... 2024-07-24T21:36:26.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:26 smithi123 ceph-mon[37143]: Reconfiguring daemon mon.b on smithi086 2024-07-24T21:36:26.495 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:26 smithi086 ceph-mon[37128]: Reconfiguring mon.b (monmap changed)... 2024-07-24T21:36:26.496 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:26 smithi086 ceph-mon[37128]: Reconfiguring daemon mon.b on smithi086 2024-07-24T21:36:26.762 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:26.763 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-07-24T21:36:26.763 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:36:26.763 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2024-07-24T21:36:27.340 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-24T21:36:27.340 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:36:27.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-07-24T21:36:27.370 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:36:27.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:27.436 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:36:27.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-07-24T21:36:27.467 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:36:27.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:27.523 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:27.523 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: Reconfiguring mon.c (monmap changed)... 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: Reconfiguring daemon mon.c on smithi123 2024-07-24T21:36:27.524 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3744411542' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:27.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: Reconfiguring mon.c (monmap changed)... 2024-07-24T21:36:27.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:27.530 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:27.530 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.530 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: Reconfiguring daemon mon.c on smithi123 2024-07-24T21:36:27.530 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3744411542' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.533 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:36:27.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-07-24T21:36:27.561 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:36:27.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: Reconfiguring mon.c (monmap changed)... 2024-07-24T21:36:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-24T21:36:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-24T21:36:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: Reconfiguring daemon mon.c on smithi123 2024-07-24T21:36:27.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3744411542' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:27.626 INFO:tasks.cephadm:enabling module snap_schedule 2024-07-24T21:36:27.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-07-24T21:36:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:28.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:28 smithi060 ceph-mon[33000]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:28 smithi123 ceph-mon[37143]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:29.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:28 smithi086 ceph-mon[37128]: from='mgr.14150 172.21.15.60:0/2550798655' entity='mgr.x' 2024-07-24T21:36:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:29 smithi123 ceph-mon[37143]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:30.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:29 smithi086 ceph-mon[37128]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:29 smithi060 ceph-mon[33000]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:30.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:30 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:36:30.916 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:30 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setuser ceph since I am not root 2024-07-24T21:36:30.916 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:30 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ignoring --setgroup ceph since I am not root 2024-07-24T21:36:30.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:30 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:36:31.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:30 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:36:31.204 INFO:tasks.cephadm:Adding mgr.x on smithi060 2024-07-24T21:36:31.205 INFO:tasks.cephadm:Adding mgr.y on smithi086 2024-07-24T21:36:31.205 INFO:tasks.cephadm:Adding mgr.z on smithi123 2024-07-24T21:36:31.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch apply mgr '3;smithi060=x;smithi086=y;smithi123=z' 2024-07-24T21:36:31.460 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:36:31.893 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:31 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:36:31.893 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:31 smithi123 ceph-mon[37143]: mgrmap e14: x(active, since 66s) 2024-07-24T21:36:32.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:31 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:36:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:31 smithi086 ceph-mon[37128]: mgrmap e14: x(active, since 66s) 2024-07-24T21:36:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:31 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/727172269' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:36:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:31 smithi060 ceph-mon[33000]: mgrmap e14: x(active, since 66s) 2024-07-24T21:36:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: Active manager daemon x restarted 2024-07-24T21:36:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: Activating manager daemon x 2024-07-24T21:36:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: osdmap e5: 0 total, 0 up, 0 in 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: mgrmap e15: x(active, starting, since 0.00433725s) 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: Manager daemon x is now available 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:36.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:36:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: Active manager daemon x restarted 2024-07-24T21:36:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: Activating manager daemon x 2024-07-24T21:36:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: osdmap e5: 0 total, 0 up, 0 in 2024-07-24T21:36:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: mgrmap e15: x(active, starting, since 0.00433725s) 2024-07-24T21:36:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: Manager daemon x is now available 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:36.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: Active manager daemon x restarted 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: Activating manager daemon x 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: osdmap e5: 0 total, 0 up, 0 in 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: mgrmap e15: x(active, starting, since 0.00433725s) 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-24T21:36:36.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: Manager daemon x is now available 2024-07-24T21:36:36.364 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:36.364 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-24T21:36:36.970 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mgr update... 2024-07-24T21:36:37.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:36:37.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:37 smithi123 ceph-mon[37143]: mgrmap e16: x(active, since 1.00824s) 2024-07-24T21:36:37.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:37.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:36:37.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:37 smithi060 ceph-mon[33000]: mgrmap e16: x(active, since 1.00824s) 2024-07-24T21:36:37.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:37.283 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-24T21:36:37.283 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:37 smithi086 ceph-mon[37128]: mgrmap e16: x(active, since 1.00824s) 2024-07-24T21:36:37.283 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:37.916 DEBUG:teuthology.orchestra.run.smithi086:mgr.y> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.y.service 2024-07-24T21:36:37.918 DEBUG:teuthology.orchestra.run.smithi123:mgr.z> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.z.service 2024-07-24T21:36:37.920 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-07-24T21:36:37.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-07-24T21:36:37.923 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-07-24T21:36:37.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set client debug_client 20 2024-07-24T21:36:37.965 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-07-24T21:36:37.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set client debug_ms 1 2024-07-24T21:36:38.009 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-07-24T21:36:38.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-07-24T21:36:38.052 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-07-24T21:36:38.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-07-24T21:36:38.095 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-07-24T21:36:38.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds debug_mds 20 2024-07-24T21:36:38.140 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-07-24T21:36:38.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-07-24T21:36:38.183 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2024-07-24T21:36:38.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds debug_ms 1 2024-07-24T21:36:38.237 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-07-24T21:36:38.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-07-24T21:36:38.284 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-07-24T21:36:38.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-07-24T21:36:38.338 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-07-24T21:36:38.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-07-24T21:36:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:38.373 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-07-24T21:36:38.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-07-24T21:36:38.393 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:38.404 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-07-24T21:36:38.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-07-24T21:36:38.462 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-07-24T21:36:38.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-07-24T21:36:38.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: Saving service mgr spec with placement smithi060=x;smithi086=y;smithi123=z;count:3 2024-07-24T21:36:38.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:36:38.233+0000 7f88f4b9d640 -1 log_channel(cephadm) log [ERR] : [24/Jul/2024:21:36:38] ENGINE Error in HTTPServer.serve 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: Traceback (most recent call last): 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._connections.run(self.expiration_interval) 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._run(expiration_interval) 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:36:38.500 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: s = self.context.wrap_socket( 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: return self.sslsocket_class._create( 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self.do_handshake() 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: self._sslobj.do_handshake() 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:36:38.501 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:36:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:36:38.502 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-07-24T21:36:38.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-07-24T21:36:38.545 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-07-24T21:36:38.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-07-24T21:36:38.603 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-07-24T21:36:38.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-07-24T21:36:38.660 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-07-24T21:36:38.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-07-24T21:36:38.720 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-07-24T21:36:38.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-07-24T21:36:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: Saving service mgr spec with placement smithi060=x;smithi086=y;smithi123=z;count:3 2024-07-24T21:36:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:38.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:38.794 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-07-24T21:36:38.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-07-24T21:36:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: Saving service mgr spec with placement smithi060=x;smithi086=y;smithi123=z;count:3 2024-07-24T21:36:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:36:38.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:38.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:38.844 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-07-24T21:36:38.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-24T21:36:38.890 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-07-24T21:36:38.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-24T21:36:38.934 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-07-24T21:36:38.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-07-24T21:36:38.997 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-07-24T21:36:38.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-07-24T21:36:39.042 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-24T21:36:39.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-07-24T21:36:39.090 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-07-24T21:36:39.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-07-24T21:36:39.141 DEBUG:tasks.cephadm:set 26 configs 2024-07-24T21:36:39.141 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi060') 2024-07-24T21:36:39.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: [24/Jul/2024:21:36:38] ENGINE Bus STARTING 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: [24/Jul/2024:21:36:38] ENGINE Serving on https://172.21.15.60:7150 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: [24/Jul/2024:21:36:38] ENGINE Error in HTTPServer.serve 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:36:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: [24/Jul/2024:21:36:38] ENGINE Serving on http://172.21.15.60:8765 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: [24/Jul/2024:21:36:38] ENGINE Bus STARTED 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:39.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:39.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:39 smithi060 ceph-mon[33000]: mgrmap e17: x(active, since 2s) 2024-07-24T21:36:39.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: [24/Jul/2024:21:36:38] ENGINE Bus STARTING 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: [24/Jul/2024:21:36:38] ENGINE Serving on https://172.21.15.60:7150 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: [24/Jul/2024:21:36:38] ENGINE Error in HTTPServer.serve 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: Traceback (most recent call last): 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._run(expiration_interval) 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:36:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: s = self.context.wrap_socket( 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: return self.sslsocket_class._create( 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: self.do_handshake() 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._sslobj.do_handshake() 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: [24/Jul/2024:21:36:38] ENGINE Serving on http://172.21.15.60:8765 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: [24/Jul/2024:21:36:38] ENGINE Bus STARTED 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:39 smithi123 ceph-mon[37143]: mgrmap e17: x(active, since 2s) 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: [24/Jul/2024:21:36:38] ENGINE Bus STARTING 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: [24/Jul/2024:21:36:38] ENGINE Serving on https://172.21.15.60:7150 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: [24/Jul/2024:21:36:38] ENGINE Error in HTTPServer.serve 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: Traceback (most recent call last): 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: self._connections.run(self.expiration_interval) 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: self._run(expiration_interval) 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-24T21:36:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: s = self.context.wrap_socket( 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: return self.sslsocket_class._create( 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: self.do_handshake() 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: self._sslobj.do_handshake() 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: [24/Jul/2024:21:36:38] ENGINE Serving on http://172.21.15.60:8765 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: [24/Jul/2024:21:36:38] ENGINE Bus STARTED 2024-07-24T21:36:40.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: Updating smithi060:/etc/ceph/ceph.conf 2024-07-24T21:36:40.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: Updating smithi086:/etc/ceph/ceph.conf 2024-07-24T21:36:40.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: Updating smithi123:/etc/ceph/ceph.conf 2024-07-24T21:36:40.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:39 smithi086 ceph-mon[37128]: mgrmap e17: x(active, since 2s) 2024-07-24T21:36:40.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:40.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:40.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:40.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:40.828 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.828 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.828 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:40.828 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:40.828 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:40.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:41 smithi123 ceph-mon[37143]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:41 smithi123 ceph-mon[37143]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:41 smithi123 ceph-mon[37143]: Deploying daemon mgr.z on smithi123 2024-07-24T21:36:41.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:41 smithi060 ceph-mon[33000]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:41.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:41 smithi060 ceph-mon[33000]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:41.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:41 smithi060 ceph-mon[33000]: Deploying daemon mgr.z on smithi123 2024-07-24T21:36:42.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:41 smithi086 ceph-mon[37128]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-07-24T21:36:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:41 smithi086 ceph-mon[37128]: Updating smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-24T21:36:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:41 smithi086 ceph-mon[37128]: Deploying daemon mgr.z on smithi123 2024-07-24T21:36:42.441 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mgr[39978]: -- 172.21.15.123:0/3625071141 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572a891d0e0 con 0x5572a88fa800 2024-07-24T21:36:42.742 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 21:36:42 smithi123 bash[39959]: 9c73f2b52023b825d1fff57dcfdce60275e75979b7af647e49d73dfc04fd6786 2024-07-24T21:36:42.742 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 21:36:42 smithi123 systemd[1]: Started Ceph mgr.z for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:36:42.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='client.? ' entity='client.admin' 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3031749155' entity='client.admin' 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:42.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:42 smithi060 ceph-mon[33000]: Deploying daemon mgr.y on smithi086 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='client.? ' entity='client.admin' 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3031749155' entity='client.admin' 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:43.080 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:43.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:43.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:43.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:42 smithi086 ceph-mon[37128]: Deploying daemon mgr.y on smithi086 2024-07-24T21:36:43.182 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi060') 2024-07-24T21:36:43.182 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi060') 2024-07-24T21:36:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='client.? ' entity='client.admin' 2024-07-24T21:36:43.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3031749155' entity='client.admin' 2024-07-24T21:36:43.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:42 smithi123 ceph-mon[37143]: Deploying daemon mgr.y on smithi086 2024-07-24T21:36:43.253 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.253 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.259 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.260 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.268 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.268 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.274 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.279 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.282 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.288 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.304 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.308 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.326 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:36:44.794 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jul 24 21:36:44 smithi086 systemd[1]: Started Ceph mgr.y for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:36:45.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:46.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:36:46.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: Reconfiguring mgr.x (unknown last config time)... 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: Reconfiguring daemon mgr.x on smithi060 2024-07-24T21:36:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: Reconfiguring mgr.x (unknown last config time)... 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: Reconfiguring daemon mgr.x on smithi060 2024-07-24T21:36:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: Reconfiguring mgr.x (unknown last config time)... 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: Reconfiguring daemon mgr.x on smithi060 2024-07-24T21:36:47.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:36:47.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:47 smithi060 ceph-mon[33000]: Standby manager daemon z started 2024-07-24T21:36:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:47 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-24T21:36:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:47 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:47 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-24T21:36:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:47 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:47 smithi123 ceph-mon[37143]: Standby manager daemon z started 2024-07-24T21:36:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:47 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-24T21:36:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:47 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:47 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-24T21:36:47.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:47 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:48.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:47 smithi086 ceph-mon[37128]: Standby manager daemon z started 2024-07-24T21:36:48.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:47 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-24T21:36:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:47 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:47 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-24T21:36:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:47 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.123:0/447547973' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:48.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:48 smithi123 ceph-mon[37143]: mgrmap e18: x(active, since 11s), standbys: z 2024-07-24T21:36:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:48 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-24T21:36:49.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:48 smithi060 ceph-mon[33000]: mgrmap e18: x(active, since 11s), standbys: z 2024-07-24T21:36:49.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:48 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-24T21:36:49.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:48 smithi086 ceph-mon[37128]: mgrmap e18: x(active, since 11s), standbys: z 2024-07-24T21:36:49.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:48 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-24T21:36:49.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:49 smithi060 ceph-mon[33000]: Standby manager daemon y started 2024-07-24T21:36:49.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:49 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-24T21:36:49.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:49 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:49.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:49 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-24T21:36:49.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:49 smithi060 ceph-mon[33000]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:49.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:49 smithi123 ceph-mon[37143]: Standby manager daemon y started 2024-07-24T21:36:49.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:49 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-24T21:36:49.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:49 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:49 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-24T21:36:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:49 smithi123 ceph-mon[37143]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:49 smithi086 ceph-mon[37128]: Standby manager daemon y started 2024-07-24T21:36:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:49 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-24T21:36:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:49 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-24T21:36:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:49 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-24T21:36:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:49 smithi086 ceph-mon[37128]: from='mgr.? 172.21.15.86:0/3217800082' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-24T21:36:50.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:50 smithi060 ceph-mon[33000]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-07-24T21:36:50.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-24T21:36:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:50 smithi123 ceph-mon[37143]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-07-24T21:36:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-24T21:36:51.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:50 smithi086 ceph-mon[37128]: mgrmap e19: x(active, since 13s), standbys: z, y 2024-07-24T21:36:51.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-24T21:36:57.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:57 smithi086 ceph-mon[37128]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:57.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:57 smithi060 ceph-mon[33000]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:57.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:57 smithi123 ceph-mon[37143]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:58.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:58 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/298767234' entity='client.admin' 2024-07-24T21:36:58.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:58 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/298767234' entity='client.admin' 2024-07-24T21:36:58.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:58 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/298767234' entity='client.admin' 2024-07-24T21:36:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:36:59 smithi060 ceph-mon[33000]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:36:59 smithi123 ceph-mon[37143]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:36:59.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:36:59 smithi086 ceph-mon[37128]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:01.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:01 smithi060 ceph-mon[33000]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:01 smithi123 ceph-mon[37143]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:01 smithi086 ceph-mon[37128]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:03.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:03 smithi060 ceph-mon[33000]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:03.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:03 smithi123 ceph-mon[37143]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:03.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:03 smithi086 ceph-mon[37128]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:03.556 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:37:04.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:04 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/666218194' entity='client.admin' 2024-07-24T21:37:04.561 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:04 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/666218194' entity='client.admin' 2024-07-24T21:37:04.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:04 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/666218194' entity='client.admin' 2024-07-24T21:37:05.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:05 smithi060 ceph-mon[33000]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:05.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:05 smithi123 ceph-mon[37143]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:05.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:05 smithi086 ceph-mon[37128]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:07.427 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:07 smithi060 ceph-mon[33000]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:07.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1353019088' entity='client.admin' 2024-07-24T21:37:07.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4003857449' entity='client.admin' 2024-07-24T21:37:07.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2732480064' entity='client.admin' 2024-07-24T21:37:07.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1525481852' entity='client.admin' 2024-07-24T21:37:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:07 smithi123 ceph-mon[37143]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:07 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1353019088' entity='client.admin' 2024-07-24T21:37:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:07 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4003857449' entity='client.admin' 2024-07-24T21:37:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:07 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2732480064' entity='client.admin' 2024-07-24T21:37:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:07 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1525481852' entity='client.admin' 2024-07-24T21:37:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:07 smithi086 ceph-mon[37128]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:07 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1353019088' entity='client.admin' 2024-07-24T21:37:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:07 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4003857449' entity='client.admin' 2024-07-24T21:37:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:07 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2732480064' entity='client.admin' 2024-07-24T21:37:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:07 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1525481852' entity='client.admin' 2024-07-24T21:37:08.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3e780b3c10 msgr2=0x7f3e780b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:08.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3e780b3c10 0x7f3e780b60d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000f80 tx=0x7f3e88031570 comp rx=0 tx=0).stop 2024-07-24T21:37:08.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e8c1081a0 msgr2=0x7f3e8c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e8c1081a0 0x7f3e8c198f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00d950 tx=0x7f3e7c00de10 comp rx=0 tx=0).stop 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 shutdown_connections 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3e780b3c10 0x7f3e780b60d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e8c108b20 0x7f3e8c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e8c1081a0 0x7f3e8c198f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 --2- 172.21.15.60:0/409764121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e8c102790 0x7f3e8c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 >> 172.21.15.60:0/409764121 conn(0x7f3e8c0fe200 msgr2=0x7f3e8c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 shutdown_connections 2024-07-24T21:37:08.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.116+0000 7f3e946c6640 1 -- 172.21.15.60:0/409764121 wait complete. 2024-07-24T21:37:08.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.957+0000 7f7f2ca95640 1 -- 172.21.15.60:0/785864552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f281081a0 msgr2=0x7f7f281085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:08.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.957+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/785864552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f281081a0 0x7f7f281085e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009900 tx=0x7f7f1c02a410 comp rx=0 tx=0).stop 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 -- 172.21.15.60:0/785864552 shutdown_connections 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/785864552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f28108b20 0x7f7f2810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/785864552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f281081a0 0x7f7f281085e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/785864552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 0x7f7f28102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 -- 172.21.15.60:0/785864552 >> 172.21.15.60:0/785864552 conn(0x7f7f280fe200 msgr2=0x7f7f28100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 -- 172.21.15.60:0/785864552 shutdown_connections 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 -- 172.21.15.60:0/785864552 wait complete. 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 Processor -- start 2024-07-24T21:37:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.958+0000 7f7f2ca95640 1 -- start start 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 0x7f7f28074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281081a0 0x7f7f280753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281125f0 con 0x7f7f28108b20 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112470 con 0x7f7f281081a0 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2ca95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112770 con 0x7f7f28102790 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50352/0 (socket says 172.21.15.60:50352) 2024-07-24T21:37:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 -- 172.21.15.60:0/1612524632 learned_addr learned my addr 172.21.15.60:0/1612524632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f2659c640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 0x7f7f28074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f25d9b640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281081a0 0x7f7f280753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 -- 172.21.15.60:0/1612524632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 msgr2=0x7f7f28074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 0x7f7f28074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 -- 172.21.15.60:0/1612524632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281081a0 msgr2=0x7f7f280753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281081a0 0x7f7f280753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 -- 172.21.15.60:0/1612524632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f281a6fa0 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.959+0000 7f7f26d9d640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000efd0 tx=0x7f7f2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.960+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20003b70 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.960+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f20003d10 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.960+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20010860 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.960+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281a7290 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.960+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a7750 con 0x7f7f28108b20 2024-07-24T21:37:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.961+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f28102b70 con 0x7f7f28108b20 2024-07-24T21:37:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.963+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f7f20004b40 con 0x7f7f28108b20 2024-07-24T21:37:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.964+0000 7f7f177fe640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 0x7f7f100b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.964+0000 7f7f2659c640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 0x7f7f100b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:08.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.964+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7f200e2900 con 0x7f7f28108b20 2024-07-24T21:37:08.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.964+0000 7f7f2659c640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 0x7f7f100b60d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bbe0 tx=0x7f7f1800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:08.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:08.967+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7f200abab0 con 0x7f7f28108b20 2024-07-24T21:37:09.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.164+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f7f28076190 con 0x7f7f28108b20 2024-07-24T21:37:09.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.168+0000 7f7f177fe640 1 -- 172.21.15.60:0/1612524632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v19)=0 v19) ==== 129+0+0 (secure 0 0 0) 0x7f7f28076190 con 0x7f7f28108b20 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 msgr2=0x7f7f100b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 0x7f7f100b60d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bbe0 tx=0x7f7f1800b390 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 msgr2=0x7f7f28075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000efd0 tx=0x7f7f2000c6a0 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 shutdown_connections 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f100b3c10 0x7f7f100b60d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f28108b20 0x7f7f28075910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281081a0 0x7f7f280753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 --2- 172.21.15.60:0/1612524632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f28102790 0x7f7f28074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.171+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 >> 172.21.15.60:0/1612524632 conn(0x7f7f280fe200 msgr2=0x7f7f281005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.172+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 shutdown_connections 2024-07-24T21:37:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.172+0000 7f7f2ca95640 1 -- 172.21.15.60:0/1612524632 wait complete. 2024-07-24T21:37:09.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:09 smithi060 ceph-mon[33000]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:09.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:09 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/409764121' entity='client.admin' 2024-07-24T21:37:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:09 smithi123 ceph-mon[37143]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:09 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/409764121' entity='client.admin' 2024-07-24T21:37:09.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:09 smithi086 ceph-mon[37128]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:09.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:09 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/409764121' entity='client.admin' 2024-07-24T21:37:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 -- 172.21.15.60:0/1654405688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958107060 msgr2=0x7f89581074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 --2- 172.21.15.60:0/1654405688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958107060 0x7f89581074a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f894802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 -- 172.21.15.60:0/1654405688 shutdown_connections 2024-07-24T21:37:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 --2- 172.21.15.60:0/1654405688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89581079e0 0x7f895810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 --2- 172.21.15.60:0/1654405688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958107060 0x7f89581074a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 --2- 172.21.15.60:0/1654405688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8958101650 0x7f8958101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 -- 172.21.15.60:0/1654405688 >> 172.21.15.60:0/1654405688 conn(0x7f89580fd0c0 msgr2=0x7f89580ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 -- 172.21.15.60:0/1654405688 shutdown_connections 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.082+0000 7f895f39a640 1 -- 172.21.15.60:0/1654405688 wait complete. 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 Processor -- start 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 -- start start 2024-07-24T21:37:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958107060 0x7f8958199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89581079e0 0x7f895819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8958112630 con 0x7f8958101650 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89581124b0 con 0x7f89581079e0 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895f39a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89581127b0 con 0x7f8958107060 2024-07-24T21:37:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50380/0 (socket says 172.21.15.60:50380) 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 -- 172.21.15.60:0/3411182831 learned_addr learned my addr 172.21.15.60:0/3411182831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895c935640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958107060 0x7f8958199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 -- 172.21.15.60:0/3411182831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958107060 msgr2=0x7f8958199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958107060 0x7f8958199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 -- 172.21.15.60:0/3411182831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89581079e0 msgr2=0x7f895819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89581079e0 0x7f895819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.083+0000 7f895d136640 1 -- 172.21.15.60:0/3411182831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895819ec70 con 0x7f8958101650 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f895d136640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f895400c890 tx=0x7f895400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954004220 con 0x7f8958101650 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8954007e40 con 0x7f8958101650 2024-07-24T21:37:10.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895819ef60 con 0x7f8958101650 2024-07-24T21:37:10.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954004a00 con 0x7f8958101650 2024-07-24T21:37:10.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.084+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a57f0 con 0x7f8958101650 2024-07-24T21:37:10.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.085+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958101a30 con 0x7f8958101650 2024-07-24T21:37:10.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.086+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f89540079b0 con 0x7f8958101650 2024-07-24T21:37:10.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.087+0000 7f894e7fc640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 0x7f89440b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:10.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.087+0000 7f895c935640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 0x7f89440b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:10.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.088+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f89540e2200 con 0x7f8958101650 2024-07-24T21:37:10.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.088+0000 7f895c935640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 0x7f89440b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8958105af0 tx=0x7f8948005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:10.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.090+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8954014030 con 0x7f8958101650 2024-07-24T21:37:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.286+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f895802e000 con 0x7f8958101650 2024-07-24T21:37:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.290+0000 7f894e7fc640 1 -- 172.21.15.60:0/3411182831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f89540ab430 con 0x7f8958101650 2024-07-24T21:37:10.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 msgr2=0x7f89440b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 0x7f89440b60d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8958105af0 tx=0x7f8948005a20 comp rx=0 tx=0).stop 2024-07-24T21:37:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 msgr2=0x7f8958198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f895400c890 tx=0x7f895400cd50 comp rx=0 tx=0).stop 2024-07-24T21:37:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 shutdown_connections 2024-07-24T21:37:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89440b3c10 0x7f89440b60d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89581079e0 0x7f895819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958107060 0x7f8958199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 --2- 172.21.15.60:0/3411182831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8958101650 0x7f8958198bc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 >> 172.21.15.60:0/3411182831 conn(0x7f89580fd0c0 msgr2=0x7f89580ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 shutdown_connections 2024-07-24T21:37:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:10.292+0000 7f895f39a640 1 -- 172.21.15.60:0/3411182831 wait complete. 2024-07-24T21:37:10.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:10 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1612524632' entity='client.admin' 2024-07-24T21:37:10.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:10 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/627189387' entity='client.admin' 2024-07-24T21:37:10.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:10 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1612524632' entity='client.admin' 2024-07-24T21:37:10.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:10 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/627189387' entity='client.admin' 2024-07-24T21:37:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:10 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1612524632' entity='client.admin' 2024-07-24T21:37:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:10 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/627189387' entity='client.admin' 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/2254846137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e010cae0 msgr2=0x7fa5e010ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/2254846137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e010cae0 0x7fa5e010ef40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009ca0 tx=0x7fa5d802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/2254846137 shutdown_connections 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/2254846137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e010cae0 0x7fa5e010ef40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/2254846137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e0103fa0 0x7fa5e010c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/2254846137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 0x7fa5e0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/2254846137 >> 172.21.15.60:0/2254846137 conn(0x7fa5e00fd0c0 msgr2=0x7fa5e00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/2254846137 shutdown_connections 2024-07-24T21:37:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.713+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/2254846137 wait complete. 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 Processor -- start 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 -- start start 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 0x7fa5e0101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e010cae0 0x7fa5e0101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0112600 con 0x7fa5e0103fa0 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0112480 con 0x7fa5e010cae0 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5e5d2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0112780 con 0x7fa5e0103680 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50362/0 (socket says 172.21.15.60:50362) 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5deffd640 1 -- 172.21.15.60:0/627189387 learned_addr learned my addr 172.21.15.60:0/627189387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5dffff640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e010cae0 0x7fa5e0101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.714+0000 7fa5df7fe640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 0x7fa5e0101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 -- 172.21.15.60:0/627189387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 msgr2=0x7fa5e0101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 0x7fa5e0101330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 -- 172.21.15.60:0/627189387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e010cae0 msgr2=0x7fa5e0101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e010cae0 0x7fa5e0101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 -- 172.21.15.60:0/627189387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01a6f70 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5deffd640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d740 tx=0x7fa5d400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4017020 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01a7260 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5d400de50 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400f620 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.715+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a77a0 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.716+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0103a60 con 0x7fa5e0103fa0 2024-07-24T21:37:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.719+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fa5d401a020 con 0x7fa5e0103fa0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.719+0000 7fa5dcff9640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 0x7fa5c80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.719+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa5d40e1c70 con 0x7fa5e0103fa0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.719+0000 7fa5df7fe640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 0x7fa5c80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.720+0000 7fa5df7fe640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 0x7fa5c80b6050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009bd0 tx=0x7fa5d0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.722+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa5d4021070 con 0x7fa5e0103fa0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.916+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fa5e0102630 con 0x7fa5e0103fa0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.920+0000 7fa5dcff9640 1 -- 172.21.15.60:0/627189387 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7fa5d40aaec0 con 0x7fa5e0103fa0 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 msgr2=0x7fa5c80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 0x7fa5c80b6050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009bd0 tx=0x7fa5d0005f90 comp rx=0 tx=0).stop 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 msgr2=0x7fa5e0101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d740 tx=0x7fa5d400dc00 comp rx=0 tx=0).stop 2024-07-24T21:37:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 shutdown_connections 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5c80b3b90 0x7fa5c80b6050 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e010cae0 0x7fa5e0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e0103fa0 0x7fa5e0101870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 --2- 172.21.15.60:0/627189387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5e0103680 0x7fa5e0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 >> 172.21.15.60:0/627189387 conn(0x7fa5e00fd0c0 msgr2=0x7fa5e00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 shutdown_connections 2024-07-24T21:37:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:09.922+0000 7fa5e5d2c640 1 -- 172.21.15.60:0/627189387 wait complete. 2024-07-24T21:37:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:11 smithi086 ceph-mon[37128]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:11 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3411182831' entity='client.admin' 2024-07-24T21:37:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:11 smithi060 ceph-mon[33000]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:11 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3411182831' entity='client.admin' 2024-07-24T21:37:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:11 smithi123 ceph-mon[37143]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:11 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3411182831' entity='client.admin' 2024-07-24T21:37:12.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 -- 172.21.15.60:0/327334429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c107a20 msgr2=0x7f3b6c10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:12.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 --2- 172.21.15.60:0/327334429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c107a20 0x7f3b6c10e220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009ca0 tx=0x7f3b6402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 -- 172.21.15.60:0/327334429 shutdown_connections 2024-07-24T21:37:12.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 --2- 172.21.15.60:0/327334429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c107a20 0x7f3b6c10e220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 --2- 172.21.15.60:0/327334429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c1074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 --2- 172.21.15.60:0/327334429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c101690 0x7f3b6c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 -- 172.21.15.60:0/327334429 >> 172.21.15.60:0/327334429 conn(0x7f3b6c0fd100 msgr2=0x7f3b6c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 -- 172.21.15.60:0/327334429 shutdown_connections 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 -- 172.21.15.60:0/327334429 wait complete. 2024-07-24T21:37:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.145+0000 7f3b72313640 1 Processor -- start 2024-07-24T21:37:12.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 -- start start 2024-07-24T21:37:12.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c101690 0x7f3b6c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:12.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:12.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 0x7f3b6c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:12.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c1126e0 con 0x7f3b6c1070a0 2024-07-24T21:37:12.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c112560 con 0x7f3b6c107a20 2024-07-24T21:37:12.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b72313640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c112860 con 0x7f3b6c101690 2024-07-24T21:37:12.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:12.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50390/0 (socket says 172.21.15.60:50390) 2024-07-24T21:37:12.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 -- 172.21.15.60:0/2435287286 learned_addr learned my addr 172.21.15.60:0/2435287286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b6bfff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c101690 0x7f3b6c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b708b0640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 0x7f3b6c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 -- 172.21.15.60:0/2435287286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c101690 msgr2=0x7f3b6c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c101690 0x7f3b6c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 -- 172.21.15.60:0/2435287286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 msgr2=0x7f3b6c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 0x7f3b6c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b63fff640 1 -- 172.21.15.60:0/2435287286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c19ebe0 con 0x7f3b6c1070a0 2024-07-24T21:37:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.146+0000 7f3b708b0640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 0x7f3b6c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:37:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b63fff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400d770 tx=0x7f3b5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54017070 con 0x7f3b6c1070a0 2024-07-24T21:37:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b540054b0 con 0x7f3b6c1070a0 2024-07-24T21:37:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c19eed0 con 0x7f3b6c1070a0 2024-07-24T21:37:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54004c70 con 0x7f3b6c1070a0 2024-07-24T21:37:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.147+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c1a5590 con 0x7f3b6c1070a0 2024-07-24T21:37:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.148+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c101a70 con 0x7f3b6c1070a0 2024-07-24T21:37:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.149+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f3b54004e10 con 0x7f3b6c1070a0 2024-07-24T21:37:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.150+0000 7f3b69ffb640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 0x7f3b580b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.150+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3b540e1e30 con 0x7f3b6c1070a0 2024-07-24T21:37:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.150+0000 7f3b6bfff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 0x7f3b580b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.151+0000 7f3b6bfff640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 0x7f3b580b60d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00bbe0 tx=0x7f3b5c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.153+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3b540ab100 con 0x7f3b6c1070a0 2024-07-24T21:37:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.348+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f3b6c02e000 con 0x7f3b6c1070a0 2024-07-24T21:37:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.352+0000 7f3b69ffb640 1 -- 172.21.15.60:0/2435287286 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v22)=0 v22) ==== 116+0+0 (secure 0 0 0) 0x7f3b540aaa00 con 0x7f3b6c1070a0 2024-07-24T21:37:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 msgr2=0x7f3b580b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 0x7f3b580b60d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00bbe0 tx=0x7f3b5c00b390 comp rx=0 tx=0).stop 2024-07-24T21:37:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 msgr2=0x7f3b6c199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:12.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400d770 tx=0x7f3b5400dc30 comp rx=0 tx=0).stop 2024-07-24T21:37:12.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 shutdown_connections 2024-07-24T21:37:12.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b580b3c10 0x7f3b580b60d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b6c107a20 0x7f3b6c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b6c1070a0 0x7f3b6c199070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 --2- 172.21.15.60:0/2435287286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b6c101690 0x7f3b6c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 >> 172.21.15.60:0/2435287286 conn(0x7f3b6c0fd100 msgr2=0x7f3b6c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 shutdown_connections 2024-07-24T21:37:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:12.354+0000 7f3b72313640 1 -- 172.21.15.60:0/2435287286 wait complete. 2024-07-24T21:37:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:13 smithi060 ceph-mon[33000]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2435287286' entity='client.admin' 2024-07-24T21:37:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:13 smithi123 ceph-mon[37143]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2435287286' entity='client.admin' 2024-07-24T21:37:13.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:13 smithi086 ceph-mon[37128]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2435287286' entity='client.admin' 2024-07-24T21:37:15.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 -- 172.21.15.60:0/4292518911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40115400 msgr2=0x7f6f401177f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 --2- 172.21.15.60:0/4292518911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40115400 0x7f6f401177f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009ca0 tx=0x7f6f3802a950 comp rx=0 tx=0).stop 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 -- 172.21.15.60:0/4292518911 shutdown_connections 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 --2- 172.21.15.60:0/4292518911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40115400 0x7f6f401177f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 --2- 172.21.15.60:0/4292518911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f4010a460 0x7f6f40114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 --2- 172.21.15.60:0/4292518911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40109b40 0x7f6f40109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 -- 172.21.15.60:0/4292518911 >> 172.21.15.60:0/4292518911 conn(0x7f6f4006d980 msgr2=0x7f6f4006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.012+0000 7f6f461d8640 1 -- 172.21.15.60:0/4292518911 shutdown_connections 2024-07-24T21:37:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.013+0000 7f6f461d8640 1 -- 172.21.15.60:0/4292518911 wait complete. 2024-07-24T21:37:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.013+0000 7f6f461d8640 1 Processor -- start 2024-07-24T21:37:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 -- start start 2024-07-24T21:37:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 0x7f6f401a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4010a460 0x7f6f401a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 0x7f6f401a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4011ae20 con 0x7f6f40109b40 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4011aca0 con 0x7f6f4010a460 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f461d8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4011afa0 con 0x7f6f40115400 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 0x7f6f401a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 0x7f6f401a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50402/0 (socket says 172.21.15.60:50402) 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f3f7fe640 1 -- 172.21.15.60:0/1518623754 learned_addr learned my addr 172.21.15.60:0/1518623754 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f3ffff640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 0x7f6f401a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.014+0000 7f6f3effd640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4010a460 0x7f6f401a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 -- 172.21.15.60:0/1518623754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4010a460 msgr2=0x7f6f401a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4010a460 0x7f6f401a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 -- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 msgr2=0x7f6f401a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 0x7f6f401a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 -- 172.21.15.60:0/1518623754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f401a7420 con 0x7f6f40115400 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3ffff640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 0x7f6f401a6c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802ae70 tx=0x7f6f38003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38034040 con 0x7f6f40115400 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f38003e10 con 0x7f6f40115400 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f401a76b0 con 0x7f6f40115400 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380043f0 con 0x7f6f40115400 2024-07-24T21:37:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.015+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401addd0 con 0x7f6f40115400 2024-07-24T21:37:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.016+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f40074300 con 0x7f6f40115400 2024-07-24T21:37:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.018+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f6f38037020 con 0x7f6f40115400 2024-07-24T21:37:15.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.019+0000 7f6f3cff9640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 0x7f6f280b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.019+0000 7f6f3f7fe640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 0x7f6f280b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.019+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6f38102d00 con 0x7f6f40115400 2024-07-24T21:37:15.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.019+0000 7f6f3f7fe640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 0x7f6f280b60d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009c90 tx=0x7f6f34005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:15.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.021+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6f380098b0 con 0x7f6f40115400 2024-07-24T21:37:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 -- 172.21.15.60:0/367205679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103550 msgr2=0x7fe724103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/367205679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103550 0x7fe724103930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe720009900 tx=0x7fe72002a410 comp rx=0 tx=0).stop 2024-07-24T21:37:15.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 -- 172.21.15.60:0/367205679 shutdown_connections 2024-07-24T21:37:15.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/367205679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 0x7fe72410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/367205679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103f50 0x7fe72410c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/367205679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103550 0x7fe724103930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.195+0000 7fe72b6e2640 1 -- 172.21.15.60:0/367205679 >> 172.21.15.60:0/367205679 conn(0x7fe7240fd120 msgr2=0x7fe7240ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.196+0000 7fe72b6e2640 1 -- 172.21.15.60:0/367205679 shutdown_connections 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.196+0000 7fe72b6e2640 1 -- 172.21.15.60:0/367205679 wait complete. 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 Processor -- start 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 -- start start 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103550 0x7fe724198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 0x7fe72419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe724112670 con 0x7fe724103f50 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241124f0 con 0x7fe724103550 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72b6e2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241127f0 con 0x7fe72410cb00 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50408/0 (socket says 172.21.15.60:50408) 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 -- 172.21.15.60:0/1271724890 learned_addr learned my addr 172.21.15.60:0/1271724890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe72947e640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103550 0x7fe724198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe729c7f640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 0x7fe72419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 -- 172.21.15.60:0/1271724890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 msgr2=0x7fe72419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 0x7fe72419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 -- 172.21.15.60:0/1271724890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103550 msgr2=0x7fe724198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103550 0x7fe724198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.197+0000 7fe728c7d640 1 -- 172.21.15.60:0/1271724890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72419ec50 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe728c7d640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9f0 tx=0x7fe71400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cd80 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7140108c0 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72419ef40 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140187d0 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.198+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe724101480 con 0x7fe724103f50 2024-07-24T21:37:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.199+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72406b310 con 0x7fe724103f50 2024-07-24T21:37:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.201+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fe714010430 con 0x7fe724103f50 2024-07-24T21:37:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.201+0000 7fe71a7fc640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 0x7fe7100b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.202+0000 7fe72947e640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 0x7fe7100b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.202+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe7140e2880 con 0x7fe724103f50 2024-07-24T21:37:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.202+0000 7fe72947e640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 0x7fe7100b60d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe720000c00 tx=0x7fe720005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:15.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.208+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe7140abad0 con 0x7fe724103f50 2024-07-24T21:37:15.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.224+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f6f401ae000 con 0x7f6f40115400 2024-07-24T21:37:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.230+0000 7f6f3cff9640 1 -- 172.21.15.60:0/1518623754 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v23)=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f6f380cbf40 con 0x7f6f40115400 2024-07-24T21:37:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 msgr2=0x7f6f280b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 0x7f6f280b60d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009c90 tx=0x7f6f34005f90 comp rx=0 tx=0).stop 2024-07-24T21:37:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 msgr2=0x7f6f401a6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 0x7f6f401a6c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802ae70 tx=0x7f6f38003c20 comp rx=0 tx=0).stop 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 shutdown_connections 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6f280b3c10 0x7f6f280b60d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f40115400 0x7f6f401a6c50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4010a460 0x7f6f401a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 --2- 172.21.15.60:0/1518623754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f40109b40 0x7f6f401a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 >> 172.21.15.60:0/1518623754 conn(0x7f6f4006d980 msgr2=0x7f6f40071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 shutdown_connections 2024-07-24T21:37:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.232+0000 7f6f461d8640 1 -- 172.21.15.60:0/1518623754 wait complete. 2024-07-24T21:37:15.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.417+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fe724103930 con 0x7fe724103f50 2024-07-24T21:37:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.421+0000 7fe71a7fc640 1 -- 172.21.15.60:0/1271724890 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fe7140ab3d0 con 0x7fe724103f50 2024-07-24T21:37:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 msgr2=0x7fe7100b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 0x7fe7100b60d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe720000c00 tx=0x7fe720005e80 comp rx=0 tx=0).stop 2024-07-24T21:37:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 msgr2=0x7fe7241990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9f0 tx=0x7fe71400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 shutdown_connections 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3c10 0x7fe7100b60d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe72410cb00 0x7fe72419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724103f50 0x7fe7241990e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 --2- 172.21.15.60:0/1271724890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724103550 0x7fe724198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 >> 172.21.15.60:0/1271724890 conn(0x7fe7240fd120 msgr2=0x7fe72410ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 shutdown_connections 2024-07-24T21:37:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:15.423+0000 7fe72b6e2640 1 -- 172.21.15.60:0/1271724890 wait complete. 2024-07-24T21:37:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:15 smithi123 ceph-mon[37143]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:15 smithi123 ceph-mon[37143]: from='client.? ' entity='client.admin' 2024-07-24T21:37:15.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:15 smithi060 ceph-mon[33000]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:15.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:15 smithi060 ceph-mon[33000]: from='client.? ' entity='client.admin' 2024-07-24T21:37:15.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:15 smithi086 ceph-mon[37128]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:15 smithi086 ceph-mon[37128]: from='client.? ' entity='client.admin' 2024-07-24T21:37:15.799 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-07-24T21:37:15.800 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-07-24T21:37:16.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:16 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1271724890' entity='client.admin' 2024-07-24T21:37:16.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:16 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1271724890' entity='client.admin' 2024-07-24T21:37:16.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:16 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1271724890' entity='client.admin' 2024-07-24T21:37:17.717 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:17.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 -- 172.21.15.60:0/3803755065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb81081c0 msgr2=0x7fbcb8108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/3803755065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb81081c0 0x7fbcb8108600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009900 tx=0x7fbca802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 -- 172.21.15.60:0/3803755065 shutdown_connections 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/3803755065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb8108b40 0x7fbcb810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/3803755065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb81081c0 0x7fbcb8108600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/3803755065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 0x7fbcb8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 -- 172.21.15.60:0/3803755065 >> 172.21.15.60:0/3803755065 conn(0x7fbcb80fe200 msgr2=0x7fbcb8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:19.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 -- 172.21.15.60:0/3803755065 shutdown_connections 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.242+0000 7fbcbfe12640 1 -- 172.21.15.60:0/3803755065 wait complete. 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 Processor -- start 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 -- start start 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 0x7fbcb8074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb81081c0 0x7fbcb80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112610 con 0x7fbcb8108b40 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112490 con 0x7fbcb81027b0 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbfe12640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112790 con 0x7fbcb81081c0 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbe3af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbe3af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36006/0 (socket says 172.21.15.60:36006) 2024-07-24T21:37:19.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbe3af640 1 -- 172.21.15.60:0/276286690 learned_addr learned my addr 172.21.15.60:0/276286690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbdbae640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 0x7fbcb8074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.243+0000 7fbcbd3ad640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb81081c0 0x7fbcb80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 -- 172.21.15.60:0/276286690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb81081c0 msgr2=0x7fbcb80753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb81081c0 0x7fbcb80753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 -- 172.21.15.60:0/276286690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 msgr2=0x7fbcb8074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 0x7fbcb8074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 -- 172.21.15.60:0/276286690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb81a6fa0 con 0x7fbcb8108b40 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbe3af640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000eb10 tx=0x7fbcb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004b20 con 0x7fbcb8108b40 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbcb0004cc0 con 0x7fbcb8108b40 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0010690 con 0x7fbcb8108b40 2024-07-24T21:37:19.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81a7290 con 0x7fbcb8108b40 2024-07-24T21:37:19.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.244+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a77a0 con 0x7fbcb8108b40 2024-07-24T21:37:19.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.245+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb8102b90 con 0x7fbcb8108b40 2024-07-24T21:37:19.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.247+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fbcb0005000 con 0x7fbcb8108b40 2024-07-24T21:37:19.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.247+0000 7fbcaeffd640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 0x7fbca40b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:19.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.248+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbcb00e2760 con 0x7fbcb8108b40 2024-07-24T21:37:19.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.248+0000 7fbcbdbae640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 0x7fbca40b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:19.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.248+0000 7fbcbdbae640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 0x7fbca40b60d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400bbe0 tx=0x7fbcb400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:19.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.250+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbcb00555e0 con 0x7fbcb8108b40 2024-07-24T21:37:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.453+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fbcb8076190 con 0x7fbcb8108b40 2024-07-24T21:37:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.457+0000 7fbcaeffd640 1 -- 172.21.15.60:0/276286690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7fbcb00ab9b0 con 0x7fbcb8108b40 2024-07-24T21:37:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 msgr2=0x7fbca40b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 0x7fbca40b60d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400bbe0 tx=0x7fbcb400b390 comp rx=0 tx=0).stop 2024-07-24T21:37:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 msgr2=0x7fbcb8075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000eb10 tx=0x7fbcb000c2d0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 shutdown_connections 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbca40b3c10 0x7fbca40b60d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcb8108b40 0x7fbcb8075910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcb81081c0 0x7fbcb80753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 --2- 172.21.15.60:0/276286690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb81027b0 0x7fbcb8074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 >> 172.21.15.60:0/276286690 conn(0x7fbcb80fe200 msgr2=0x7fbcb81005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 shutdown_connections 2024-07-24T21:37:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.459+0000 7fbcbfe12640 1 -- 172.21.15.60:0/276286690 wait complete. 2024-07-24T21:37:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/291384895' entity='client.admin' 2024-07-24T21:37:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/291384895' entity='client.admin' 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/291384895' entity='client.admin' 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:20.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/276286690' entity='client.admin' 2024-07-24T21:37:20.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/276286690' entity='client.admin' 2024-07-24T21:37:20.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/276286690' entity='client.admin' 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.072+0000 7f025d409640 1 -- 172.21.15.60:0/259494964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02581079e0 msgr2=0x7f025810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 --2- 172.21.15.60:0/259494964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02581079e0 0x7f025810e1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f025002a410 comp rx=0 tx=0).stop 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 -- 172.21.15.60:0/259494964 shutdown_connections 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 --2- 172.21.15.60:0/259494964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02581079e0 0x7f025810e1e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 --2- 172.21.15.60:0/259494964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258107060 0x7f02581074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 --2- 172.21.15.60:0/259494964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0258101650 0x7f0258101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 -- 172.21.15.60:0/259494964 >> 172.21.15.60:0/259494964 conn(0x7f02580fd0c0 msgr2=0x7f02580ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 -- 172.21.15.60:0/259494964 shutdown_connections 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 -- 172.21.15.60:0/259494964 wait complete. 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 Processor -- start 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.073+0000 7f025d409640 1 -- start start 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 0x7f02580fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 0x7f02580fe360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02581114f0 con 0x7f0258107060 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0258111370 con 0x7f02581079e0 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f025d409640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0258111670 con 0x7f0258101650 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 0x7f02580fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36024/0 (socket says 172.21.15.60:36024) 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 -- 172.21.15.60:0/650116405 learned_addr learned my addr 172.21.15.60:0/650116405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 0x7f02580fd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43228/0 (socket says 172.21.15.60:43228) 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02577fe640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 0x7f02580fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 -- 172.21.15.60:0/650116405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 msgr2=0x7f02580fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 0x7f02580fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 -- 172.21.15.60:0/650116405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 msgr2=0x7f02580fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 0x7f02580fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 -- 172.21.15.60:0/650116405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581a6f80 con 0x7f0258107060 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02577fe640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 0x7f02580fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.074+0000 7f02567fc640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f024000d950 tx=0x7f024000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.075+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240017070 con 0x7f0258107060 2024-07-24T21:37:21.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.075+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581a7270 con 0x7f0258107060 2024-07-24T21:37:21.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.075+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0240009c30 con 0x7f0258107060 2024-07-24T21:37:21.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.075+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024000f620 con 0x7f0258107060 2024-07-24T21:37:21.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.075+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a77b0 con 0x7f0258107060 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.076+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0258101a30 con 0x7f0258107060 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.077+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f0240010460 con 0x7f0258107060 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.078+0000 7f024ffff640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 0x7f02440b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.078+0000 7f0256ffd640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 0x7f02440b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.078+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f02400e1bb0 con 0x7f0258107060 2024-07-24T21:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.079+0000 7f0256ffd640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 0x7f02440b6050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0248005fd0 tx=0x7f0248009760 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.082+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f02400aae00 con 0x7f0258107060 2024-07-24T21:37:21.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.277+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f02580febe0 con 0x7f0258107060 2024-07-24T21:37:21.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.280+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) ==== 123+0+0 (secure 0 0 0) 0x7f02400a4e20 con 0x7f0258107060 2024-07-24T21:37:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.281+0000 7f024ffff640 1 -- 172.21.15.60:0/650116405 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02400e7080 con 0x7f0258107060 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 msgr2=0x7f02440b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 0x7f02440b6050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0248005fd0 tx=0x7f0248009760 comp rx=0 tx=0).stop 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 msgr2=0x7f02580fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f024000d950 tx=0x7f024000de10 comp rx=0 tx=0).stop 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 shutdown_connections 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02440b3b90 0x7f02440b6050 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02581079e0 0x7f02580fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0258107060 0x7f02580fde20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 --2- 172.21.15.60:0/650116405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0258101650 0x7f02580fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 >> 172.21.15.60:0/650116405 conn(0x7f02580fd0c0 msgr2=0x7f025806a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 shutdown_connections 2024-07-24T21:37:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.283+0000 7f025d409640 1 -- 172.21.15.60:0/650116405 wait complete. 2024-07-24T21:37:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 -- 172.21.15.60:0/2958358513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e41079e0 msgr2=0x7f31e410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 --2- 172.21.15.60:0/2958358513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e41079e0 0x7f31e410e1e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009900 tx=0x7f31dc02a410 comp rx=0 tx=0).stop 2024-07-24T21:37:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 -- 172.21.15.60:0/2958358513 shutdown_connections 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 --2- 172.21.15.60:0/2958358513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e41079e0 0x7f31e410e1e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 --2- 172.21.15.60:0/2958358513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e4107060 0x7f31e41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 --2- 172.21.15.60:0/2958358513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 0x7f31e4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 -- 172.21.15.60:0/2958358513 >> 172.21.15.60:0/2958358513 conn(0x7f31e40fd0c0 msgr2=0x7f31e40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 -- 172.21.15.60:0/2958358513 shutdown_connections 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.700+0000 7f31e9808640 1 -- 172.21.15.60:0/2958358513 wait complete. 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 Processor -- start 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 -- start start 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 0x7f31e41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e41079e0 0x7f31e419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4111610 con 0x7f31e4107060 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4111490 con 0x7f31e4101650 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.701+0000 7f31e9808640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4111790 con 0x7f31e41079e0 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36044/0 (socket says 172.21.15.60:36044) 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 -- 172.21.15.60:0/2350522096 learned_addr learned my addr 172.21.15.60:0/2350522096 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e2ffd640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 0x7f31e41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e37fe640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e41079e0 0x7f31e419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 -- 172.21.15.60:0/2350522096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e41079e0 msgr2=0x7f31e419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e41079e0 0x7f31e419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 -- 172.21.15.60:0/2350522096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 msgr2=0x7f31e41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 0x7f31e41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 -- 172.21.15.60:0/2350522096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e41a4da0 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e27fc640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d950 tx=0x7f31d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8017030 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d8004020 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d80047a0 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e41a5090 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.702+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a5620 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.703+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4101a30 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.705+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f31d8009a30 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.705+0000 7f31e8806640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 0x7f31cc0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.705+0000 7f31e2ffd640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 0x7f31cc0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.706+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f31d80e1a30 con 0x7f31e4107060 2024-07-24T21:37:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.706+0000 7f31e2ffd640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 0x7f31cc0b60d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007ec0 tx=0x7f31d40075b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.709+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f31d80aac80 con 0x7f31e4107060 2024-07-24T21:37:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:21 smithi123 ceph-mon[37143]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:21 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/650116405' entity='client.admin' 2024-07-24T21:37:21.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:21 smithi086 ceph-mon[37128]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:21 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/650116405' entity='client.admin' 2024-07-24T21:37:21.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:21 smithi060 ceph-mon[33000]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:21.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:21 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/650116405' entity='client.admin' 2024-07-24T21:37:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.915+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f31e4109230 con 0x7f31e4107060 2024-07-24T21:37:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.918+0000 7f31e8806640 1 -- 172.21.15.60:0/2350522096 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v28)=0 v28) ==== 125+0+0 (secure 0 0 0) 0x7f31d80aa580 con 0x7f31e4107060 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 msgr2=0x7f31cc0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 0x7f31cc0b60d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007ec0 tx=0x7f31d40075b0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 msgr2=0x7f31e4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d950 tx=0x7f31d800de10 comp rx=0 tx=0).stop 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 shutdown_connections 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31cc0b3c10 0x7f31cc0b60d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e41079e0 0x7f31e419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31e4107060 0x7f31e4199f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 --2- 172.21.15.60:0/2350522096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e4101650 0x7f31e41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 >> 172.21.15.60:0/2350522096 conn(0x7f31e40fd0c0 msgr2=0x7f31e40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 shutdown_connections 2024-07-24T21:37:21.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.921+0000 7f31e9808640 1 -- 172.21.15.60:0/2350522096 wait complete. 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 -- 172.21.15.60:0/4234773141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 msgr2=0x7f314c1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 --2- 172.21.15.60:0/4234773141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c1074a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3134009900 tx=0x7f313402a410 comp rx=0 tx=0).stop 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 -- 172.21.15.60:0/4234773141 shutdown_connections 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 --2- 172.21.15.60:0/4234773141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 0x7f314c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 --2- 172.21.15.60:0/4234773141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c1074a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 --2- 172.21.15.60:0/4234773141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 0x7f314c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 -- 172.21.15.60:0/4234773141 >> 172.21.15.60:0/4234773141 conn(0x7f314c0fd0c0 msgr2=0x7f314c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 -- 172.21.15.60:0/4234773141 shutdown_connections 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.151+0000 7f3151177640 1 -- 172.21.15.60:0/4234773141 wait complete. 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 Processor -- start 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 -- start start 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 0x7f314c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 0x7f314c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f314c1126a0 con 0x7f314c107060 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112520 con 0x7f314c1079e0 2024-07-24T21:37:21.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f3151177640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112820 con 0x7f314c101650 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35996/0 (socket says 172.21.15.60:35996) 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 -- 172.21.15.60:0/291384895 learned_addr learned my addr 172.21.15.60:0/291384895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314ad9d640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 0x7f314c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314b59e640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 0x7f314c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 -- 172.21.15.60:0/291384895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 msgr2=0x7f314c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 0x7f314c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 -- 172.21.15.60:0/291384895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 msgr2=0x7f314c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.152+0000 7f314a59c640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 0x7f314c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f314a59c640 1 -- 172.21.15.60:0/291384895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c19eca0 con 0x7f314c107060 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f314a59c640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f313402aa00 tx=0x7f3134003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313403f070 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f313403bb00 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c19ef30 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134035030 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.153+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1a5650 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.154+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c101a30 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.156+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f3134042070 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.156+0000 7f3143fff640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 0x7f31380b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.157+0000 7f314ad9d640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 0x7f31380b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.157+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3134102b10 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.157+0000 7f314ad9d640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 0x7f31380b60d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3144005fd0 tx=0x7f314400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.160+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f31340cbd60 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.359+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f314c199d20 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.363+0000 7f3143fff640 1 -- 172.21.15.60:0/291384895 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v25)=0 v25) ==== 138+0+0 (secure 0 0 0) 0x7f314c199d20 con 0x7f314c107060 2024-07-24T21:37:21.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 msgr2=0x7f31380b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 0x7f31380b60d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3144005fd0 tx=0x7f314400b390 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 msgr2=0x7f314c199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f313402aa00 tx=0x7f3134003c10 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 shutdown_connections 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31380b3c10 0x7f31380b60d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c1079e0 0x7f314c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314c107060 0x7f314c199130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 --2- 172.21.15.60:0/291384895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f314c101650 0x7f314c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.365+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 >> 172.21.15.60:0/291384895 conn(0x7f314c0fd0c0 msgr2=0x7f314c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.366+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 shutdown_connections 2024-07-24T21:37:21.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:19.366+0000 7f3151177640 1 -- 172.21.15.60:0/291384895 wait complete. 2024-07-24T21:37:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1581048577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 msgr2=0x7fd2c0101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1581048577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c0101a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009900 tx=0x7fd2b002a410 comp rx=0 tx=0).stop 2024-07-24T21:37:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1581048577 shutdown_connections 2024-07-24T21:37:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1581048577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1581048577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1581048577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c0101a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1581048577 >> 172.21.15.60:0/1581048577 conn(0x7fd2c00fd0c0 msgr2=0x7fd2c00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1581048577 shutdown_connections 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.482+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1581048577 wait complete. 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 Processor -- start 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 -- start start 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c00fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:22.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c00fe310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0111610 con 0x7fd2c0101650 2024-07-24T21:37:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0111490 con 0x7fd2c0107060 2024-07-24T21:37:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2c4b27640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0111790 con 0x7fd2c01079e0 2024-07-24T21:37:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.483+0000 7fd2be59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36072/0 (socket says 172.21.15.60:36072) 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 -- 172.21.15.60:0/1269641192 learned_addr learned my addr 172.21.15.60:0/1269641192 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2bed9d640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c00fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2b5d9b640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c00fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 -- 172.21.15.60:0/1269641192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 msgr2=0x7fd2c00fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c00fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 -- 172.21.15.60:0/1269641192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 msgr2=0x7fd2c00fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c00fddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 -- 172.21.15.60:0/1269641192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c01a7070 con 0x7fd2c0101650 2024-07-24T21:37:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2bed9d640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c00fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:37:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2b5d9b640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c00fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:37:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2be59c640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005940 tx=0x7fd2b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b003f030 con 0x7fd2c0101650 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c01a7300 con 0x7fd2c0101650 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b0003aa0 con 0x7fd2c0101650 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.484+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0035070 con 0x7fd2c0101650 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.485+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c01a7830 con 0x7fd2c0101650 2024-07-24T21:37:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.486+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c0101a30 con 0x7fd2c0101650 2024-07-24T21:37:22.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.487+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd2b0042030 con 0x7fd2c0101650 2024-07-24T21:37:22.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.488+0000 7fd2b7fff640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 0x7fd2ac0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:22.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.488+0000 7fd2b5d9b640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 0x7fd2ac0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:22.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.488+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd2b0102800 con 0x7fd2c0101650 2024-07-24T21:37:22.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.488+0000 7fd2b5d9b640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 0x7fd2ac0b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00ff380 tx=0x7fd2a800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:22.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.491+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd2b00cb400 con 0x7fd2c0101650 2024-07-24T21:37:22.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.688+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fd2c00feb90 con 0x7fd2c0101650 2024-07-24T21:37:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.692+0000 7fd2b7fff640 1 -- 172.21.15.60:0/1269641192 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v30)=0 v30) ==== 115+0+0 (secure 0 0 0) 0x7fd2b00cf950 con 0x7fd2c0101650 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 msgr2=0x7fd2ac0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 0x7fd2ac0b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00ff380 tx=0x7fd2a800b390 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 msgr2=0x7fd2c00fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005940 tx=0x7fd2b0004270 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 shutdown_connections 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd2ac0b3c10 0x7fd2ac0b60d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c01079e0 0x7fd2c00fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2c0107060 0x7fd2c00fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.694+0000 7fd2c4b27640 1 --2- 172.21.15.60:0/1269641192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c0101650 0x7fd2c00fd890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.695+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 >> 172.21.15.60:0/1269641192 conn(0x7fd2c00fd0c0 msgr2=0x7fd2c006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.695+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 shutdown_connections 2024-07-24T21:37:22.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.695+0000 7fd2c4b27640 1 -- 172.21.15.60:0/1269641192 wait complete. 2024-07-24T21:37:23.195 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2350522096' entity='client.admin' 2024-07-24T21:37:23.196 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:22 smithi060 ceph-mon[33000]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:23.196 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3238837394' entity='client.admin' 2024-07-24T21:37:23.196 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1269641192' entity='client.admin' 2024-07-24T21:37:23.196 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3658710081' entity='client.admin' 2024-07-24T21:37:23.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2350522096' entity='client.admin' 2024-07-24T21:37:23.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:22 smithi123 ceph-mon[37143]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:23.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3238837394' entity='client.admin' 2024-07-24T21:37:23.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1269641192' entity='client.admin' 2024-07-24T21:37:23.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3658710081' entity='client.admin' 2024-07-24T21:37:23.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2350522096' entity='client.admin' 2024-07-24T21:37:23.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:22 smithi086 ceph-mon[37128]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:23.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3238837394' entity='client.admin' 2024-07-24T21:37:23.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1269641192' entity='client.admin' 2024-07-24T21:37:23.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3658710081' entity='client.admin' 2024-07-24T21:37:23.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 -- 172.21.15.60:0/2869704485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 msgr2=0x7f7f9c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/2869704485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c1038d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009900 tx=0x7f7f9802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 -- 172.21.15.60:0/2869704485 shutdown_connections 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/2869704485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c10caa0 0x7f7f9c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/2869704485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c103ef0 0x7f7f9c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/2869704485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c1038d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 -- 172.21.15.60:0/2869704485 >> 172.21.15.60:0/2869704485 conn(0x7f7f9c0fd0c0 msgr2=0x7f7f9c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 -- 172.21.15.60:0/2869704485 shutdown_connections 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.657+0000 7f7fa38b4640 1 -- 172.21.15.60:0/2869704485 wait complete. 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 Processor -- start 2024-07-24T21:37:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 -- start start 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c103ef0 0x7f7f9c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c10caa0 0x7f7f9c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c1127e0 con 0x7f7f9c1034f0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c112660 con 0x7f7f9c10caa0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa38b4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c112960 con 0x7f7f9c103ef0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa1650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa1650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36088/0 (socket says 172.21.15.60:36088) 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.658+0000 7f7fa1650640 1 -- 172.21.15.60:0/3658710081 learned_addr learned my addr 172.21.15.60:0/3658710081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1e51640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c10caa0 0x7f7f9c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa0e4f640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c103ef0 0x7f7f9c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 -- 172.21.15.60:0/3658710081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c103ef0 msgr2=0x7f7f9c1018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c103ef0 0x7f7f9c1018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 -- 172.21.15.60:0/3658710081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c10caa0 msgr2=0x7f7f9c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c10caa0 0x7f7f9c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 -- 172.21.15.60:0/3658710081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c1a71d0 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa1650640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f98006010 tx=0x7f7f980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9803f030 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f98003800 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98035070 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9c1a7460 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.659+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9c1a7970 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.660+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9c103950 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.662+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f7f98003cb0 con 0x7f7f9c1034f0 2024-07-24T21:37:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.663+0000 7f7f927fc640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 0x7f7f880b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.663+0000 7f7fa0e4f640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 0x7f7f880b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.663+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7f98102bd0 con 0x7f7f9c1034f0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.664+0000 7f7fa0e4f640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 0x7f7f880b60d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c102e60 tx=0x7f7f8c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.667+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7f980cbd80 con 0x7f7f9c1034f0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.867+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f7f9c02e000 con 0x7f7f9c1034f0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.871+0000 7f7f927fc640 1 -- 172.21.15.60:0/3658710081 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f7f980cb680 con 0x7f7f9c1034f0 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 msgr2=0x7f7f880b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 0x7f7f880b60d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c102e60 tx=0x7f7f8c008040 comp rx=0 tx=0).stop 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 msgr2=0x7f7f9c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f98006010 tx=0x7f7f980042f0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 shutdown_connections 2024-07-24T21:37:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f880b3c10 0x7f7f880b60d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f9c10caa0 0x7f7f9c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f9c103ef0 0x7f7f9c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 --2- 172.21.15.60:0/3658710081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f9c1034f0 0x7f7f9c101370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 >> 172.21.15.60:0/3658710081 conn(0x7f7f9c0fd0c0 msgr2=0x7f7f9c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 shutdown_connections 2024-07-24T21:37:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.873+0000 7f7fa38b4640 1 -- 172.21.15.60:0/3658710081 wait complete. 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.269+0000 7ffba049d640 1 -- 172.21.15.60:0/859813281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9810ed70 msgr2=0x7ffb98111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.269+0000 7ffba049d640 1 --2- 172.21.15.60:0/859813281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9810ed70 0x7ffb98111160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009ca0 tx=0x7ffb9002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 -- 172.21.15.60:0/859813281 shutdown_connections 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 --2- 172.21.15.60:0/859813281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9810ed70 0x7ffb98111160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 --2- 172.21.15.60:0/859813281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb98075930 0x7ffb9810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 --2- 172.21.15.60:0/859813281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98074f30 0x7ffb98075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 -- 172.21.15.60:0/859813281 >> 172.21.15.60:0/859813281 conn(0x7ffb980fe200 msgr2=0x7ffb98100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 -- 172.21.15.60:0/859813281 shutdown_connections 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 -- 172.21.15.60:0/859813281 wait complete. 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 Processor -- start 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 -- start start 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.270+0000 7ffba049d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98075930 0x7ffb9819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 0x7ffb981a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98114920 con 0x7ffb98074f30 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981147a0 con 0x7ffb98075930 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98114aa0 con 0x7ffb9810ed70 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36134/0 (socket says 172.21.15.60:36134) 2024-07-24T21:37:25.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 -- 172.21.15.60:0/847621364 learned_addr learned my addr 172.21.15.60:0/847621364 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9ea3a640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 0x7ffb981a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9da38640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98075930 0x7ffb9819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 -- 172.21.15.60:0/847621364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 msgr2=0x7ffb981a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 0x7ffb981a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 -- 172.21.15.60:0/847621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98075930 msgr2=0x7ffb9819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98075930 0x7ffb9819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 -- 172.21.15.60:0/847621364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb981a0f40 con 0x7ffb98074f30 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9ea3a640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 0x7ffb981a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb9e239640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400d950 tx=0x7ffb9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94017070 con 0x7ffb98074f30 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb94009c30 con 0x7ffb98074f30 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb981a1230 con 0x7ffb98074f30 2024-07-24T21:37:25.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400f620 con 0x7ffb98074f30 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.271+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb981a7ac0 con 0x7ffb98074f30 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.272+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb98075310 con 0x7ffb98074f30 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.275+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7ffb94010460 con 0x7ffb98074f30 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.275+0000 7ffb8f7fe640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 0x7ffb840b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.275+0000 7ffb9da38640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 0x7ffb840b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.275+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ffb940e1c70 con 0x7ffb98074f30 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.276+0000 7ffb9da38640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 0x7ffb840b60d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb88005fd0 tx=0x7ffb88005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:25.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.278+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ffb940aaf40 con 0x7ffb98074f30 2024-07-24T21:37:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.472+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ffb9819bfc0 con 0x7ffb98074f30 2024-07-24T21:37:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.477+0000 7ffb8f7fe640 1 -- 172.21.15.60:0/847621364 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v33)=0 v33) ==== 120+0+0 (secure 0 0 0) 0x7ffb94009de0 con 0x7ffb98074f30 2024-07-24T21:37:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 msgr2=0x7ffb840b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 0x7ffb840b60d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb88005fd0 tx=0x7ffb88005850 comp rx=0 tx=0).stop 2024-07-24T21:37:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 msgr2=0x7ffb9819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400d950 tx=0x7ffb9400de10 comp rx=0 tx=0).stop 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 shutdown_connections 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb840b3c10 0x7ffb840b60d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb9810ed70 0x7ffb981a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb98075930 0x7ffb9819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 --2- 172.21.15.60:0/847621364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb98074f30 0x7ffb9819ae90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 >> 172.21.15.60:0/847621364 conn(0x7ffb980fe200 msgr2=0x7ffb980fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 shutdown_connections 2024-07-24T21:37:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:25.479+0000 7ffba049d640 1 -- 172.21.15.60:0/847621364 wait complete. 2024-07-24T21:37:25.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:25 smithi060 ceph-mon[33000]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:25.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:25 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/352468014' entity='client.admin' 2024-07-24T21:37:25.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:25 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/847621364' entity='client.admin' 2024-07-24T21:37:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:25 smithi123 ceph-mon[37143]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:25 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/352468014' entity='client.admin' 2024-07-24T21:37:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:25 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/847621364' entity='client.admin' 2024-07-24T21:37:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:25 smithi086 ceph-mon[37128]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:25 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/352468014' entity='client.admin' 2024-07-24T21:37:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:25 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/847621364' entity='client.admin' 2024-07-24T21:37:26.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 -- 172.21.15.60:0/1933795870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 msgr2=0x7f53941015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 --2- 172.21.15.60:0/1933795870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f53941015a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5390009900 tx=0x7f539002a410 comp rx=0 tx=0).stop 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 -- 172.21.15.60:0/1933795870 shutdown_connections 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 --2- 172.21.15.60:0/1933795870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 --2- 172.21.15.60:0/1933795870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 0x7f5394106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 --2- 172.21.15.60:0/1933795870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f53941015a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 -- 172.21.15.60:0/1933795870 >> 172.21.15.60:0/1933795870 conn(0x7f5394076980 msgr2=0x7f5394076d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 -- 172.21.15.60:0/1933795870 shutdown_connections 2024-07-24T21:37:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.218+0000 7f539b50c640 1 -- 172.21.15.60:0/1933795870 wait complete. 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 Processor -- start 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 -- start start 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 0x7f5394199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5394111700 con 0x7f53941011c0 2024-07-24T21:37:26.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5394111580 con 0x7f5394107170 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f539b50c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5394111880 con 0x7f53941067f0 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f53992a8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f53992a8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46038/0 (socket says 172.21.15.60:46038) 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f53992a8640 1 -- 172.21.15.60:0/1835500168 learned_addr learned my addr 172.21.15.60:0/1835500168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f5398aa7640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 0x7f5394199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.219+0000 7f5399aa9640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 -- 172.21.15.60:0/1835500168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 msgr2=0x7f5394199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 0x7f5394199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 -- 172.21.15.60:0/1835500168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 msgr2=0x7f539419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 -- 172.21.15.60:0/1835500168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539419ecb0 con 0x7f53941011c0 2024-07-24T21:37:26.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f5399aa9640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f53992a8640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5390005ab0 tx=0x7f5390003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539003f070 con 0x7f53941011c0 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5390031d60 con 0x7f53941011c0 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539419ef40 con 0x7f53941011c0 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390035030 con 0x7f53941011c0 2024-07-24T21:37:26.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.220+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5394102580 con 0x7f53941011c0 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.221+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53941015a0 con 0x7f53941011c0 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.223+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f5390042070 con 0x7f53941011c0 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.223+0000 7f538a7fc640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 0x7f53800b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.224+0000 7f5398aa7640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 0x7f53800b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.224+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5390102c30 con 0x7f53941011c0 2024-07-24T21:37:26.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.224+0000 7f5398aa7640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 0x7f53800b60d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53940ff5d0 tx=0x7f538400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:26.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.226+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f53900cbe80 con 0x7f53941011c0 2024-07-24T21:37:26.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.422+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f5394063740 con 0x7f53941011c0 2024-07-24T21:37:26.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.425+0000 7f538a7fc640 1 -- 172.21.15.60:0/1835500168 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v34)=0 v34) ==== 117+0+0 (secure 0 0 0) 0x7f53900cb780 con 0x7f53941011c0 2024-07-24T21:37:26.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 msgr2=0x7f53800b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 0x7f53800b60d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53940ff5d0 tx=0x7f538400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 msgr2=0x7f5394198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5390005ab0 tx=0x7f5390003d50 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 shutdown_connections 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53800b3c10 0x7f53800b60d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5394107170 0x7f539419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53941067f0 0x7f5394199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 --2- 172.21.15.60:0/1835500168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53941011c0 0x7f5394198c00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 >> 172.21.15.60:0/1835500168 conn(0x7f5394076980 msgr2=0x7f5394100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 shutdown_connections 2024-07-24T21:37:26.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:26.428+0000 7f539b50c640 1 -- 172.21.15.60:0/1835500168 wait complete. 2024-07-24T21:37:26.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/1915604477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 msgr2=0x7f2bc010c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/1915604477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc010c530 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009900 tx=0x7f2ba802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/1915604477 shutdown_connections 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/1915604477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc010cb00 0x7f2bc010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/1915604477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc010c530 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/1915604477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0103550 0x7f2bc0103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/1915604477 >> 172.21.15.60:0/1915604477 conn(0x7f2bc00fd0c0 msgr2=0x7f2bc00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/1915604477 shutdown_connections 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/1915604477 wait complete. 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 Processor -- start 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.888+0000 7f2bc5ea2640 1 -- start start 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc0103550 0x7f2bc0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 0x7f2bc019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0112700 con 0x7f2bc0103f50 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0112580 con 0x7f2bc010cb00 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bc5ea2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0112880 con 0x7f2bc0103550 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36064/0 (socket says 172.21.15.60:36064) 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 -- 172.21.15.60:0/3238837394 learned_addr learned my addr 172.21.15.60:0/3238837394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbf7fe640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc0103550 0x7f2bc0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbffff640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 0x7f2bc019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 -- 172.21.15.60:0/3238837394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc0103550 msgr2=0x7f2bc0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc0103550 0x7f2bc0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 -- 172.21.15.60:0/3238837394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 msgr2=0x7f2bc019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 0x7f2bc019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 -- 172.21.15.60:0/3238837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc019ed20 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbffff640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 0x7f2bc019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbeffd640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8005940 tx=0x7f2ba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.889+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba803f030 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.890+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba8003cb0 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.890+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8035070 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.890+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc019efb0 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.890+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a56d0 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.891+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc0103930 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.892+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f2ba8042030 con 0x7f2bc0103f50 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.893+0000 7f2bbcff9640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 0x7f2bac0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.893+0000 7f2bbf7fe640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 0x7f2bac0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.893+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2ba8102b40 con 0x7f2bc0103f50 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.893+0000 7f2bbf7fe640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 0x7f2bac0b60d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000bbe0 tx=0x7f2bb000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:21.896+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2ba80cbd90 con 0x7f2bc0103f50 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.094+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f2bc0199da0 con 0x7f2bc0103f50 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.098+0000 7f2bbcff9640 1 -- 172.21.15.60:0/3238837394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v29)=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f2bc0199da0 con 0x7f2bc0103f50 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 msgr2=0x7f2bac0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 0x7f2bac0b60d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000bbe0 tx=0x7f2bb000b390 comp rx=0 tx=0).stop 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 msgr2=0x7f2bc01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8005940 tx=0x7f2ba80042f0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 shutdown_connections 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2bac0b3c10 0x7f2bac0b60d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc010cb00 0x7f2bc019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bc0103f50 0x7f2bc01991b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 --2- 172.21.15.60:0/3238837394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc0103550 0x7f2bc0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 >> 172.21.15.60:0/3238837394 conn(0x7f2bc00fd0c0 msgr2=0x7f2bc00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 shutdown_connections 2024-07-24T21:37:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:22.100+0000 7f2bc5ea2640 1 -- 172.21.15.60:0/3238837394 wait complete. 2024-07-24T21:37:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 -- 172.21.15.60:0/3097667417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8102790 msgr2=0x7f4db8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/3097667417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8102790 0x7f4db8102b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009900 tx=0x7f4da802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 -- 172.21.15.60:0/3097667417 shutdown_connections 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/3097667417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8108b20 0x7f4db810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/3097667417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 0x7f4db81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/3097667417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8102790 0x7f4db8102b70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 -- 172.21.15.60:0/3097667417 >> 172.21.15.60:0/3097667417 conn(0x7f4db80fe200 msgr2=0x7f4db8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 -- 172.21.15.60:0/3097667417 shutdown_connections 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.400+0000 7f4dbcc98640 1 -- 172.21.15.60:0/3097667417 wait complete. 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.401+0000 7f4dbcc98640 1 Processor -- start 2024-07-24T21:37:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.401+0000 7f4dbcc98640 1 -- start start 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8102790 0x7f4db8074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 0x7f4db80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4db8112720 con 0x7f4db8108b20 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81125a0 con 0x7f4db8102790 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4dbcc98640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81128a0 con 0x7f4db81081a0 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36110/0 (socket says 172.21.15.60:36110) 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 -- 172.21.15.60:0/352468014 learned_addr learned my addr 172.21.15.60:0/352468014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db659c640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8102790 0x7f4db8074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db5d9b640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 0x7f4db80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 -- 172.21.15.60:0/352468014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 msgr2=0x7f4db80753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 0x7f4db80753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 -- 172.21.15.60:0/352468014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8102790 msgr2=0x7f4db8074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8102790 0x7f4db8074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 -- 172.21.15.60:0/352468014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db81a7010 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.402+0000 7f4db6d9d640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4db000eaa0 tx=0x7f4db000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.403+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000cdb0 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.403+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db81a7300 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.403+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db0005020 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.403+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db00116d0 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.403+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db81a7840 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.404+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db8102b70 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.405+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f4db00051d0 con 0x7f4db8108b20 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.406+0000 7f4da77fe640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 0x7f4da00b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.406+0000 7f4db659c640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 0x7f4da00b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.406+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4db00e2110 con 0x7f4db8108b20 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.407+0000 7f4db659c640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 0x7f4da00b6050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4da8031440 tx=0x7f4da8005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.410+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4db0016030 con 0x7f4db8108b20 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.613+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f4db810a960 con 0x7f4db8108b20 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.617+0000 7f4da77fe640 1 -- 172.21.15.60:0/352468014 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f4db00aad10 con 0x7f4db8108b20 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 msgr2=0x7f4da00b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 0x7f4da00b6050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4da8031440 tx=0x7f4da8005b50 comp rx=0 tx=0).stop 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 msgr2=0x7f4db8075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4db000eaa0 tx=0x7f4db000ef60 comp rx=0 tx=0).stop 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 shutdown_connections 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4da00b3b90 0x7f4da00b6050 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db8108b20 0x7f4db8075910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db81081a0 0x7f4db80753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 --2- 172.21.15.60:0/352468014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db8102790 0x7f4db8074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 >> 172.21.15.60:0/352468014 conn(0x7f4db80fe200 msgr2=0x7f4db80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:27.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 shutdown_connections 2024-07-24T21:37:27.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:24.620+0000 7f4dbcc98640 1 -- 172.21.15.60:0/352468014 wait complete. 2024-07-24T21:37:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:27 smithi123 ceph-mon[37143]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1835500168' entity='client.admin' 2024-07-24T21:37:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:27 smithi086 ceph-mon[37128]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1835500168' entity='client.admin' 2024-07-24T21:37:27.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:27 smithi060 ceph-mon[33000]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:27.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1835500168' entity='client.admin' 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 -- 172.21.15.60:0/2473829191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170102790 msgr2=0x7ff170102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 --2- 172.21.15.60:0/2473829191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170102790 0x7ff170102b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff16002a410 comp rx=0 tx=0).stop 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 -- 172.21.15.60:0/2473829191 shutdown_connections 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 --2- 172.21.15.60:0/2473829191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff170108b20 0x7ff17010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 --2- 172.21.15.60:0/2473829191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1701081a0 0x7ff1701085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 --2- 172.21.15.60:0/2473829191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170102790 0x7ff170102b70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 -- 172.21.15.60:0/2473829191 >> 172.21.15.60:0/2473829191 conn(0x7ff1700fe200 msgr2=0x7ff170100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 -- 172.21.15.60:0/2473829191 shutdown_connections 2024-07-24T21:37:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.965+0000 7ff175498640 1 -- 172.21.15.60:0/2473829191 wait complete. 2024-07-24T21:37:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 Processor -- start 2024-07-24T21:37:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 -- start start 2024-07-24T21:37:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff170102790 0x7ff170198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 0x7ff1701991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff170112790 con 0x7ff170108b20 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff170112610 con 0x7ff1701081a0 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.966+0000 7ff175498640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff170112910 con 0x7ff170102790 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46062/0 (socket says 172.21.15.60:46062) 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 -- 172.21.15.60:0/3027571304 learned_addr learned my addr 172.21.15.60:0/3027571304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16effd640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff170102790 0x7ff170198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16e7fc640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 0x7ff1701991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 -- 172.21.15.60:0/3027571304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff170102790 msgr2=0x7ff170198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff170102790 0x7ff170198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 -- 172.21.15.60:0/3027571304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 msgr2=0x7ff1701991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 0x7ff1701991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 -- 172.21.15.60:0/3027571304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17019ed30 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16e7fc640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 0x7ff1701991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff16f7fe640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff168009fd0 tx=0x7ff16800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168017070 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17019f020 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff168003e20 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168010460 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.967+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1701a56e0 con 0x7ff170108b20 2024-07-24T21:37:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.968+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff170102b70 con 0x7ff170108b20 2024-07-24T21:37:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.970+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7ff168003970 con 0x7ff170108b20 2024-07-24T21:37:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.971+0000 7ff157fff640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 0x7ff1580b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.971+0000 7ff16effd640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 0x7ff1580b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.971+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff1680e1d40 con 0x7ff170108b20 2024-07-24T21:37:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.971+0000 7ff16effd640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 0x7ff1580b60d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff160005ab0 tx=0x7ff160005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:27.975+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff1680aaf90 con 0x7ff170108b20 2024-07-24T21:37:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.170+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7ff17010a930 con 0x7ff170108b20 2024-07-24T21:37:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.174+0000 7ff157fff640 1 -- 172.21.15.60:0/3027571304 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v35)=0 v35) ==== 116+0+0 (secure 0 0 0) 0x7ff1680b3260 con 0x7ff170108b20 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.176+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 msgr2=0x7ff1580b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.176+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 0x7ff1580b60d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff160005ab0 tx=0x7ff160005a20 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.176+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 msgr2=0x7ff17019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.176+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff168009fd0 tx=0x7ff16800b260 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 shutdown_connections 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1580b3c10 0x7ff1580b60d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff170108b20 0x7ff17019e560 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1701081a0 0x7ff1701991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 --2- 172.21.15.60:0/3027571304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff170102790 0x7ff170198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 >> 172.21.15.60:0/3027571304 conn(0x7ff1700fe200 msgr2=0x7ff1701005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:28.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 shutdown_connections 2024-07-24T21:37:28.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:28.177+0000 7ff175498640 1 -- 172.21.15.60:0/3027571304 wait complete. 2024-07-24T21:37:29.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:29 smithi123 ceph-mon[37143]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:29.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3027571304' entity='client.admin' 2024-07-24T21:37:29.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:29 smithi086 ceph-mon[37128]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:29.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3027571304' entity='client.admin' 2024-07-24T21:37:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:29 smithi060 ceph-mon[33000]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3027571304' entity='client.admin' 2024-07-24T21:37:30.729 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi060') 2024-07-24T21:37:30.730 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi060') 2024-07-24T21:37:31.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:31 smithi060 ceph-mon[33000]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:31.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:31 smithi123 ceph-mon[37143]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:31.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:31 smithi086 ceph-mon[37128]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:31.751 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '646042b4-4a04-11ef-bcad-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-07-24T21:37:32.208 INFO:tasks.cephadm:Deploying OSDs... 2024-07-24T21:37:32.208 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:37:32.208 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-07-24T21:37:32.228 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-24T21:37:32.228 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-07-24 21:35:35.364235764 +0000 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-07-24 21:31:11.873699992 +0000 2024-07-24T21:37:32.286 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-07-24 21:31:11.873699992 +0000 2024-07-24T21:37:32.287 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-07-24 21:31:11.873699992 +0000 2024-07-24T21:37:32.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-24T21:37:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-07-24T21:37:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-07-24T21:37:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000123921 s, 4.1 MB/s 2024-07-24T21:37:32.357 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-24T21:37:32.415 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-07-24 21:35:35.395235490 +0000 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-07-24 21:31:12.201697178 +0000 2024-07-24T21:37:32.473 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-07-24 21:31:12.201697178 +0000 2024-07-24T21:37:32.474 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-07-24 21:31:12.201697178 +0000 2024-07-24T21:37:32.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-24T21:37:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-07-24T21:37:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-07-24T21:37:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000111732 s, 4.6 MB/s 2024-07-24T21:37:32.540 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-24T21:37:32.597 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-07-24 21:35:35.424235233 +0000 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-07-24 21:31:12.533694331 +0000 2024-07-24T21:37:32.654 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-07-24 21:31:12.533694331 +0000 2024-07-24T21:37:32.655 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-07-24 21:31:12.533694331 +0000 2024-07-24T21:37:32.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-24T21:37:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-07-24T21:37:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-07-24T21:37:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.2756e-05 s, 5.5 MB/s 2024-07-24T21:37:32.720 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-24T21:37:32.777 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-07-24T21:37:32.833 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-07-24 21:35:35.453234977 +0000 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-07-24 21:31:12.847691637 +0000 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-07-24 21:31:12.847691637 +0000 2024-07-24T21:37:32.834 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-07-24 21:31:12.847691637 +0000 2024-07-24T21:37:32.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-24T21:37:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-07-24T21:37:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-07-24T21:37:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000109908 s, 4.7 MB/s 2024-07-24T21:37:32.899 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-24T21:37:32.957 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-07-24T21:37:32.957 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-07-24T21:37:32.976 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-24T21:37:32.976 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-07-24T21:37:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-24T21:37:33.034 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-07-24 21:36:44.001402333 +0000 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-07-24 21:31:16.379064900 +0000 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-07-24 21:31:16.379064900 +0000 2024-07-24T21:37:33.035 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-07-24 21:31:16.379064900 +0000 2024-07-24T21:37:33.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-24T21:37:33.103 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-07-24T21:37:33.103 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-07-24T21:37:33.103 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000303287 s, 1.7 MB/s 2024-07-24T21:37:33.105 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-24T21:37:33.162 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-07-24 21:36:44.001402333 +0000 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-07-24 21:31:16.728057384 +0000 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-07-24 21:31:16.728057384 +0000 2024-07-24T21:37:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-07-24 21:31:16.728057384 +0000 2024-07-24T21:37:33.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-24T21:37:33.284 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-07-24T21:37:33.284 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-07-24T21:37:33.284 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000209008 s, 2.4 MB/s 2024-07-24T21:37:33.285 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-24T21:37:33.342 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-07-24 21:36:44.001402333 +0000 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-07-24 21:31:17.067050083 +0000 2024-07-24T21:37:33.398 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-07-24 21:31:17.067050083 +0000 2024-07-24T21:37:33.399 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-07-24 21:31:17.067050083 +0000 2024-07-24T21:37:33.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-24T21:37:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:33 smithi086 ceph-mon[37128]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-07-24T21:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-07-24T21:37:33.462 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00027262 s, 1.9 MB/s 2024-07-24T21:37:33.463 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-24T21:37:33.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:33 smithi123 ceph-mon[37143]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:33.519 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-07-24T21:37:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:33 smithi060 ceph-mon[33000]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:33.574 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-24T21:37:33.574 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.574 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-07-24 21:36:44.002402312 +0000 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-07-24 21:31:17.399042933 +0000 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-07-24 21:31:17.399042933 +0000 2024-07-24T21:37:33.575 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-07-24 21:31:17.399042933 +0000 2024-07-24T21:37:33.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-24T21:37:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-07-24T21:37:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-07-24T21:37:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000213205 s, 2.4 MB/s 2024-07-24T21:37:33.638 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-24T21:37:33.694 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-07-24T21:37:33.694 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-07-24T21:37:33.713 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-24T21:37:33.713 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-07-24 21:36:41.777216372 +0000 2024-07-24T21:37:33.771 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-07-24 21:31:15.886701426 +0000 2024-07-24T21:37:33.772 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-07-24 21:31:15.886701426 +0000 2024-07-24T21:37:33.772 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-07-24 21:31:15.886701426 +0000 2024-07-24T21:37:33.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-24T21:37:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-07-24T21:37:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-07-24T21:37:33.841 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000286154 s, 1.8 MB/s 2024-07-24T21:37:33.842 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-24T21:37:33.900 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-07-24T21:37:33.957 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-24T21:37:33.957 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-07-24 21:36:41.777216372 +0000 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-07-24 21:31:16.220693531 +0000 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-07-24 21:31:16.220693531 +0000 2024-07-24T21:37:33.958 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-07-24 21:31:16.220693531 +0000 2024-07-24T21:37:33.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-24T21:37:34.023 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-07-24T21:37:34.023 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-07-24T21:37:34.023 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000230857 s, 2.2 MB/s 2024-07-24T21:37:34.025 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-24T21:37:34.082 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-07-24 21:36:41.778216349 +0000 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-07-24 21:31:16.555685611 +0000 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-07-24 21:31:16.555685611 +0000 2024-07-24T21:37:34.141 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-07-24 21:31:16.555685611 +0000 2024-07-24T21:37:34.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-24T21:37:34.207 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-07-24T21:37:34.207 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-07-24T21:37:34.207 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000228675 s, 2.2 MB/s 2024-07-24T21:37:34.209 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-24T21:37:34.266 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-07-24T21:37:34.322 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-24T21:37:34.322 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-07-24 21:36:41.778216349 +0000 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-07-24 21:31:16.877677999 +0000 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-07-24 21:31:16.877677999 +0000 2024-07-24T21:37:34.323 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-07-24 21:31:16.877677999 +0000 2024-07-24T21:37:34.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-24T21:37:34.388 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-07-24T21:37:34.388 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-07-24T21:37:34.388 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000201689 s, 2.5 MB/s 2024-07-24T21:37:34.389 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-24T21:37:34.447 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-07-24T21:37:34.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-24T21:37:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:37:35.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:35 smithi123 ceph-mon[37143]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:35.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:35 smithi086 ceph-mon[37128]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:35.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:35 smithi060 ceph-mon[33000]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-24T21:37:36.517 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-24T21:37:36.527 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-07-24T21:37:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-07-24T21:37:36.528 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0085994 s, 1.2 GB/s 2024-07-24T21:37:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-07-24T21:37:37.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-07-24T21:37:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:37:37.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:37 smithi123 ceph-mon[37143]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:37.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:37 smithi060 ceph-mon[33000]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:37.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:37 smithi086 ceph-mon[37128]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.167+0000 7f49202a1640 1 -- 172.21.15.60:0/1631965360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49181081a0 msgr2=0x7f49181085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.167+0000 7f49202a1640 1 --2- 172.21.15.60:0/1631965360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49181081a0 0x7f49181085e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4908009900 tx=0x7f490802a410 comp rx=0 tx=0).stop 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 -- 172.21.15.60:0/1631965360 shutdown_connections 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 --2- 172.21.15.60:0/1631965360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 0x7f491810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 --2- 172.21.15.60:0/1631965360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49181081a0 0x7f49181085e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 --2- 172.21.15.60:0/1631965360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4918102790 0x7f4918102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 -- 172.21.15.60:0/1631965360 >> 172.21.15.60:0/1631965360 conn(0x7f49180fe200 msgr2=0x7f4918100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 -- 172.21.15.60:0/1631965360 shutdown_connections 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 -- 172.21.15.60:0/1631965360 wait complete. 2024-07-24T21:37:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 Processor -- start 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 -- start start 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.168+0000 7f49202a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49181081a0 0x7f49180753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 0x7f4918075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112780 con 0x7f4918102790 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112600 con 0x7f49181081a0 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4918112900 con 0x7f4918108b20 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50636/0 (socket says 172.21.15.60:50636) 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 -- 172.21.15.60:0/2309530370 learned_addr learned my addr 172.21.15.60:0/2309530370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:37:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491d83c640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49181081a0 0x7f49180753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e83e640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 0x7f4918075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 -- 172.21.15.60:0/2309530370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 msgr2=0x7f4918075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 0x7f4918075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 -- 172.21.15.60:0/2309530370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49181081a0 msgr2=0x7f49180753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49181081a0 0x7f49180753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 -- 172.21.15.60:0/2309530370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49181a70b0 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f491e03d640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491400cd80 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140108e0 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49181a73a0 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914018980 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.169+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49181a7900 con 0x7f4918102790 2024-07-24T21:37:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.170+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4918102b70 con 0x7f4918102790 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.172+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f4914010430 con 0x7f4918102790 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.173+0000 7f490f7fe640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 0x7f49040b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.173+0000 7f491d83c640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 0x7f49040b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.173+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f49140e2820 con 0x7f4918102790 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.173+0000 7f491d83c640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 0x7f49040b60d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4908005ab0 tx=0x7f4908005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.176+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f49140aba70 con 0x7f4918102790 2024-07-24T21:37:39.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:39 smithi060 ceph-mon[33000]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:39.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:39.379+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4918062f80 con 0x7f49040b3c10 2024-07-24T21:37:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:39 smithi123 ceph-mon[37143]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:39.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:39 smithi086 ceph-mon[37128]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:40.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:40 smithi123 ceph-mon[37143]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:37:40.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:37:40.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:37:40.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:40.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:40 smithi086 ceph-mon[37128]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:37:40.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:37:40.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:37:40.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:40.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:40 smithi060 ceph-mon[33000]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:37:40.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:37:40.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:37:40.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:41.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:41 smithi123 ceph-mon[37143]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:41.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:41 smithi086 ceph-mon[37128]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:41.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:41 smithi060 ceph-mon[33000]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:42.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:42 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]: dispatch 2024-07-24T21:37:42.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:42 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]': finished 2024-07-24T21:37:42.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:42 smithi123 ceph-mon[37143]: osdmap e6: 1 total, 0 up, 1 in 2024-07-24T21:37:42.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:37:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:42 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]: dispatch 2024-07-24T21:37:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:42 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]': finished 2024-07-24T21:37:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:42 smithi086 ceph-mon[37128]: osdmap e6: 1 total, 0 up, 1 in 2024-07-24T21:37:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:37:42.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:42 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]: dispatch 2024-07-24T21:37:42.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:42 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1540223282' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5cc77b0-13c2-4164-822c-c458ae4cac91"}]': finished 2024-07-24T21:37:42.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:42 smithi060 ceph-mon[33000]: osdmap e6: 1 total, 0 up, 1 in 2024-07-24T21:37:42.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:37:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:43 smithi123 ceph-mon[37143]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:43 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2723390297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:37:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:43 smithi086 ceph-mon[37128]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:43 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2723390297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:37:43.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:43 smithi060 ceph-mon[33000]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:43.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:43 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2723390297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:37:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:45 smithi123 ceph-mon[37143]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:45.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:45 smithi086 ceph-mon[37128]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:45 smithi060 ceph-mon[33000]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:47.488 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:47 smithi060 ceph-mon[33000]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:47 smithi123 ceph-mon[37143]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:47.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:47 smithi086 ceph-mon[37128]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:49.468 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:49 smithi060 ceph-mon[33000]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:49.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:49 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-24T21:37:49.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:49 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:49 smithi123 ceph-mon[37143]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:49 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-24T21:37:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:49 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:49 smithi086 ceph-mon[37128]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:49 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-24T21:37:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:49 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:50.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:50 smithi060 ceph-mon[33000]: Deploying daemon osd.0 on smithi060 2024-07-24T21:37:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:50 smithi123 ceph-mon[37143]: Deploying daemon osd.0 on smithi060 2024-07-24T21:37:50.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:50 smithi086 ceph-mon[37128]: Deploying daemon osd.0 on smithi060 2024-07-24T21:37:51.461 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:51 smithi060 ceph-mon[33000]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:51 smithi123 ceph-mon[37143]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:51.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:51 smithi086 ceph-mon[37128]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:53.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:53 smithi123 ceph-mon[37143]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:53.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:53 smithi060 ceph-mon[33000]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:53.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:53 smithi086 ceph-mon[37128]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:54.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:54.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:54.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:54.495 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:54.495 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:54.495 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:54.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:54.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:54.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:55 smithi123 ceph-mon[37143]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:55.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:55 smithi060 ceph-mon[33000]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:55.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:55 smithi086 ceph-mon[37128]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:57.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:57 smithi123 ceph-mon[37143]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:57.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.513 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:57 smithi060 ceph-mon[33000]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:57.514 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.514 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:57 smithi086 ceph-mon[37128]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:57.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.557+0000 7f490f7fe640 1 -- 172.21.15.60:0/2309530370 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4918062f80 con 0x7f49040b3c10 2024-07-24T21:37:57.560 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-07-24T21:37:57.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.559+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 msgr2=0x7f49040b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:57.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.559+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 0x7f49040b60d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4908005ab0 tx=0x7f4908005a20 comp rx=0 tx=0).stop 2024-07-24T21:37:57.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.559+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 msgr2=0x7f4918074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.559+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 shutdown_connections 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f49040b3c10 0x7f49040b60d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4918108b20 0x7f4918075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49181081a0 0x7f49180753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 --2- 172.21.15.60:0/2309530370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4918102790 0x7f4918074e90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:37:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 >> 172.21.15.60:0/2309530370 conn(0x7f49180fe200 msgr2=0x7f49181005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:37:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 shutdown_connections 2024-07-24T21:37:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:37:57.560+0000 7f49202a1640 1 -- 172.21.15.60:0/2309530370 wait complete. 2024-07-24T21:37:58.147 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.0.service 2024-07-24T21:37:58.149 INFO:tasks.cephadm:Deploying osd.1 on smithi086 with /dev/vg_nvme/lv_4... 2024-07-24T21:37:58.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-24T21:37:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:58 smithi060 ceph-mon[33000]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-24T21:37:58.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:58.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:58.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:58.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:58 smithi123 ceph-mon[37143]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:37:58.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:58 smithi086 ceph-mon[37128]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-24T21:37:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:59 smithi086 ceph-mon[37128]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:59 smithi086 ceph-mon[37128]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-24T21:37:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:59 smithi086 ceph-mon[37128]: osdmap e7: 1 total, 0 up, 1 in 2024-07-24T21:37:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:59 smithi086 ceph-mon[37128]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:37:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:37:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:37:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:59 smithi060 ceph-mon[33000]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:59 smithi060 ceph-mon[33000]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-24T21:37:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:59 smithi060 ceph-mon[33000]: osdmap e7: 1 total, 0 up, 1 in 2024-07-24T21:37:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:59 smithi060 ceph-mon[33000]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:37:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:37:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:37:59.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:59 smithi123 ceph-mon[37143]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:37:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:59 smithi123 ceph-mon[37143]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-24T21:37:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:59 smithi123 ceph-mon[37143]: osdmap e7: 1 total, 0 up, 1 in 2024-07-24T21:37:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:59 smithi123 ceph-mon[37143]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:37:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:37:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-24T21:38:00.254 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-24T21:38:00.268 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-07-24T21:38:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-07-24T21:38:00.268 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127727 s, 821 MB/s 2024-07-24T21:38:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-07-24T21:38:00.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:00 smithi086 ceph-mon[37128]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:38:00.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:00 smithi086 ceph-mon[37128]: osdmap e8: 1 total, 0 up, 1 in 2024-07-24T21:38:00.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:00 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:00 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:00 smithi086 ceph-mon[37128]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' 2024-07-24T21:38:00.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-mon[33000]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:38:00.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-mon[33000]: osdmap e8: 1 total, 0 up, 1 in 2024-07-24T21:38:00.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-mon[33000]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' 2024-07-24T21:38:00.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0[52988]: 2024-07-24T21:38:00.358+0000 7f3b34518640 -1 osd.0 0 waiting for initial osdmap 2024-07-24T21:38:00.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0[52988]: 2024-07-24T21:38:00.362+0000 7f3b2c337640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:38:00.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-07-24T21:38:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:00 smithi123 ceph-mon[37143]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:38:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:00 smithi123 ceph-mon[37143]: osdmap e8: 1 total, 0 up, 1 in 2024-07-24T21:38:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:00 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:00 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:00 smithi123 ceph-mon[37143]: from='osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807]' entity='osd.0' 2024-07-24T21:38:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.931 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:01.932 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:01.932 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.932 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] boot 2024-07-24T21:38:01.932 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: osdmap e9: 1 total, 1 up, 1 in 2024-07-24T21:38:01.932 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:38:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] boot 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: osdmap e9: 1 total, 1 up, 1 in 2024-07-24T21:38:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:02.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:38:02.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:38:02.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: osd.0 [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] boot 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: osdmap e9: 1 total, 1 up, 1 in 2024-07-24T21:38:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-24T21:38:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- 172.21.15.86:0/830835646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 msgr2=0x7fa70410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 --2- 172.21.15.86:0/830835646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70410ef20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009ca0 tx=0x7fa6fc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- 172.21.15.86:0/830835646 shutdown_connections 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 --2- 172.21.15.86:0/830835646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70410ef20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 --2- 172.21.15.86:0/830835646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103f80 0x7fa70410c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 --2- 172.21.15.86:0/830835646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103660 0x7fa704103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- 172.21.15.86:0/830835646 >> 172.21.15.86:0/830835646 conn(0x7fa7040fd0c0 msgr2=0x7fa7040ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- 172.21.15.86:0/830835646 shutdown_connections 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- 172.21.15.86:0/830835646 wait complete. 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 Processor -- start 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.698+0000 7fa709b9c640 1 -- start start 2024-07-24T21:38:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 0x7fa704198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103f80 0x7fa7041991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041127f0 con 0x7fa704103f80 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa704112670 con 0x7fa70410cac0 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa704112970 con 0x7fa704103660 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38190/0 (socket says 172.21.15.86:38190) 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 -- 172.21.15.86:0/4018943548 learned_addr learned my addr 172.21.15.86:0/4018943548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-07-24T21:38:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa6faffd640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103f80 0x7fa7041991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa7037fe640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 0x7fa704198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 -- 172.21.15.86:0/4018943548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 msgr2=0x7fa704198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 0x7fa704198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 -- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103f80 msgr2=0x7fa7041991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103f80 0x7fa7041991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 -- 172.21.15.86:0/4018943548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70419ed30 con 0x7fa70410cac0 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa7037fe640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 0x7fa704198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa703fff640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc031a20 tx=0x7fa6fc031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc003d30 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc0044f0 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70419efc0 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc02fd90 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.699+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a5760 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.700+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa704103a40 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.701+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fa6fc004040 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.702+0000 7fa7017fa640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 0x7fa6f00b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.702+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7fa6fc103aa0 con 0x7fa70410cac0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.702+0000 7fa7037fe640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 0x7fa6f00b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.703+0000 7fa7037fe640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 0x7fa6f00b60d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4006fd0 tx=0x7fa6f4008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.705+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa6fc0ccb20 con 0x7fa70410cac0 2024-07-24T21:38:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:02 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi060 2024-07-24T21:38:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:02 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 17092M 2024-07-24T21:38:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:02.900+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa7040fecd0 con 0x7fa6f00b3c10 2024-07-24T21:38:02.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:02 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi060 2024-07-24T21:38:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:02 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 17092M 2024-07-24T21:38:03.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:02 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi060 2024-07-24T21:38:03.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:02 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 17092M 2024-07-24T21:38:03.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:03 smithi123 ceph-mon[37143]: pgmap v40: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:03.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:03 smithi123 ceph-mon[37143]: osdmap e10: 1 total, 1 up, 1 in 2024-07-24T21:38:03.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:03.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:03.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:03.923 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:03 smithi086 ceph-mon[37128]: pgmap v40: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:03.923 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:03 smithi086 ceph-mon[37128]: osdmap e10: 1 total, 1 up, 1 in 2024-07-24T21:38:03.923 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:03.923 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:03.923 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:03 smithi060 ceph-mon[33000]: pgmap v40: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:03 smithi060 ceph-mon[33000]: osdmap e10: 1 total, 1 up, 1 in 2024-07-24T21:38:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:04.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:04 smithi086 ceph-mon[37128]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:04 smithi123 ceph-mon[37143]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:05.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:04 smithi060 ceph-mon[33000]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:05.732 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:05.733 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/3034690043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:05.733 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:05.733 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]': finished 2024-07-24T21:38:05.733 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: osdmap e11: 2 total, 1 up, 2 in 2024-07-24T21:38:05.733 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/3034690043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]': finished 2024-07-24T21:38:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: osdmap e11: 2 total, 1 up, 2 in 2024-07-24T21:38:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:06.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:06.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/3034690043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:06.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]: dispatch 2024-07-24T21:38:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9d0f0b-703f-4b07-b1c3-6c12a00261d6"}]': finished 2024-07-24T21:38:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: osdmap e11: 2 total, 1 up, 2 in 2024-07-24T21:38:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:06.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:06 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/3339416465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:07.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:06 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/3339416465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:07.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:06 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/3339416465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:07.946 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:07 smithi086 ceph-mon[37128]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:07 smithi123 ceph-mon[37143]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:08.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:07 smithi060 ceph-mon[33000]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:09.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:09 smithi123 ceph-mon[37143]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:09 smithi086 ceph-mon[37128]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:10.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:09 smithi060 ceph-mon[33000]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:11.744 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:11 smithi086 ceph-mon[37128]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:11 smithi123 ceph-mon[37143]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:12.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:11 smithi060 ceph-mon[33000]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-24T21:38:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:12.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-24T21:38:12.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-24T21:38:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:13.655 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:13 smithi086 ceph-mon[37128]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:13.655 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:13 smithi086 ceph-mon[37128]: Deploying daemon osd.1 on smithi086 2024-07-24T21:38:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:13 smithi123 ceph-mon[37143]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:13 smithi123 ceph-mon[37143]: Deploying daemon osd.1 on smithi086 2024-07-24T21:38:14.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:13 smithi060 ceph-mon[33000]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:14.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:13 smithi060 ceph-mon[33000]: Deploying daemon osd.1 on smithi086 2024-07-24T21:38:15.674 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:15 smithi086 ceph-mon[37128]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:15 smithi060 ceph-mon[33000]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:15.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:15 smithi123 ceph-mon[37143]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:17 smithi086 ceph-mon[37128]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:17 smithi123 ceph-mon[37143]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:17 smithi060 ceph-mon[33000]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.662+0000 7fa7017fa640 1 -- 172.21.15.86:0/4018943548 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7040fecd0 con 0x7fa6f00b3c10 2024-07-24T21:38:19.664 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 1 on host 'smithi086' 2024-07-24T21:38:19.664 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:19 smithi086 ceph-mon[37128]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 msgr2=0x7fa6f00b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 0x7fa6f00b60d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4006fd0 tx=0x7fa6f4008040 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 msgr2=0x7fa70419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc031a20 tx=0x7fa6fc031a50 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 shutdown_connections 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6f00b3c10 0x7fa6f00b60d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa70410cac0 0x7fa70419e560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704103f80 0x7fa7041991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 --2- 172.21.15.86:0/4018943548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa704103660 0x7fa704198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 >> 172.21.15.86:0/4018943548 conn(0x7fa7040fd0c0 msgr2=0x7fa70410ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:38:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 shutdown_connections 2024-07-24T21:38:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:38:19.665+0000 7fa709b9c640 1 -- 172.21.15.86:0/4018943548 wait complete. 2024-07-24T21:38:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:19 smithi123 ceph-mon[37143]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:19 smithi060 ceph-mon[33000]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:20.324 DEBUG:teuthology.orchestra.run.smithi086:osd.1> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.1.service 2024-07-24T21:38:20.326 INFO:tasks.cephadm:Deploying osd.2 on smithi123 with /dev/vg_nvme/lv_4... 2024-07-24T21:38:20.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-24T21:38:20.592 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:38:20.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:20.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:20.956 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:20.956 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:20.956 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:20 smithi123 ceph-mon[37143]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-24T21:38:21.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-mon[37128]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-24T21:38:21.015 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T21:38:20.641+0000 7f9fd74a3640 -1 osd.1 0 log_to_monitors true 2024-07-24T21:38:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:20 smithi060 ceph-mon[33000]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-24T21:38:21.895 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:21 smithi123 ceph-mon[37143]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:21.896 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:21 smithi123 ceph-mon[37143]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-24T21:38:21.896 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:21 smithi123 ceph-mon[37143]: osdmap e12: 2 total, 1 up, 2 in 2024-07-24T21:38:21.896 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:21 smithi123 ceph-mon[37143]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:38:21.896 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:21.914 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:21 smithi086 ceph-mon[37128]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:21.914 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:21 smithi086 ceph-mon[37128]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-24T21:38:21.914 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:21 smithi086 ceph-mon[37128]: osdmap e12: 2 total, 1 up, 2 in 2024-07-24T21:38:21.914 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:21 smithi086 ceph-mon[37128]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:38:21.914 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:21 smithi060 ceph-mon[33000]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:21 smithi060 ceph-mon[33000]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-24T21:38:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:21 smithi060 ceph-mon[33000]: osdmap e12: 2 total, 1 up, 2 in 2024-07-24T21:38:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:21 smithi060 ceph-mon[33000]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:38:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:22.276 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-24T21:38:22.402 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-24T21:38:22.416 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-07-24T21:38:22.416 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-07-24T21:38:22.416 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119068 s, 881 MB/s 2024-07-24T21:38:22.416 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-07-24T21:38:22.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-07-24T21:38:23.201 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:38:23.794 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T21:38:23.536+0000 7f9fd3426640 -1 osd.1 0 waiting for initial osdmap 2024-07-24T21:38:23.795 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T21:38:23.727+0000 7f9fcb245640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: osdmap e13: 2 total, 1 up, 2 in 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:23.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:23.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:23.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:38:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: osdmap e13: 2 total, 1 up, 2 in 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:38:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: osdmap e13: 2 total, 1 up, 2 in 2024-07-24T21:38:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191]' entity='osd.1' 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:38:24.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:38:24.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi086 2024-07-24T21:38:24.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 17088M 2024-07-24T21:38:24.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:24.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:24.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:24.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:24.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:24.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- 172.21.15.123:0/1585104442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4102760 msgr2=0x7fbfa4102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 --2- 172.21.15.123:0/1585104442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4102760 0x7fbfa4102b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009ca0 tx=0x7fbf9402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- 172.21.15.123:0/1585104442 shutdown_connections 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 --2- 172.21.15.123:0/1585104442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4108af0 0x7fbfa410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 --2- 172.21.15.123:0/1585104442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 0x7fbfa41085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 --2- 172.21.15.123:0/1585104442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4102760 0x7fbfa4102b40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- 172.21.15.123:0/1585104442 >> 172.21.15.123:0/1585104442 conn(0x7fbfa40fe210 msgr2=0x7fbfa4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- 172.21.15.123:0/1585104442 shutdown_connections 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- 172.21.15.123:0/1585104442 wait complete. 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 Processor -- start 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.697+0000 7fbfac385640 1 -- start start 2024-07-24T21:38:24.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4102760 0x7fbfa4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 0x7fbfa41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4112840 con 0x7fbfa4102760 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41126c0 con 0x7fbfa4108170 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41129c0 con 0x7fbfa4108af0 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37080/0 (socket says 172.21.15.123:37080) 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 -- 172.21.15.123:0/592938558 learned_addr learned my addr 172.21.15.123:0/592938558 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa121640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4102760 0x7fbfa4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfa9920640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 0x7fbfa41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 -- 172.21.15.123:0/592938558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 msgr2=0x7fbfa41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:24.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 0x7fbfa41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 -- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4102760 msgr2=0x7fbfa4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4102760 0x7fbfa4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 -- 172.21.15.123:0/592938558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa419ee60 con 0x7fbfa4108af0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfaa922640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009fd0 tx=0x7fbf9c00b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c015020 con 0x7fbfa4108af0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c011ce0 con 0x7fbfa4108af0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa419f150 con 0x7fbfa4108af0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c010dc0 con 0x7fbfa4108af0 2024-07-24T21:38:24.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.698+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa41a5890 con 0x7fbfa4108af0 2024-07-24T21:38:24.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.699+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa4102bc0 con 0x7fbfa4108af0 2024-07-24T21:38:24.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.700+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fbf9c018020 con 0x7fbfa4108af0 2024-07-24T21:38:24.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.701+0000 7fbfa37fe640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 0x7fbf8c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:24.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.701+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2524+0+0 (secure 0 0 0) 0x7fbf9c0e1cb0 con 0x7fbfa4108af0 2024-07-24T21:38:24.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.701+0000 7fbfaa121640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 0x7fbf8c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:24.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.702+0000 7fbfaa121640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 0x7fbf8c0b6050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006010 tx=0x7fbf940093a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.705+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbf9c01f020 con 0x7fbfa4108af0 2024-07-24T21:38:24.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:24.897+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbfa4062f80 con 0x7fbf8c0b3b90 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi086 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 17088M 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi086 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 17088M 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:25.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] boot 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: osdmap e14: 2 total, 2 up, 2 in 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:25.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:26.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] boot 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: osdmap e14: 2 total, 2 up, 2 in 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: osd.1 [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] boot 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: osdmap e14: 2 total, 2 up, 2 in 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:26.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:26 smithi123 ceph-mon[37143]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:26.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:26 smithi123 ceph-mon[37143]: osdmap e15: 2 total, 2 up, 2 in 2024-07-24T21:38:27.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:26 smithi086 ceph-mon[37128]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:27.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:26 smithi086 ceph-mon[37128]: osdmap e15: 2 total, 2 up, 2 in 2024-07-24T21:38:27.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:26 smithi060 ceph-mon[33000]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:27.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:26 smithi060 ceph-mon[33000]: osdmap e15: 2 total, 2 up, 2 in 2024-07-24T21:38:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/1115087263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]': finished 2024-07-24T21:38:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: osdmap e16: 3 total, 2 up, 3 in 2024-07-24T21:38:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/37115747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/1115087263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]': finished 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: osdmap e16: 3 total, 2 up, 3 in 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/37115747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/1115087263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]: dispatch 2024-07-24T21:38:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9f91f69-71de-4bb0-81b0-a3a5da7c0733"}]': finished 2024-07-24T21:38:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: osdmap e16: 3 total, 2 up, 3 in 2024-07-24T21:38:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/37115747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:29 smithi123 ceph-mon[37143]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:30.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:29 smithi086 ceph-mon[37128]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:29 smithi060 ceph-mon[33000]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:31.927 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:31 smithi123 ceph-mon[37143]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:32.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:31 smithi086 ceph-mon[37128]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:31 smithi060 ceph-mon[33000]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:33.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:33 smithi123 ceph-mon[37143]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:34.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:33 smithi086 ceph-mon[37128]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:34.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:33 smithi060 ceph-mon[33000]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:34.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-24T21:38:34.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:35.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-24T21:38:35.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-24T21:38:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:35.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:35 smithi123 ceph-mon[37143]: Deploying daemon osd.2 on smithi123 2024-07-24T21:38:35.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:35 smithi123 ceph-mon[37143]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:36.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:35 smithi086 ceph-mon[37128]: Deploying daemon osd.2 on smithi123 2024-07-24T21:38:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:35 smithi086 ceph-mon[37128]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:36.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:35 smithi060 ceph-mon[33000]: Deploying daemon osd.2 on smithi123 2024-07-24T21:38:36.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:35 smithi060 ceph-mon[33000]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:37 smithi123 ceph-mon[37143]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:37 smithi086 ceph-mon[37128]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:38.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:37 smithi060 ceph-mon[33000]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:38.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:39.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:39.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:39.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:39.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:39 smithi123 ceph-mon[37143]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:40.043 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:39 smithi086 ceph-mon[37128]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:40.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:39 smithi060 ceph-mon[33000]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:41.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.653+0000 7fbfa37fe640 1 -- 172.21.15.123:0/592938558 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfa4062f80 con 0x7fbf8c0b3b90 2024-07-24T21:38:41.654 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 2 on host 'smithi123' 2024-07-24T21:38:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 msgr2=0x7fbf8c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 0x7fbf8c0b6050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006010 tx=0x7fbf940093a0 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 msgr2=0x7fbfa419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009fd0 tx=0x7fbf9c00b820 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 shutdown_connections 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf8c0b3b90 0x7fbf8c0b6050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa4108af0 0x7fbfa419e690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa4108170 0x7fbfa41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 --2- 172.21.15.123:0/592938558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa4102760 0x7fbfa4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 >> 172.21.15.123:0/592938558 conn(0x7fbfa40fe210 msgr2=0x7fbfa4100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 shutdown_connections 2024-07-24T21:38:41.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:38:41.655+0000 7fbfac385640 1 -- 172.21.15.123:0/592938558 wait complete. 2024-07-24T21:38:41.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:41 smithi123 ceph-mon[37143]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:41.956 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:41.956 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:41 smithi086 ceph-mon[37128]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:42.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:41 smithi060 ceph-mon[33000]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:42.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.393 DEBUG:teuthology.orchestra.run.smithi123:osd.2> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.2.service 2024-07-24T21:38:42.396 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_3... 2024-07-24T21:38:42.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-24T21:38:42.632 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T21:38:42.513+0000 7f38c7bcc640 -1 osd.2 0 log_to_monitors true 2024-07-24T21:38:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:38:42.910 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.910 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:42.910 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-mon[37143]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:42.910 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-mon[37143]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:42.910 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:42 smithi123 ceph-mon[37143]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:43.005 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:43.005 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:43.005 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:42 smithi060 ceph-mon[33000]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:43.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:42 smithi060 ceph-mon[33000]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:43.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:42 smithi060 ceph-mon[33000]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:43.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:43.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:42 smithi086 ceph-mon[37128]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:42 smithi086 ceph-mon[37128]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:42 smithi086 ceph-mon[37128]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-24T21:38:43.816 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T21:38:43.697+0000 7f38c3b4f640 -1 osd.2 0 waiting for initial osdmap 2024-07-24T21:38:43.816 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T21:38:43.702+0000 7f38bf976640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:38:43.817 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-mon[37143]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-24T21:38:43.817 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-mon[37143]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:43.817 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-mon[37143]: osdmap e17: 3 total, 2 up, 3 in 2024-07-24T21:38:43.817 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:43.817 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:43 smithi123 ceph-mon[37143]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:43.978 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:43 smithi060 ceph-mon[33000]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-24T21:38:43.978 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:43 smithi060 ceph-mon[33000]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:43.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:43 smithi060 ceph-mon[33000]: osdmap e17: 3 total, 2 up, 3 in 2024-07-24T21:38:43.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:43.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:43 smithi060 ceph-mon[33000]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:44.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:43 smithi086 ceph-mon[37128]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-24T21:38:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:43 smithi086 ceph-mon[37128]: from='osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:43 smithi086 ceph-mon[37128]: osdmap e17: 3 total, 2 up, 3 in 2024-07-24T21:38:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:43 smithi086 ceph-mon[37128]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:38:44.385 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-24T21:38:44.469 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-24T21:38:44.479 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-07-24T21:38:44.479 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-07-24T21:38:44.479 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00855272 s, 1.2 GB/s 2024-07-24T21:38:44.479 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-07-24T21:38:44.827 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:44 smithi123 ceph-mon[37143]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:38:44.827 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:44 smithi123 ceph-mon[37143]: osdmap e18: 3 total, 2 up, 3 in 2024-07-24T21:38:44.827 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:44 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:44.827 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:44 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:44.827 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:44 smithi123 ceph-mon[37143]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:45.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:44 smithi060 ceph-mon[33000]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:38:45.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:44 smithi060 ceph-mon[33000]: osdmap e18: 3 total, 2 up, 3 in 2024-07-24T21:38:45.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:44 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:45.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:44 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:45.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:44 smithi060 ceph-mon[33000]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:45.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:44 smithi086 ceph-mon[37128]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:38:45.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:44 smithi086 ceph-mon[37128]: osdmap e18: 3 total, 2 up, 3 in 2024-07-24T21:38:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:44 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:44 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:44 smithi086 ceph-mon[37128]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-07-24T21:38:45.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-07-24T21:38:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:38:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:38:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:38:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] boot 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: osdmap e19: 3 total, 3 up, 3 in 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi123 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 17088M 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] boot 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: osdmap e19: 3 total, 3 up, 3 in 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi123 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 17088M 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:38:46.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:38:46.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: osd.2 [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] boot 2024-07-24T21:38:46.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: osdmap e19: 3 total, 3 up, 3 in 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi123 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 17088M 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:38:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 -- 172.21.15.60:0/2983459012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2708101690 msgr2=0x7f2708101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 --2- 172.21.15.60:0/2983459012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2708101690 0x7f2708101a70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009900 tx=0x7f26f802a410 comp rx=0 tx=0).stop 2024-07-24T21:38:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 -- 172.21.15.60:0/2983459012 shutdown_connections 2024-07-24T21:38:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 --2- 172.21.15.60:0/2983459012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708107a20 0x7f270810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 --2- 172.21.15.60:0/2983459012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27081070a0 0x7f27081074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 --2- 172.21.15.60:0/2983459012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2708101690 0x7f2708101a70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 -- 172.21.15.60:0/2983459012 >> 172.21.15.60:0/2983459012 conn(0x7f27080fd100 msgr2=0x7f27080ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:38:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 -- 172.21.15.60:0/2983459012 shutdown_connections 2024-07-24T21:38:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.926+0000 7f270e61a640 1 -- 172.21.15.60:0/2983459012 wait complete. 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 Processor -- start 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 -- start start 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27081070a0 0x7f2708199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708107a20 0x7f270819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2708112800 con 0x7f2708101690 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2708112680 con 0x7f27081070a0 2024-07-24T21:38:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270e61a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2708112980 con 0x7f2708107a20 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f2707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f2707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59964/0 (socket says 172.21.15.60:59964) 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f2707fff640 1 -- 172.21.15.60:0/3139368308 learned_addr learned my addr 172.21.15.60:0/3139368308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f26fffff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27081070a0 0x7f2708199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.927+0000 7f270cbb7640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708107a20 0x7f270819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 -- 172.21.15.60:0/3139368308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708107a20 msgr2=0x7f270819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708107a20 0x7f270819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 -- 172.21.15.60:0/3139368308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27081070a0 msgr2=0x7f2708199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27081070a0 0x7f2708199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 -- 172.21.15.60:0/3139368308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270819ec80 con 0x7f2708101690 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2707fff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26f8005940 tx=0x7f26f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f803f030 con 0x7f2708101690 2024-07-24T21:38:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270819ef10 con 0x7f2708101690 2024-07-24T21:38:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f8003cb0 con 0x7f2708101690 2024-07-24T21:38:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8035070 con 0x7f2708101690 2024-07-24T21:38:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.928+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27081a5630 con 0x7f2708101690 2024-07-24T21:38:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.929+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2708101a70 con 0x7f2708101690 2024-07-24T21:38:46.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:46 smithi060 ceph-mon[33000]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:46.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:46.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:46.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:46 smithi060 ceph-mon[33000]: osdmap e20: 3 total, 3 up, 3 in 2024-07-24T21:38:46.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.931+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f26f8042030 con 0x7f2708101690 2024-07-24T21:38:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.932+0000 7f2705ffb640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 0x7f26f40b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:38:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.932+0000 7f26fffff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 0x7f26f40b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:38:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.932+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3441+0+0 (secure 0 0 0) 0x7f26f8103370 con 0x7f2708101690 2024-07-24T21:38:46.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.933+0000 7f26fffff640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 0x7f26f40b6050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2708100580 tx=0x7f26f000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:38:46.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:46.935+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f26f80cb750 con 0x7f2708101690 2024-07-24T21:38:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:46 smithi123 ceph-mon[37143]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:46 smithi123 ceph-mon[37143]: osdmap e20: 3 total, 3 up, 3 in 2024-07-24T21:38:46.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:47.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:46 smithi086 ceph-mon[37128]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:47.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:47.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:47.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:46 smithi086 ceph-mon[37128]: osdmap e20: 3 total, 3 up, 3 in 2024-07-24T21:38:47.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T21:38:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:47.139+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2708104660 con 0x7f26f40b3b90 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57747]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57747]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57747]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57750]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57750]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.601 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57686]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.601 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57686]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57686]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57689]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57689]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.602 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 21:38:47 smithi060 sudo[57689]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47003]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47003]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47006]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.731 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47006]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[46997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[46997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[46997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[46997]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47000]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.732 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 21:38:47 smithi086 sudo[47000]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47485]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47485]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47488]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47488]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='client.14469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: osdmap e21: 3 total, 3 up, 3 in 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:47.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47479]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47479]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-24T21:38:47.745 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47482]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-24T21:38:47.746 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 21:38:47 smithi123 sudo[47482]: pam_unix(sudo:session): session closed for user root 2024-07-24T21:38:47.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:47 smithi123 ceph-mon[37143]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='client.14469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: osdmap e21: 3 total, 3 up, 3 in 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:47.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:47.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:47.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:47 smithi060 ceph-mon[33000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='client.14469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: osdmap e21: 3 total, 3 up, 3 in 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:48.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-24T21:38:48.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-24T21:38:48.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-24T21:38:48.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-24T21:38:48.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:47 smithi086 ceph-mon[37128]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-24T21:38:48.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:38:48.743+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f26f80c79e0 con 0x7f2708101690 2024-07-24T21:38:49.542 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:49 smithi086 ceph-mon[37128]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:49.542 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:49 smithi086 ceph-mon[37128]: osdmap e22: 3 total, 3 up, 3 in 2024-07-24T21:38:49.542 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:49 smithi086 ceph-mon[37128]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-24T21:38:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:49 smithi060 ceph-mon[33000]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:49 smithi060 ceph-mon[33000]: osdmap e22: 3 total, 3 up, 3 in 2024-07-24T21:38:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:49 smithi060 ceph-mon[33000]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-24T21:38:49.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:49 smithi123 ceph-mon[37143]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:49.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:49 smithi123 ceph-mon[37143]: osdmap e22: 3 total, 3 up, 3 in 2024-07-24T21:38:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:49 smithi123 ceph-mon[37143]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-24T21:38:50.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]: dispatch 2024-07-24T21:38:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]': finished 2024-07-24T21:38:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:50 smithi086 ceph-mon[37128]: osdmap e23: 4 total, 3 up, 4 in 2024-07-24T21:38:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:38:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2193318702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]: dispatch 2024-07-24T21:38:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]': finished 2024-07-24T21:38:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:50 smithi060 ceph-mon[33000]: osdmap e23: 4 total, 3 up, 4 in 2024-07-24T21:38:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:38:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2193318702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:50.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]: dispatch 2024-07-24T21:38:50.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1558270751' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "824da9ca-8ade-4664-8ec4-084f409bcacc"}]': finished 2024-07-24T21:38:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:50 smithi123 ceph-mon[37143]: osdmap e23: 4 total, 3 up, 4 in 2024-07-24T21:38:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:38:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2193318702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:38:51.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:51 smithi123 ceph-mon[37143]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:52.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:51 smithi086 ceph-mon[37128]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:51 smithi060 ceph-mon[33000]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:53.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:53 smithi060 ceph-mon[33000]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:53.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:53 smithi123 ceph-mon[37143]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:54.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:53 smithi086 ceph-mon[37128]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:55 smithi060 ceph-mon[33000]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:55.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:55 smithi123 ceph-mon[37143]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:56.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:55 smithi086 ceph-mon[37128]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:56.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:56 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-24T21:38:56.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:56 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:56.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:56 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-24T21:38:56.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:56 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:57.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:56 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-24T21:38:57.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:56 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:38:57.903 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:57 smithi060 ceph-mon[33000]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:57.904 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:57 smithi060 ceph-mon[33000]: Deploying daemon osd.3 on smithi060 2024-07-24T21:38:57.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:57 smithi123 ceph-mon[37143]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:57.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:57 smithi123 ceph-mon[37143]: Deploying daemon osd.3 on smithi060 2024-07-24T21:38:58.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:57 smithi086 ceph-mon[37128]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:58.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:57 smithi086 ceph-mon[37128]: Deploying daemon osd.3 on smithi060 2024-07-24T21:38:59.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:38:59 smithi060 ceph-mon[33000]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:38:59.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:38:59 smithi123 ceph-mon[37143]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:00.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:38:59 smithi086 ceph-mon[37128]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:01.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:01 smithi060 ceph-mon[33000]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:01 smithi123 ceph-mon[37143]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:02.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:01 smithi086 ceph-mon[37128]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:02.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:02.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:02.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:03.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:03.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:03.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:03.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:03 smithi060 ceph-mon[33000]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:03.893 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:03 smithi123 ceph-mon[37143]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:04.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:03 smithi086 ceph-mon[37128]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.853+0000 7f2705ffb640 1 -- 172.21.15.60:0/3139368308 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2708104660 con 0x7f26f40b3b90 2024-07-24T21:39:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 msgr2=0x7f26f40b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 0x7f26f40b6050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2708100580 tx=0x7f26f000b390 comp rx=0 tx=0).stop 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 msgr2=0x7f2708198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f26f8005940 tx=0x7f26f80042f0 comp rx=0 tx=0).stop 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 shutdown_connections 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f40b3b90 0x7f26f40b6050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708107a20 0x7f270819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:04.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27081070a0 0x7f2708199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 --2- 172.21.15.60:0/3139368308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708101690 0x7f2708198bd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 >> 172.21.15.60:0/3139368308 conn(0x7f27080fd100 msgr2=0x7f27080ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 shutdown_connections 2024-07-24T21:39:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:04.856+0000 7f270e61a640 1 -- 172.21.15.60:0/3139368308 wait complete. 2024-07-24T21:39:05.555 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.3.service 2024-07-24T21:39:05.557 INFO:tasks.cephadm:Deploying osd.4 on smithi086 with /dev/vg_nvme/lv_3... 2024-07-24T21:39:05.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-24T21:39:05.820 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:39:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-mon[33000]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.821 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 21:39:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T21:39:05.620+0000 7f53cab4c640 -1 osd.3 0 log_to_monitors true 2024-07-24T21:39:05.942 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:05 smithi086 ceph-mon[37128]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:05.942 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.942 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.942 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.942 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:05 smithi123 ceph-mon[37143]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:06.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:06 smithi060 ceph-mon[33000]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-24T21:39:06.953 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:06 smithi086 ceph-mon[37128]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-24T21:39:06.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:06 smithi123 ceph-mon[37143]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-24T21:39:07.502 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-24T21:39:07.585 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-24T21:39:07.598 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-07-24T21:39:07.598 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-07-24T21:39:07.598 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115755 s, 906 MB/s 2024-07-24T21:39:07.598 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 2024-07-24T21:39:07.599 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-07-24T21:39:07.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:07 smithi060 ceph-mon[33000]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:07.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:07 smithi060 ceph-mon[33000]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-24T21:39:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:07 smithi060 ceph-mon[33000]: osdmap e24: 4 total, 3 up, 4 in 2024-07-24T21:39:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:07 smithi060 ceph-mon[33000]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:39:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:07.954 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:07 smithi086 ceph-mon[37128]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:07.954 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:07 smithi086 ceph-mon[37128]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-24T21:39:07.954 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:07 smithi086 ceph-mon[37128]: osdmap e24: 4 total, 3 up, 4 in 2024-07-24T21:39:07.954 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:07 smithi086 ceph-mon[37128]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:39:07.954 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:07 smithi123 ceph-mon[37143]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:07 smithi123 ceph-mon[37143]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-24T21:39:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:07 smithi123 ceph-mon[37143]: osdmap e24: 4 total, 3 up, 4 in 2024-07-24T21:39:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:07 smithi123 ceph-mon[37143]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:39:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-07-24T21:39:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:39:08.571 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T21:39:08.483+0000 7f53c72e3640 -1 osd.3 0 waiting for initial osdmap 2024-07-24T21:39:08.571 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T21:39:08.489+0000 7f53be8ee640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:39:08.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:39:08.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: osdmap e25: 4 total, 3 up, 4 in 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: osdmap e26: 4 total, 3 up, 4 in 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:08 smithi086 ceph-mon[37128]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: osdmap e25: 4 total, 3 up, 4 in 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: osdmap e26: 4 total, 3 up, 4 in 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:08 smithi060 ceph-mon[33000]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' 2024-07-24T21:39:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:39:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:39:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:39:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: osdmap e25: 4 total, 3 up, 4 in 2024-07-24T21:39:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: osdmap e26: 4 total, 3 up, 4 in 2024-07-24T21:39:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:08 smithi123 ceph-mon[37143]: from='osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260]' entity='osd.3' 2024-07-24T21:39:09.906 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:09.906 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] boot 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: osdmap e27: 4 total, 4 up, 4 in 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.907 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:09.908 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:09.908 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:09.908 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] boot 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: osdmap e27: 4 total, 4 up, 4 in 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:09.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:09.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:09.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:09.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- 172.21.15.86:0/2241931829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb41079e0 msgr2=0x7f0eb410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/2241931829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb41079e0 0x7f0eb410e1e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009900 tx=0x7f0eac02a410 comp rx=0 tx=0).stop 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- 172.21.15.86:0/2241931829 shutdown_connections 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/2241931829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb41079e0 0x7f0eb410e1e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/2241931829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/2241931829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb4101650 0x7f0eb4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- 172.21.15.86:0/2241931829 >> 172.21.15.86:0/2241931829 conn(0x7f0eb40fd0c0 msgr2=0x7f0eb40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- 172.21.15.86:0/2241931829 shutdown_connections 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- 172.21.15.86:0/2241931829 wait complete. 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 Processor -- start 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.988+0000 7f0eb8c21640 1 -- start start 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 0x7f0eb41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41127c0 con 0x7f0eb41079e0 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112640 con 0x7f0eb4101650 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112940 con 0x7f0eb4107060 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41266/0 (socket says 172.21.15.86:41266) 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 -- 172.21.15.86:0/3033042832 learned_addr learned my addr 172.21.15.86:0/3033042832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb2d9d640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 0x7f0eb41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb1d9b640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 -- 172.21.15.86:0/3033042832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 msgr2=0x7f0eb419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 -- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 msgr2=0x7f0eb41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 0x7f0eb41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 -- 172.21.15.86:0/3033042832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb41a0eb0 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb1d9b640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb2d9d640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 0x7f0eb41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb259c640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400ebc0 tx=0x7f0ea400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4010630 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4010c80 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb41a11a0 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4018b60 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.989+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a7860 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.990+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb4101a30 con 0x7f0eb4101650 2024-07-24T21:39:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.992+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0ea4018d00 con 0x7f0eb4101650 2024-07-24T21:39:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.992+0000 7f0eab7fe640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 0x7f0ea00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.992+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3884+0+0 (secure 0 0 0) 0x7f0ea40e36d0 con 0x7f0eb4101650 2024-07-24T21:39:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.993+0000 7f0eb1d9b640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 0x7f0ea00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.993+0000 7f0eb1d9b640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 0x7f0ea00b60e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4103cc0 tx=0x7f0e9c005d10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:09.995+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0ea40abe70 con 0x7f0eb4101650 2024-07-24T21:39:10.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: osd.3 [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] boot 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: osdmap e27: 4 total, 4 up, 4 in 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:10.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:10.189+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0eb4062f80 con 0x7f0ea00b3c20 2024-07-24T21:39:10.892 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi060 2024-07-24T21:39:10.893 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 8546M 2024-07-24T21:39:10.893 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: osdmap e28: 4 total, 4 up, 4 in 2024-07-24T21:39:10.893 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:10.893 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:10.893 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:10.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi060 2024-07-24T21:39:10.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 8546M 2024-07-24T21:39:10.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: osdmap e28: 4 total, 4 up, 4 in 2024-07-24T21:39:10.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:10.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi060 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 8546M 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: osdmap e28: 4 total, 4 up, 4 in 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:11.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:11.752 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:11 smithi086 ceph-mon[37128]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:11.752 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:11 smithi086 ceph-mon[37128]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:11.752 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:11 smithi086 ceph-mon[37128]: osdmap e29: 4 total, 4 up, 4 in 2024-07-24T21:39:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:11 smithi123 ceph-mon[37143]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:11 smithi123 ceph-mon[37143]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:11 smithi123 ceph-mon[37143]: osdmap e29: 4 total, 4 up, 4 in 2024-07-24T21:39:12.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:11 smithi060 ceph-mon[33000]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:12.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:11 smithi060 ceph-mon[33000]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:12.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:11 smithi060 ceph-mon[33000]: osdmap e29: 4 total, 4 up, 4 in 2024-07-24T21:39:12.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:12 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/2370045586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:12.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:12 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:12.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:12 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]': finished 2024-07-24T21:39:12.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:12 smithi123 ceph-mon[37143]: osdmap e30: 5 total, 4 up, 5 in 2024-07-24T21:39:12.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:13.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:12 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/2370045586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:13.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:12 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:13.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:12 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]': finished 2024-07-24T21:39:13.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:12 smithi086 ceph-mon[37128]: osdmap e30: 5 total, 4 up, 5 in 2024-07-24T21:39:13.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:12 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/2370045586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:12 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]: dispatch 2024-07-24T21:39:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:12 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfc77006-c095-442e-8894-7d5901632cc0"}]': finished 2024-07-24T21:39:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:12 smithi060 ceph-mon[33000]: osdmap e30: 5 total, 4 up, 5 in 2024-07-24T21:39:13.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:13 smithi123 ceph-mon[37143]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/915348125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:14.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:13 smithi086 ceph-mon[37128]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/915348125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:14.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:13 smithi060 ceph-mon[33000]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:14.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/915348125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:15.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:15 smithi123 ceph-mon[37143]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:16.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:15 smithi086 ceph-mon[37128]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:16.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:15 smithi060 ceph-mon[33000]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:17 smithi123 ceph-mon[37143]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-24T21:39:18.027 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:17 smithi086 ceph-mon[37128]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-24T21:39:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:17 smithi060 ceph-mon[33000]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-24T21:39:19.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:19 smithi086 ceph-mon[37128]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:39:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-24T21:39:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:19 smithi123 ceph-mon[37143]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:39:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-24T21:39:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:19 smithi060 ceph-mon[33000]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:39:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-24T21:39:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:20 smithi123 ceph-mon[37143]: Deploying daemon osd.4 on smithi086 2024-07-24T21:39:21.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:20 smithi086 ceph-mon[37128]: Deploying daemon osd.4 on smithi086 2024-07-24T21:39:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:20 smithi060 ceph-mon[33000]: Deploying daemon osd.4 on smithi086 2024-07-24T21:39:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:21 smithi123 ceph-mon[37143]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-07-24T21:39:22.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:21 smithi086 ceph-mon[37128]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-07-24T21:39:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:21 smithi060 ceph-mon[33000]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-07-24T21:39:23.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:23 smithi086 ceph-mon[37128]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:39:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:23 smithi123 ceph-mon[37143]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:39:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:23 smithi060 ceph-mon[33000]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:39:24.941 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:24.941 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:24.941 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:24.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:24.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:24.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:25.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:25.773 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:25 smithi086 ceph-mon[37128]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-24T21:39:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:25 smithi123 ceph-mon[37143]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-24T21:39:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:25 smithi060 ceph-mon[33000]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-24T21:39:27.454 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 4 on host 'smithi086' 2024-07-24T21:39:27.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.452+0000 7f0eab7fe640 1 -- 172.21.15.86:0/3033042832 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0eb4062f80 con 0x7f0ea00b3c20 2024-07-24T21:39:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.454+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 msgr2=0x7f0ea00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 0x7f0ea00b60e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4103cc0 tx=0x7f0e9c005d10 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 msgr2=0x7f0eb419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400ebc0 tx=0x7f0ea400c6a0 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 shutdown_connections 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ea00b3c20 0x7f0ea00b60e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0eb41079e0 0x7f0eb41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0eb4107060 0x7f0eb419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 --2- 172.21.15.86:0/3033042832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb4101650 0x7f0eb419ae00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 >> 172.21.15.86:0/3033042832 conn(0x7f0eb40fd0c0 msgr2=0x7f0eb410f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 shutdown_connections 2024-07-24T21:39:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:39:27.455+0000 7f0eb8c21640 1 -- 172.21.15.86:0/3033042832 wait complete. 2024-07-24T21:39:27.778 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:27 smithi086 ceph-mon[37128]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:39:27.778 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.778 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.778 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.778 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:27 smithi123 ceph-mon[37143]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:39:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:27 smithi060 ceph-mon[33000]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:39:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:28.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:28.137 DEBUG:teuthology.orchestra.run.smithi086:osd.4> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.4.service 2024-07-24T21:39:28.140 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_3... 2024-07-24T21:39:28.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-24T21:39:28.395 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 21:39:28 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T21:39:28.281+0000 7fd4b2858640 -1 osd.4 0 log_to_monitors true 2024-07-24T21:39:28.402 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:39:28.801 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:28 smithi123 ceph-mon[37143]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:28.802 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:28 smithi123 ceph-mon[37143]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:29.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:28 smithi086 ceph-mon[37128]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:28 smithi086 ceph-mon[37128]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:28 smithi060 ceph-mon[33000]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:28 smithi060 ceph-mon[33000]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-24T21:39:29.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:29.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-24T21:39:29.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: osdmap e31: 5 total, 4 up, 5 in 2024-07-24T21:39:29.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:29.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:29 smithi123 ceph-mon[37143]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:30.030 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: osdmap e31: 5 total, 4 up, 5 in 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-mon[37128]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:30.046 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T21:39:29.686+0000 7fd4ae7db640 -1 osd.4 0 waiting for initial osdmap 2024-07-24T21:39:30.046 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 21:39:29 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T21:39:29.692+0000 7fd4aa602640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: osdmap e31: 5 total, 4 up, 5 in 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: from='osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:29 smithi060 ceph-mon[33000]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:39:30.109 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-24T21:39:30.123 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-07-24T21:39:30.123 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-07-24T21:39:30.123 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123938 s, 846 MB/s 2024-07-24T21:39:30.123 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-07-24T21:39:30.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-07-24T21:39:30.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:30 smithi123 ceph-mon[37143]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:39:30.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:30 smithi123 ceph-mon[37143]: osdmap e32: 5 total, 4 up, 5 in 2024-07-24T21:39:30.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:30 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:30.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:30 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:30.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:30 smithi123 ceph-mon[37143]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:31.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:30 smithi086 ceph-mon[37128]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:39:31.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:30 smithi086 ceph-mon[37128]: osdmap e32: 5 total, 4 up, 5 in 2024-07-24T21:39:31.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:30 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:30 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:30 smithi086 ceph-mon[37128]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:31.059 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:39:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:30 smithi060 ceph-mon[33000]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:39:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:30 smithi060 ceph-mon[33000]: osdmap e32: 5 total, 4 up, 5 in 2024-07-24T21:39:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:30 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:30 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:30 smithi060 ceph-mon[33000]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-07-24T21:39:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] boot 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: osdmap e33: 5 total, 5 up, 5 in 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi086 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 8544M 2024-07-24T21:39:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:31.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] boot 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: osdmap e33: 5 total, 5 up, 5 in 2024-07-24T21:39:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi086 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 8544M 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:32.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:32.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:32.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:39:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:39:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: osd.4 [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] boot 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: osdmap e33: 5 total, 5 up, 5 in 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi086 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 8544M 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:32.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:32.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:32.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:32.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 -- 172.21.15.123:0/3488966789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 msgr2=0x7f9af8103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 --2- 172.21.15.123:0/3488966789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8103a40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009900 tx=0x7f9aec02a410 comp rx=0 tx=0).stop 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 -- 172.21.15.123:0/3488966789 shutdown_connections 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 --2- 172.21.15.123:0/3488966789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af810cac0 0x7f9af810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 --2- 172.21.15.123:0/3488966789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af8103f80 0x7f9af810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 --2- 172.21.15.123:0/3488966789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8103a40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 -- 172.21.15.123:0/3488966789 >> 172.21.15.123:0/3488966789 conn(0x7f9af80fd0c0 msgr2=0x7f9af80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 -- 172.21.15.123:0/3488966789 shutdown_connections 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.560+0000 7f9afec2a640 1 -- 172.21.15.123:0/3488966789 wait complete. 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 Processor -- start 2024-07-24T21:39:32.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 -- start start 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 0x7f9af81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af810cac0 0x7f9af819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8112720 con 0x7f9af8103f80 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81125a0 con 0x7f9af810cac0 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afec2a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81128a0 con 0x7f9af8103660 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afc9c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afc9c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35688/0 (socket says 172.21.15.123:35688) 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afc9c6640 1 -- 172.21.15.123:0/373795928 learned_addr learned my addr 172.21.15.123:0/373795928 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9af7fff640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 0x7f9af81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:32.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afd1c7640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af810cac0 0x7f9af819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afc9c6640 1 -- 172.21.15.123:0/373795928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af810cac0 msgr2=0x7f9af819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.561+0000 7f9afc9c6640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af810cac0 0x7f9af819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afc9c6640 1 -- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 msgr2=0x7f9af81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afc9c6640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 0x7f9af81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afc9c6640 1 -- 172.21.15.123:0/373795928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af819ed40 con 0x7f9af8103660 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9af7fff640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 0x7f9af81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afc9c6640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9aec006010 tx=0x7f9aec0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec03f030 con 0x7f9af8103660 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af819efd0 con 0x7f9af8103660 2024-07-24T21:39:32.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec003800 con 0x7f9af8103660 2024-07-24T21:39:32.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec035070 con 0x7f9af8103660 2024-07-24T21:39:32.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.562+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a56f0 con 0x7f9af8103660 2024-07-24T21:39:32.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.563+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af8103a40 con 0x7f9af8103660 2024-07-24T21:39:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.565+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f9aec003cb0 con 0x7f9af8103660 2024-07-24T21:39:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.565+0000 7f9af5ffb640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 0x7f9ae80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.565+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4316+0+0 (secure 0 0 0) 0x7f9aec1039a0 con 0x7f9af8103660 2024-07-24T21:39:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.566+0000 7f9af7fff640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 0x7f9ae80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:32.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.566+0000 7f9af7fff640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 0x7f9ae80b60e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4005d90 tx=0x7f9ae4005d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:32.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.568+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9aec0cbef0 con 0x7f9af8103660 2024-07-24T21:39:32.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:32.762+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9af8062f80 con 0x7f9ae80b3c20 2024-07-24T21:39:33.171 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:32 smithi123 ceph-mon[37143]: osdmap e34: 5 total, 5 up, 5 in 2024-07-24T21:39:33.171 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:32 smithi123 ceph-mon[37143]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:33.171 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:32 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:33.171 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:32 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:33.171 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:32 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:33.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:32 smithi086 ceph-mon[37128]: osdmap e34: 5 total, 5 up, 5 in 2024-07-24T21:39:33.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:32 smithi086 ceph-mon[37128]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:33.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:32 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:33.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:32 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:33.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:32 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:32 smithi060 ceph-mon[33000]: osdmap e34: 5 total, 5 up, 5 in 2024-07-24T21:39:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:32 smithi060 ceph-mon[33000]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:32 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:32 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:32 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:34.195 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:33 smithi123 ceph-mon[37143]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:34.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:33 smithi086 ceph-mon[37128]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:34.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:33 smithi060 ceph-mon[33000]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:35.231 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:34 smithi123 ceph-mon[37143]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:35.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:34 smithi086 ceph-mon[37128]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:35.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:34 smithi060 ceph-mon[33000]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:36.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/170582474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]': finished 2024-07-24T21:39:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: osdmap e35: 6 total, 5 up, 6 in 2024-07-24T21:39:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/1642152922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/170582474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]': finished 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: osdmap e35: 6 total, 5 up, 6 in 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/1642152922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/170582474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]: dispatch 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a841523e-44a1-4112-81ab-b61d962ed02e"}]': finished 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: osdmap e35: 6 total, 5 up, 6 in 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:36.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/1642152922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:37.196 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:36 smithi086 ceph-mon[37128]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:37.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:36 smithi123 ceph-mon[37143]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:37.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:36 smithi060 ceph-mon[33000]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:39.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:39 smithi086 ceph-mon[37128]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:39.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:39 smithi060 ceph-mon[33000]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:39.376 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:39 smithi123 ceph-mon[37143]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:41.160 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:41 smithi123 ceph-mon[37143]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:41.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:41 smithi060 ceph-mon[33000]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:41.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:41 smithi086 ceph-mon[37128]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:42.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-24T21:39:42.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:42.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-24T21:39:42.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-24T21:39:42.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:43.317 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:43 smithi123 ceph-mon[37143]: Deploying daemon osd.5 on smithi123 2024-07-24T21:39:43.317 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:43 smithi123 ceph-mon[37143]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:43 smithi060 ceph-mon[33000]: Deploying daemon osd.5 on smithi123 2024-07-24T21:39:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:43 smithi060 ceph-mon[33000]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:43 smithi086 ceph-mon[37128]: Deploying daemon osd.5 on smithi123 2024-07-24T21:39:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:43 smithi086 ceph-mon[37128]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:45.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:45 smithi060 ceph-mon[33000]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:45.349 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:45 smithi123 ceph-mon[37143]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:45.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:45 smithi086 ceph-mon[37128]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:47.316 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:47 smithi123 ceph-mon[37143]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:47.316 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:47.316 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:47.316 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:47 smithi060 ceph-mon[33000]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:47 smithi086 ceph-mon[37128]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:49.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:49 smithi060 ceph-mon[33000]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:49 smithi123 ceph-mon[37143]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:49.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:49 smithi086 ceph-mon[37128]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:49.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.849+0000 7f9af5ffb640 1 -- 172.21.15.123:0/373795928 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9af8062f80 con 0x7f9ae80b3c20 2024-07-24T21:39:49.851 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 msgr2=0x7f9ae80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 0x7f9ae80b60e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4005d90 tx=0x7f9ae4005d20 comp rx=0 tx=0).stop 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 msgr2=0x7f9af8198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9aec006010 tx=0x7f9aec0042f0 comp rx=0 tx=0).stop 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 shutdown_connections 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ae80b3c20 0x7f9ae80b60e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:49.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9af810cac0 0x7f9af819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:49.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8103f80 0x7f9af81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:49.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 --2- 172.21.15.123:0/373795928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9af8103660 0x7f9af8198c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:49.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 >> 172.21.15.123:0/373795928 conn(0x7f9af80fd0c0 msgr2=0x7f9af80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:49.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 shutdown_connections 2024-07-24T21:39:49.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:39:49.852+0000 7f9afec2a640 1 -- 172.21.15.123:0/373795928 wait complete. 2024-07-24T21:39:50.462 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.5.service 2024-07-24T21:39:50.464 INFO:tasks.cephadm:Deploying osd.6 on smithi060 with /dev/vg_nvme/lv_2... 2024-07-24T21:39:50.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-24T21:39:50.743 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:39:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:50 smithi123 ceph-mon[37143]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:50.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:50 smithi123 ceph-mon[37143]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-24T21:39:50.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 21:39:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T21:39:50.739+0000 7f186e5eb640 -1 osd.5 0 log_to_monitors true 2024-07-24T21:39:51.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:51.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:51.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:50 smithi060 ceph-mon[33000]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:51.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:50 smithi060 ceph-mon[33000]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-24T21:39:51.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:51.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:51.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:50 smithi086 ceph-mon[37128]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:51.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:50 smithi086 ceph-mon[37128]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: osdmap e36: 6 total, 5 up, 6 in 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: osdmap e36: 6 total, 5 up, 6 in 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.236 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-24T21:39:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: osdmap e36: 6 total, 5 up, 6 in 2024-07-24T21:39:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:39:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-24T21:39:52.661 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-24T21:39:52.671 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-07-24T21:39:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-07-24T21:39:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00832311 s, 1.3 GB/s 2024-07-24T21:39:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-07-24T21:39:52.865 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T21:39:52.758+0000 7f186a56e640 -1 osd.5 0 waiting for initial osdmap 2024-07-24T21:39:53.242 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T21:39:52.879+0000 7f186238d640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:39:53.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: osdmap e37: 6 total, 5 up, 6 in 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: osdmap e38: 6 total, 5 up, 6 in 2024-07-24T21:39:53.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:52 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: osdmap e37: 6 total, 5 up, 6 in 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: osdmap e38: 6 total, 5 up, 6 in 2024-07-24T21:39:53.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:52 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:39:53.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: osdmap e37: 6 total, 5 up, 6 in 2024-07-24T21:39:53.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-07-24T21:39:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: from='osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122]' entity='osd.5' 2024-07-24T21:39:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: osdmap e38: 6 total, 5 up, 6 in 2024-07-24T21:39:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:52 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:53.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-07-24T21:39:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:39:54.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:39:54.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:39:54.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:54.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] boot 2024-07-24T21:39:54.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: osdmap e39: 6 total, 6 up, 6 in 2024-07-24T21:39:54.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:54.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:39:54.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:39:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] boot 2024-07-24T21:39:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: osdmap e39: 6 total, 6 up, 6 in 2024-07-24T21:39:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: osd.5 [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] boot 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: osdmap e39: 6 total, 6 up, 6 in 2024-07-24T21:39:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-24T21:39:55.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi123 2024-07-24T21:39:55.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 8544M 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.211 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:54 smithi060 ceph-mon[33000]: osdmap e40: 6 total, 6 up, 6 in 2024-07-24T21:39:55.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi123 2024-07-24T21:39:55.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 8544M 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:54 smithi123 ceph-mon[37143]: osdmap e40: 6 total, 6 up, 6 in 2024-07-24T21:39:55.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi123 2024-07-24T21:39:55.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 8544M 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:55.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:39:55.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:39:55.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:54 smithi086 ceph-mon[37128]: osdmap e40: 6 total, 6 up, 6 in 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 -- 172.21.15.60:0/3864864953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 msgr2=0x7fc16810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 --2- 172.21.15.60:0/3864864953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16810e1e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc160009ca0 tx=0x7fc16002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 -- 172.21.15.60:0/3864864953 shutdown_connections 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 --2- 172.21.15.60:0/3864864953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16810e1e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 --2- 172.21.15.60:0/3864864953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc1681074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 --2- 172.21.15.60:0/3864864953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 0x7fc168101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 -- 172.21.15.60:0/3864864953 >> 172.21.15.60:0/3864864953 conn(0x7fc1680fd0c0 msgr2=0x7fc1680ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:39:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 -- 172.21.15.60:0/3864864953 shutdown_connections 2024-07-24T21:39:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.432+0000 7fc16feda640 1 -- 172.21.15.60:0/3864864953 wait complete. 2024-07-24T21:39:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 Processor -- start 2024-07-24T21:39:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 -- start start 2024-07-24T21:39:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 0x7fc168199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc168199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111610 con 0x7fc1681079e0 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111490 con 0x7fc168101650 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16feda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111790 con 0x7fc168107060 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16e477640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16e477640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57054/0 (socket says 172.21.15.60:57054) 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16e477640 1 -- 172.21.15.60:0/63212605 learned_addr learned my addr 172.21.15.60:0/63212605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.433+0000 7fc16dc76640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 0x7fc168199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16d475640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc168199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 -- 172.21.15.60:0/63212605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 msgr2=0x7fc168199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc168199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 -- 172.21.15.60:0/63212605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 msgr2=0x7fc168199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 0x7fc168199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 -- 172.21.15.60:0/63212605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1681a4e60 con 0x7fc1681079e0 2024-07-24T21:39:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16d475640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc168199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16e477640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc160005cc0 tx=0x7fc16002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160003bf0 con 0x7fc1681079e0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1681a50f0 con 0x7fc1681079e0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc160003d90 con 0x7fc1681079e0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16002fbc0 con 0x7fc1681079e0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.434+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1681a5630 con 0x7fc1681079e0 2024-07-24T21:39:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.435+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc168101a30 con 0x7fc1681079e0 2024-07-24T21:39:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.437+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc16002fd60 con 0x7fc1681079e0 2024-07-24T21:39:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.438+0000 7fc15effd640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 0x7fc1540b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:39:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.438+0000 7fc16dc76640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 0x7fc1540b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:39:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.438+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4748+0+0 (secure 0 0 0) 0x7fc1601047d0 con 0x7fc1681079e0 2024-07-24T21:39:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.438+0000 7fc16dc76640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 0x7fc1540b60e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc16400bbe0 tx=0x7fc16400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:39:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.441+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc1600ccb70 con 0x7fc1681079e0 2024-07-24T21:39:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:39:55.646+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc168062f80 con 0x7fc1540b3c20 2024-07-24T21:39:56.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:55 smithi060 ceph-mon[33000]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:56.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:55 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:56.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:55 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:56.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:55 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:56.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:55 smithi060 ceph-mon[33000]: osdmap e41: 6 total, 6 up, 6 in 2024-07-24T21:39:56.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:55 smithi123 ceph-mon[37143]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:56.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:55 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:56.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:55 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:56.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:55 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:56.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:55 smithi123 ceph-mon[37143]: osdmap e41: 6 total, 6 up, 6 in 2024-07-24T21:39:56.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:55 smithi086 ceph-mon[37128]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:39:56.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:55 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:39:56.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:55 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:39:56.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:55 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:39:56.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:55 smithi086 ceph-mon[37128]: osdmap e41: 6 total, 6 up, 6 in 2024-07-24T21:39:57.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:56 smithi060 ceph-mon[33000]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:57.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:56 smithi123 ceph-mon[37143]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:57.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:56 smithi086 ceph-mon[37128]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]: dispatch 2024-07-24T21:39:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]': finished 2024-07-24T21:39:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: osdmap e42: 7 total, 6 up, 7 in 2024-07-24T21:39:59.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:39:59.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:39:59 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/295606490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]: dispatch 2024-07-24T21:39:59.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]': finished 2024-07-24T21:39:59.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: osdmap e42: 7 total, 6 up, 7 in 2024-07-24T21:39:59.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:39:59.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:39:59 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/295606490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]: dispatch 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3890205464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdea9ac1-bcee-44d9-be88-5b587cb1f7bf"}]': finished 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: osdmap e42: 7 total, 6 up, 7 in 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:39:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:39:59 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/295606490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:00.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:00 smithi060 ceph-mon[33000]: overall HEALTH_OK 2024-07-24T21:40:00.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:00 smithi123 ceph-mon[37143]: overall HEALTH_OK 2024-07-24T21:40:00.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:00 smithi086 ceph-mon[37128]: overall HEALTH_OK 2024-07-24T21:40:01.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:01 smithi060 ceph-mon[33000]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:01 smithi123 ceph-mon[37143]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:01 smithi086 ceph-mon[37128]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:03 smithi123 ceph-mon[37143]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-24T21:40:03.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:03 smithi060 ceph-mon[33000]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-24T21:40:03.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:03 smithi086 ceph-mon[37128]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-24T21:40:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:05 smithi060 ceph-mon[33000]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-24T21:40:05.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:05 smithi123 ceph-mon[37143]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-24T21:40:05.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:05 smithi086 ceph-mon[37128]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-24T21:40:06.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:06 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-24T21:40:06.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:06 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:06.210 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:06 smithi060 ceph-mon[33000]: Deploying daemon osd.6 on smithi060 2024-07-24T21:40:06.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:06 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-24T21:40:06.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:06 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:06.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:06 smithi123 ceph-mon[37143]: Deploying daemon osd.6 on smithi060 2024-07-24T21:40:06.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:06 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-24T21:40:06.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:06 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:06.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:06 smithi086 ceph-mon[37128]: Deploying daemon osd.6 on smithi060 2024-07-24T21:40:07.216 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:07 smithi060 ceph-mon[33000]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:07.446 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:07 smithi086 ceph-mon[37128]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:07 smithi123 ceph-mon[37143]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:09.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:09 smithi060 ceph-mon[33000]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:09 smithi123 ceph-mon[37143]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:09.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:09 smithi086 ceph-mon[37128]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:11.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:11 smithi060 ceph-mon[33000]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-24T21:40:11.341 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:11.341 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:11.341 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:11 smithi123 ceph-mon[37143]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-24T21:40:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:11.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:11 smithi086 ceph-mon[37128]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-24T21:40:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:13.330 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:13 smithi060 ceph-mon[33000]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:13.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:13 smithi123 ceph-mon[37143]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:13.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:13 smithi086 ceph-mon[37128]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:14.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.553+0000 7fc15effd640 1 -- 172.21.15.60:0/63212605 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc168062f80 con 0x7fc1540b3c20 2024-07-24T21:40:14.555 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 6 on host 'smithi060' 2024-07-24T21:40:14.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 msgr2=0x7fc1540b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 0x7fc1540b60e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc16400bbe0 tx=0x7fc16400b390 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 msgr2=0x7fc16819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc160005cc0 tx=0x7fc16002f380 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 shutdown_connections 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1540b3c20 0x7fc1540b60e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1681079e0 0x7fc16819a520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc168107060 0x7fc168199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 --2- 172.21.15.60:0/63212605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168101650 0x7fc168199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 >> 172.21.15.60:0/63212605 conn(0x7fc1680fd0c0 msgr2=0x7fc1680ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 shutdown_connections 2024-07-24T21:40:14.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:40:14.556+0000 7fc16feda640 1 -- 172.21.15.60:0/63212605 wait complete. 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:14.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:14 smithi060 ceph-mon[33000]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-24T21:40:14.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:14 smithi123 ceph-mon[37143]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:14 smithi086 ceph-mon[37128]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-24T21:40:15.156 DEBUG:teuthology.orchestra.run.smithi060:osd.6> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.6.service 2024-07-24T21:40:15.158 INFO:tasks.cephadm:Deploying osd.7 on smithi086 with /dev/vg_nvme/lv_2... 2024-07-24T21:40:15.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-24T21:40:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:40:16.734 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:16 smithi086 ceph-mon[37128]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-24T21:40:16.734 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:16 smithi086 ceph-mon[37128]: osdmap e43: 7 total, 6 up, 7 in 2024-07-24T21:40:16.734 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:16 smithi086 ceph-mon[37128]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:40:16.735 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:16 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:16.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-mon[33000]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-24T21:40:16.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-mon[33000]: osdmap e43: 7 total, 6 up, 7 in 2024-07-24T21:40:16.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-mon[33000]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:40:16.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:16.822 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T21:40:16.565+0000 7fb98e8e7640 -1 osd.6 0 waiting for initial osdmap 2024-07-24T21:40:16.822 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 21:40:16 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T21:40:16.577+0000 7fb98a70e640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:40:16.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:16 smithi123 ceph-mon[37143]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-24T21:40:16.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:16 smithi123 ceph-mon[37143]: osdmap e43: 7 total, 6 up, 7 in 2024-07-24T21:40:16.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:16 smithi123 ceph-mon[37143]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:40:16.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:16 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.076 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-24T21:40:17.174 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-24T21:40:17.187 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-07-24T21:40:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-07-24T21:40:17.187 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114039 s, 919 MB/s 2024-07-24T21:40:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-07-24T21:40:17.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:17 smithi060 ceph-mon[33000]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:17.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:17 smithi060 ceph-mon[33000]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:40:17.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:17 smithi060 ceph-mon[33000]: osdmap e44: 7 total, 6 up, 7 in 2024-07-24T21:40:17.717 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.717 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:17 smithi086 ceph-mon[37128]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:17 smithi086 ceph-mon[37128]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:40:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:17 smithi086 ceph-mon[37128]: osdmap e44: 7 total, 6 up, 7 in 2024-07-24T21:40:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-07-24T21:40:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:17 smithi123 ceph-mon[37143]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-07-24T21:40:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:17 smithi123 ceph-mon[37143]: from='osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:40:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:17 smithi123 ceph-mon[37143]: osdmap e44: 7 total, 6 up, 7 in 2024-07-24T21:40:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] boot 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: osdmap e45: 7 total, 7 up, 7 in 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:18.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:18.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] boot 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: osdmap e45: 7 total, 7 up, 7 in 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:18.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:18.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:18.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:18.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:19.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:40:19.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: osd.6 [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] boot 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: osdmap e45: 7 total, 7 up, 7 in 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:19.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 -- 172.21.15.86:0/4039570187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10cc20 msgr2=0x7fdc1c10f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 --2- 172.21.15.86:0/4039570187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10cc20 0x7fdc1c10f010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009ca0 tx=0x7fdc1802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 -- 172.21.15.86:0/4039570187 shutdown_connections 2024-07-24T21:40:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 --2- 172.21.15.86:0/4039570187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10cc20 0x7fdc1c10f010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 --2- 172.21.15.86:0/4039570187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c10c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 --2- 172.21.15.86:0/4039570187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1c1011e0 0x7fdc1c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 -- 172.21.15.86:0/4039570187 >> 172.21.15.86:0/4039570187 conn(0x7fdc1c076980 msgr2=0x7fdc1c076d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:40:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.605+0000 7fdc24444640 1 -- 172.21.15.86:0/4039570187 shutdown_connections 2024-07-24T21:40:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 -- 172.21.15.86:0/4039570187 wait complete. 2024-07-24T21:40:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 Processor -- start 2024-07-24T21:40:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 -- start start 2024-07-24T21:40:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1011e0 0x7fdc1c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1c10cc20 0x7fdc1c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c102ae0 con 0x7fdc1c1011e0 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c102960 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc24444640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c102c60 con 0x7fdc1c10cc20 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.606+0000 7fdc219df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54478/0 (socket says 172.21.15.86:54478) 2024-07-24T21:40:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 -- 172.21.15.86:0/3458139648 learned_addr learned my addr 172.21.15.86:0/3458139648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc221e0640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1011e0 0x7fdc1c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 -- 172.21.15.86:0/3458139648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1c10cc20 msgr2=0x7fdc1c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1c10cc20 0x7fdc1c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 -- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1011e0 msgr2=0x7fdc1c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1011e0 0x7fdc1c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 -- 172.21.15.86:0/3458139648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1a0e00 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc219df640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c920 tx=0x7fdc0c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:40:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007c70 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c007e10 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1a10f0 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c018820 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.607+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a77b0 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.608+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c1015c0 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.610+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fdc0c00f450 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.610+0000 7fdc137fe640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 0x7fdc080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.611+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5180+0+0 (secure 0 0 0) 0x7fdc0c0e3370 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.611+0000 7fdc221e0640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 0x7fdc080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.611+0000 7fdc221e0640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 0x7fdc080b60e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc140058c0 tx=0x7fdc1400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:40:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.614+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdc0c014030 con 0x7fdc1c10a2f0 2024-07-24T21:40:19.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:19 smithi086 ceph-mon[37128]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:19.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:19 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi060 2024-07-24T21:40:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:19 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 5697M 2024-07-24T21:40:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:19 smithi086 ceph-mon[37128]: osdmap e46: 7 total, 7 up, 7 in 2024-07-24T21:40:19.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:19.813+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdc1c100680 con 0x7fdc080b3c20 2024-07-24T21:40:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:19 smithi123 ceph-mon[37143]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:19 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi060 2024-07-24T21:40:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:19 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 5697M 2024-07-24T21:40:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:19 smithi123 ceph-mon[37143]: osdmap e46: 7 total, 7 up, 7 in 2024-07-24T21:40:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:19 smithi060 ceph-mon[33000]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:19 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi060 2024-07-24T21:40:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:19 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 5697M 2024-07-24T21:40:20.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:19 smithi060 ceph-mon[33000]: osdmap e46: 7 total, 7 up, 7 in 2024-07-24T21:40:20.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:20.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:20.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:21.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:21.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:21 smithi086 ceph-mon[37128]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:21 smithi086 ceph-mon[37128]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:21 smithi123 ceph-mon[37143]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:21 smithi123 ceph-mon[37143]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:21 smithi060 ceph-mon[33000]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:21 smithi060 ceph-mon[33000]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:22.780 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/2975720112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:22.780 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:22 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:22.780 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:22 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]': finished 2024-07-24T21:40:22.780 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:22 smithi086 ceph-mon[37128]: osdmap e47: 8 total, 7 up, 8 in 2024-07-24T21:40:22.780 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:22 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:22.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/2975720112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:22.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:22 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:22.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:22 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]': finished 2024-07-24T21:40:22.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:22 smithi123 ceph-mon[37143]: osdmap e47: 8 total, 7 up, 8 in 2024-07-24T21:40:22.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:22 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:23.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/2975720112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:23.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:22 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]: dispatch 2024-07-24T21:40:23.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:22 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc920fda-05a3-4fde-92f9-efdfb7776621"}]': finished 2024-07-24T21:40:23.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:22 smithi060 ceph-mon[33000]: osdmap e47: 8 total, 7 up, 8 in 2024-07-24T21:40:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:22 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:23 smithi123 ceph-mon[37143]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:23 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/3024438786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:24.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:23 smithi086 ceph-mon[37128]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:24.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:23 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/3024438786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:23 smithi060 ceph-mon[33000]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:24.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:23 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/3024438786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:25.936 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:25 smithi086 ceph-mon[37128]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:25 smithi123 ceph-mon[37143]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:26.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:25 smithi060 ceph-mon[33000]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:27.684 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:27 smithi086 ceph-mon[37128]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:27.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:27 smithi060 ceph-mon[33000]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:27 smithi123 ceph-mon[37143]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:29.852 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:29 smithi086 ceph-mon[37128]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:29.852 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-24T21:40:29.852 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:29 smithi123 ceph-mon[37143]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-24T21:40:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:29 smithi060 ceph-mon[33000]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-24T21:40:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:30.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:30 smithi086 ceph-mon[37128]: Deploying daemon osd.7 on smithi086 2024-07-24T21:40:30.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:30 smithi123 ceph-mon[37143]: Deploying daemon osd.7 on smithi086 2024-07-24T21:40:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:30 smithi060 ceph-mon[33000]: Deploying daemon osd.7 on smithi086 2024-07-24T21:40:31.852 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:31 smithi086 ceph-mon[37128]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:31.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:31 smithi123 ceph-mon[37143]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:31 smithi060 ceph-mon[33000]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:33 smithi086 ceph-mon[37128]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:33.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:33 smithi123 ceph-mon[37143]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:34.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:33 smithi060 ceph-mon[33000]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:34.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:34.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:34.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:34.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:34.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:34.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:35.716 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:35 smithi086 ceph-mon[37128]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:35.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:35 smithi123 ceph-mon[37143]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:36.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:35 smithi060 ceph-mon[33000]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.354+0000 7fdc137fe640 1 -- 172.21.15.86:0/3458139648 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc1c100680 con 0x7fdc080b3c20 2024-07-24T21:40:37.355 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 7 on host 'smithi086' 2024-07-24T21:40:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 msgr2=0x7fdc080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 0x7fdc080b60e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc140058c0 tx=0x7fdc1400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 msgr2=0x7fdc1c19b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c920 tx=0x7fdc0c00cde0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 shutdown_connections 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdc080b3c20 0x7fdc080b60e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc1c10cc20 0x7fdc1c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc1c10a2f0 0x7fdc1c19b4f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 --2- 172.21.15.86:0/3458139648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1011e0 0x7fdc1c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 >> 172.21.15.86:0/3458139648 conn(0x7fdc1c076980 msgr2=0x7fdc1c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 shutdown_connections 2024-07-24T21:40:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:40:37.356+0000 7fdc24444640 1 -- 172.21.15.86:0/3458139648 wait complete. 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.701 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:37.971 DEBUG:teuthology.orchestra.run.smithi086:osd.7> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.7.service 2024-07-24T21:40:37.973 INFO:tasks.cephadm:Deploying osd.8 on smithi123 with /dev/vg_nvme/lv_2... 2024-07-24T21:40:37.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-24T21:40:38.232 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:40:38.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:38 smithi086 ceph-mon[37128]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:38.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:38 smithi086 ceph-mon[37128]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:38.608 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:38 smithi123 ceph-mon[37143]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:38.608 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:38 smithi123 ceph-mon[37143]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:38.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:38 smithi060 ceph-mon[33000]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:38.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:38 smithi060 ceph-mon[33000]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-24T21:40:39.572 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:39.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-24T21:40:39.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: osdmap e48: 8 total, 7 up, 8 in 2024-07-24T21:40:39.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:39.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:39 smithi086 ceph-mon[37128]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: osdmap e48: 8 total, 7 up, 8 in 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:39 smithi123 ceph-mon[37143]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.802 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-24T21:40:39.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:39.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-24T21:40:39.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: osdmap e48: 8 total, 7 up, 8 in 2024-07-24T21:40:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: from='osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:39 smithi060 ceph-mon[33000]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:40:39.883 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-24T21:40:39.896 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-07-24T21:40:39.896 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-07-24T21:40:39.896 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115458 s, 908 MB/s 2024-07-24T21:40:39.896 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-07-24T21:40:40.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-07-24T21:40:40.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: osdmap e49: 8 total, 7 up, 8 in 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: from='osd.7 ' entity='osd.7' 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: osdmap e50: 8 total, 7 up, 8 in 2024-07-24T21:40:40.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.546 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T21:40:40.395+0000 7f4f04bb3640 -1 osd.7 0 waiting for initial osdmap 2024-07-24T21:40:40.546 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 21:40:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T21:40:40.490+0000 7f4efc1be640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:40:40.692 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:40:40.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:40:40.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: osdmap e49: 8 total, 7 up, 8 in 2024-07-24T21:40:40.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: from='osd.7 ' entity='osd.7' 2024-07-24T21:40:40.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: osdmap e50: 8 total, 7 up, 8 in 2024-07-24T21:40:40.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: osdmap e49: 8 total, 7 up, 8 in 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: from='osd.7 ' entity='osd.7' 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: osdmap e50: 8 total, 7 up, 8 in 2024-07-24T21:40:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:40:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] boot 2024-07-24T21:40:41.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: osdmap e51: 8 total, 8 up, 8 in 2024-07-24T21:40:41.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:41 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] boot 2024-07-24T21:40:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: osdmap e51: 8 total, 8 up, 8 in 2024-07-24T21:40:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:41 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:40:41.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:40:41.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: osd.7 [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] boot 2024-07-24T21:40:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: osdmap e51: 8 total, 8 up, 8 in 2024-07-24T21:40:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:41 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 -- 172.21.15.123:0/3867176984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 msgr2=0x7f298c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 --2- 172.21.15.123:0/3867176984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c101a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f297c009ca0 tx=0x7f297c02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 -- 172.21.15.123:0/3867176984 shutdown_connections 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 --2- 172.21.15.123:0/3867176984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298c1079e0 0x7f298c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 --2- 172.21.15.123:0/3867176984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 0x7f298c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 --2- 172.21.15.123:0/3867176984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c101a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 -- 172.21.15.123:0/3867176984 >> 172.21.15.123:0/3867176984 conn(0x7f298c0fd0c0 msgr2=0x7f298c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 -- 172.21.15.123:0/3867176984 shutdown_connections 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.190+0000 7f299207f640 1 -- 172.21.15.123:0/3867176984 wait complete. 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.191+0000 7f299207f640 1 Processor -- start 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.191+0000 7f299207f640 1 -- start start 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.191+0000 7f299207f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f299207f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 0x7f298c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46344/0 (socket says 172.21.15.123:46344) 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 -- 172.21.15.123:0/2057537526 learned_addr learned my addr 172.21.15.123:0/2057537526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-07-24T21:40:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298c1079e0 0x7f298c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1115f0 con 0x7f298c1079e0 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f298c111470 con 0x7f298c107060 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f298c111770 con 0x7f298c101650 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298affd640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 0x7f298c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 -- 172.21.15.123:0/2057537526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 msgr2=0x7f298c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 0x7f298c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 -- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298c1079e0 msgr2=0x7f298c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298c1079e0 0x7f298c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.192+0000 7f298b7fe640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c1a7010 con 0x7f298c101650 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f298b7fe640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c003bf0 con 0x7f298c101650 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c1a72a0 con 0x7f298c101650 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c003d90 con 0x7f298c101650 2024-07-24T21:40:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c02fc20 con 0x7f298c101650 2024-07-24T21:40:42.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.193+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1a77e0 con 0x7f298c101650 2024-07-24T21:40:42.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.194+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298c101a30 con 0x7f298c101650 2024-07-24T21:40:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.195+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f297c0046a0 con 0x7f298c101650 2024-07-24T21:40:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.196+0000 7f2988ff9640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 0x7f29780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:40:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.196+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5645+0+0 (secure 0 0 0) 0x7f297c1048a0 con 0x7f298c101650 2024-07-24T21:40:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.196+0000 7f298affd640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 0x7f29780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:40:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.197+0000 7f298affd640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 0x7f29780b60e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f298c0ffb20 tx=0x7f297400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:40:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.199+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f297c0cc960 con 0x7f298c101650 2024-07-24T21:40:42.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:42.392+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f298c062f80 con 0x7f29780b3c20 2024-07-24T21:40:42.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi086 2024-07-24T21:40:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 5696M 2024-07-24T21:40:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:42 smithi123 ceph-mon[37143]: osdmap e52: 8 total, 8 up, 8 in 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi086 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 5696M 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:42 smithi086 ceph-mon[37128]: osdmap e52: 8 total, 8 up, 8 in 2024-07-24T21:40:42.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi086 2024-07-24T21:40:42.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 5696M 2024-07-24T21:40:42.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:40:42.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:40:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:42 smithi060 ceph-mon[33000]: osdmap e52: 8 total, 8 up, 8 in 2024-07-24T21:40:43.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:43 smithi123 ceph-mon[37143]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:43.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:43 smithi123 ceph-mon[37143]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:43.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:43 smithi123 ceph-mon[37143]: osdmap e53: 8 total, 8 up, 8 in 2024-07-24T21:40:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:43 smithi086 ceph-mon[37128]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:43 smithi086 ceph-mon[37128]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:43 smithi086 ceph-mon[37128]: osdmap e53: 8 total, 8 up, 8 in 2024-07-24T21:40:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:43 smithi060 ceph-mon[33000]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:43 smithi060 ceph-mon[33000]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:40:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:43 smithi060 ceph-mon[33000]: osdmap e53: 8 total, 8 up, 8 in 2024-07-24T21:40:45.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/4069223818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]': finished 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: osdmap e54: 9 total, 8 up, 9 in 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:40:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:45 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/2480166878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:45.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:45.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/4069223818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]': finished 2024-07-24T21:40:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: osdmap e54: 9 total, 8 up, 9 in 2024-07-24T21:40:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:40:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/2480166878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/4069223818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]: dispatch 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16f5ae1a-e0de-4d8d-82da-211c1a11328a"}]': finished 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: osdmap e54: 9 total, 8 up, 9 in 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:40:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:45 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/2480166878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:40:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:47 smithi086 ceph-mon[37128]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:47.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:47 smithi060 ceph-mon[33000]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:47.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:47 smithi123 ceph-mon[37143]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:40:49.776 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:49 smithi123 ceph-mon[37143]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:49.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:49 smithi086 ceph-mon[37128]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:49 smithi060 ceph-mon[33000]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:40:51.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:51 smithi123 ceph-mon[37143]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-24T21:40:51.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:51 smithi086 ceph-mon[37128]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-24T21:40:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:51 smithi060 ceph-mon[33000]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-07-24T21:40:52.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:52 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-24T21:40:52.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:52 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:52.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:52 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-24T21:40:52.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:52 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:52.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:52 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-24T21:40:52.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:52 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:40:53.621 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:53 smithi123 ceph-mon[37143]: Deploying daemon osd.8 on smithi123 2024-07-24T21:40:53.622 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:53 smithi123 ceph-mon[37143]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-07-24T21:40:53.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:53 smithi086 ceph-mon[37128]: Deploying daemon osd.8 on smithi123 2024-07-24T21:40:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:53 smithi086 ceph-mon[37128]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-07-24T21:40:53.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:53 smithi060 ceph-mon[33000]: Deploying daemon osd.8 on smithi123 2024-07-24T21:40:53.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:53 smithi060 ceph-mon[33000]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-07-24T21:40:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:55 smithi123 ceph-mon[37143]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:55.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:55 smithi086 ceph-mon[37128]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:55 smithi060 ceph-mon[33000]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:40:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:57 smithi123 ceph-mon[37143]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:40:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:57.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:57 smithi086 ceph-mon[37128]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:40:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:57 smithi060 ceph-mon[33000]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:40:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:40:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:40:59.549 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:40:59 smithi123 ceph-mon[37143]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:40:59.588 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 8 on host 'smithi123' 2024-07-24T21:40:59.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.586+0000 7f2988ff9640 1 -- 172.21.15.123:0/2057537526 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f298c062f80 con 0x7f29780b3c20 2024-07-24T21:40:59.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 msgr2=0x7f29780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:59.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 0x7f29780b60e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f298c0ffb20 tx=0x7f297400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 msgr2=0x7f298c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:40:59.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c0319f0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 shutdown_connections 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29780b3c20 0x7f29780b60e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298c1079e0 0x7f298c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f298c107060 0x7f298c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 --2- 172.21.15.123:0/2057537526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298c101650 0x7f298c197830 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 >> 172.21.15.123:0/2057537526 conn(0x7f298c0fd0c0 msgr2=0x7f298c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 shutdown_connections 2024-07-24T21:40:59.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:40:59.589+0000 7f299207f640 1 -- 172.21.15.123:0/2057537526 wait complete. 2024-07-24T21:40:59.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:40:59 smithi086 ceph-mon[37128]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:40:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:40:59 smithi060 ceph-mon[33000]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:41:00.228 DEBUG:teuthology.orchestra.run.smithi123:osd.8> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.8.service 2024-07-24T21:41:00.230 INFO:tasks.cephadm:Deploying osd.9 on smithi060 with /dev/vg_nvme/lv_1... 2024-07-24T21:41:00.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-24T21:41:00.508 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:41:00.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:00 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:00.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:00 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:00.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:00 smithi123 ceph-mon[37143]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:00.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:00 smithi123 ceph-mon[37143]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:00.744 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 21:41:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T21:41:00.489+0000 7f37cbeeb640 -1 osd.8 0 log_to_monitors true 2024-07-24T21:41:00.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:00 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:00.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:00 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:00.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:00 smithi060 ceph-mon[33000]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:00.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:00 smithi060 ceph-mon[33000]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:01.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:00 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:00 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:00 smithi086 ceph-mon[37128]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:00 smithi086 ceph-mon[37128]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-24T21:41:01.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: osdmap e55: 9 total, 8 up, 9 in 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:01.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-24T21:41:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: osdmap e55: 9 total, 8 up, 9 in 2024-07-24T21:41:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:02.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: osdmap e55: 9 total, 8 up, 9 in 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:02.581 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-24T21:41:02.659 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-24T21:41:02.670 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-07-24T21:41:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-07-24T21:41:02.670 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00887729 s, 1.2 GB/s 2024-07-24T21:41:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-07-24T21:41:02.743 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T21:41:02.490+0000 7f37c7e6e640 -1 osd.8 0 waiting for initial osdmap 2024-07-24T21:41:02.743 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T21:41:02.667+0000 7f37bfc8d640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:41:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-mon[37143]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:41:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-mon[37143]: osdmap e56: 9 total, 8 up, 9 in 2024-07-24T21:41:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:02 smithi123 ceph-mon[37143]: from='osd.8 ' entity='osd.8' 2024-07-24T21:41:02.858 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:02 smithi060 ceph-mon[33000]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:41:02.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:02 smithi060 ceph-mon[33000]: osdmap e56: 9 total, 8 up, 9 in 2024-07-24T21:41:02.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:02.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:02.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:02 smithi060 ceph-mon[33000]: from='osd.8 ' entity='osd.8' 2024-07-24T21:41:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:02 smithi086 ceph-mon[37128]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:41:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:02 smithi086 ceph-mon[37128]: osdmap e56: 9 total, 8 up, 9 in 2024-07-24T21:41:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:02 smithi086 ceph-mon[37128]: from='osd.8 ' entity='osd.8' 2024-07-24T21:41:03.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-07-24T21:41:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:41:03.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:41:03.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:41:03.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:03.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: osdmap e57: 9 total, 8 up, 9 in 2024-07-24T21:41:03.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:41:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:41:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: osdmap e57: 9 total, 8 up, 9 in 2024-07-24T21:41:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:41:04.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:41:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: osdmap e57: 9 total, 8 up, 9 in 2024-07-24T21:41:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] boot 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: osdmap e58: 9 total, 9 up, 9 in 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:04.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] boot 2024-07-24T21:41:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: osdmap e58: 9 total, 9 up, 9 in 2024-07-24T21:41:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: osd.8 [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] boot 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: osdmap e58: 9 total, 9 up, 9 in 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:05.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:05.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:05.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:05.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:05.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 -- 172.21.15.60:0/1359197196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 msgr2=0x7f662410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 --2- 172.21.15.60:0/1359197196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662410ef20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f661c009ca0 tx=0x7f661c02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 -- 172.21.15.60:0/1359197196 shutdown_connections 2024-07-24T21:41:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 --2- 172.21.15.60:0/1359197196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662410ef20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 --2- 172.21.15.60:0/1359197196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 0x7f662410c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 --2- 172.21.15.60:0/1359197196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 0x7f6624103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 -- 172.21.15.60:0/1359197196 >> 172.21.15.60:0/1359197196 conn(0x7f66240fd0c0 msgr2=0x7f66240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:41:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 -- 172.21.15.60:0/1359197196 shutdown_connections 2024-07-24T21:41:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.126+0000 7f662baaa640 1 -- 172.21.15.60:0/1359197196 wait complete. 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 Processor -- start 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 -- start start 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 0x7f6624198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 0x7f6624199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6624112720 con 0x7f662410cac0 2024-07-24T21:41:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66241125a0 con 0x7f6624103f80 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662baaa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66241128a0 con 0x7f6624103660 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662a047640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662a047640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52436/0 (socket says 172.21.15.60:52436) 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f662a047640 1 -- 172.21.15.60:0/1356546565 learned_addr learned my addr 172.21.15.60:0/1356546565 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f6629045640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 0x7f6624199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.127+0000 7f6629846640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 0x7f6624198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 -- 172.21.15.60:0/1356546565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 msgr2=0x7f6624198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 0x7f6624198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 -- 172.21.15.60:0/1356546565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 msgr2=0x7f6624199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 0x7f6624199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 -- 172.21.15.60:0/1356546565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662419ecc0 con 0x7f662410cac0 2024-07-24T21:41:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662a047640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f661c005cc0 tx=0x7f661c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c003bf0 con 0x7f662410cac0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f661c003d90 con 0x7f662410cac0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662419ef50 con 0x7f662410cac0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c02fbc0 con 0x7f662410cac0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.128+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66241a5670 con 0x7f662410cac0 2024-07-24T21:41:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.129+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6624103a40 con 0x7f662410cac0 2024-07-24T21:41:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.130+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f661c02fd60 con 0x7f662410cac0 2024-07-24T21:41:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.131+0000 7f661affd640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 0x7f66100b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.131+0000 7f6629846640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 0x7f66100b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.131+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6044+0+0 (secure 0 0 0) 0x7f661c104b50 con 0x7f662410cac0 2024-07-24T21:41:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.132+0000 7f6629846640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 0x7f66100b6160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6620005fd0 tx=0x7f662000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.134+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f661c0cca80 con 0x7f662410cac0 2024-07-24T21:41:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:05.338+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6624062f80 con 0x7f66100b3ca0 2024-07-24T21:41:05.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:05.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi123 2024-07-24T21:41:05.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 5696M 2024-07-24T21:41:05.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: osdmap e59: 9 total, 9 up, 9 in 2024-07-24T21:41:05.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:05.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:05.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi123 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 5696M 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: osdmap e59: 9 total, 9 up, 9 in 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi123 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 5696M 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: osdmap e59: 9 total, 9 up, 9 in 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:06.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:06.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:06 smithi123 ceph-mon[37143]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:06.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:06 smithi123 ceph-mon[37143]: osdmap e60: 9 total, 9 up, 9 in 2024-07-24T21:41:06.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:06 smithi060 ceph-mon[33000]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:06.965 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:06 smithi060 ceph-mon[33000]: osdmap e60: 9 total, 9 up, 9 in 2024-07-24T21:41:07.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:06 smithi086 ceph-mon[37128]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:07.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:06 smithi086 ceph-mon[37128]: osdmap e60: 9 total, 9 up, 9 in 2024-07-24T21:41:07.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:07 smithi060 ceph-mon[33000]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:07.946 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:07 smithi086 ceph-mon[37128]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:07 smithi123 ceph-mon[37143]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:08.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:08 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]: dispatch 2024-07-24T21:41:08.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:08 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]': finished 2024-07-24T21:41:08.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:08 smithi060 ceph-mon[33000]: osdmap e61: 10 total, 9 up, 10 in 2024-07-24T21:41:08.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:08.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:08 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2761754074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:08.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:08 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]: dispatch 2024-07-24T21:41:08.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:08 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]': finished 2024-07-24T21:41:08.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:08 smithi086 ceph-mon[37128]: osdmap e61: 10 total, 9 up, 10 in 2024-07-24T21:41:08.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:08.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:08 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2761754074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:08 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]: dispatch 2024-07-24T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:08 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3256516691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b6df3-b9da-4c55-b541-2dbce29fcf6f"}]': finished 2024-07-24T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:08 smithi123 ceph-mon[37143]: osdmap e61: 10 total, 9 up, 10 in 2024-07-24T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:08 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2761754074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:09.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:09 smithi123 ceph-mon[37143]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-07-24T21:41:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:09 smithi086 ceph-mon[37128]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-07-24T21:41:10.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:09 smithi060 ceph-mon[33000]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-07-24T21:41:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:11 smithi123 ceph-mon[37143]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:41:12.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:11 smithi086 ceph-mon[37128]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:41:12.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:11 smithi060 ceph-mon[33000]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-24T21:41:13.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:13 smithi060 ceph-mon[33000]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-24T21:41:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:13 smithi123 ceph-mon[37143]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-24T21:41:14.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:13 smithi086 ceph-mon[37128]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-24T21:41:15.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:15 smithi060 ceph-mon[33000]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-24T21:41:15.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-24T21:41:15.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:15.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:15 smithi123 ceph-mon[37143]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-24T21:41:15.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-24T21:41:15.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:16.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:15 smithi086 ceph-mon[37128]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-24T21:41:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-24T21:41:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:16.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:16 smithi060 ceph-mon[33000]: Deploying daemon osd.9 on smithi060 2024-07-24T21:41:16.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:16 smithi123 ceph-mon[37143]: Deploying daemon osd.9 on smithi060 2024-07-24T21:41:17.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:16 smithi086 ceph-mon[37128]: Deploying daemon osd.9 on smithi060 2024-07-24T21:41:17.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:17 smithi060 ceph-mon[33000]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:41:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:17 smithi123 ceph-mon[37143]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:41:18.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:17 smithi086 ceph-mon[37128]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:41:19.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:19 smithi060 ceph-mon[33000]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:19 smithi123 ceph-mon[37143]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:20.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:19 smithi086 ceph-mon[37128]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:20.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:20.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:20.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:21.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:21.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:21.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:21 smithi123 ceph-mon[37143]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:21 smithi086 ceph-mon[37128]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:22.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:21 smithi060 ceph-mon[33000]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:23.637 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 9 on host 'smithi060' 2024-07-24T21:41:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.635+0000 7f661affd640 1 -- 172.21.15.60:0/1356546565 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6624062f80 con 0x7f66100b3ca0 2024-07-24T21:41:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 msgr2=0x7f66100b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 0x7f66100b6160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6620005fd0 tx=0x7f662000b390 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 msgr2=0x7f662419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f661c005cc0 tx=0x7f661c02f380 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 shutdown_connections 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66100b3ca0 0x7f66100b6160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f662410cac0 0x7f662419e4f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6624103f80 0x7f6624199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 --2- 172.21.15.60:0/1356546565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6624103660 0x7f6624198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.638+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 >> 172.21.15.60:0/1356546565 conn(0x7f66240fd0c0 msgr2=0x7f66240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.639+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 shutdown_connections 2024-07-24T21:41:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:41:23.639+0000 7f662baaa640 1 -- 172.21.15.60:0/1356546565 wait complete. 2024-07-24T21:41:23.976 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:24.373 DEBUG:teuthology.orchestra.run.smithi060:osd.9> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.9.service 2024-07-24T21:41:24.375 INFO:tasks.cephadm:Deploying osd.10 on smithi086 with /dev/vg_nvme/lv_1... 2024-07-24T21:41:24.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-24T21:41:24.638 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:41:24.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:24 smithi060 ceph-mon[33000]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-24T21:41:24.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:24 smithi086 ceph-mon[37128]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-24T21:41:24.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:24 smithi123 ceph-mon[37143]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-24T21:41:25.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:25 smithi060 ceph-mon[33000]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:25.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:25 smithi060 ceph-mon[33000]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-24T21:41:25.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:25 smithi060 ceph-mon[33000]: osdmap e62: 10 total, 9 up, 10 in 2024-07-24T21:41:25.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:25 smithi060 ceph-mon[33000]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:41:25.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:25 smithi123 ceph-mon[37143]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:25 smithi123 ceph-mon[37143]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-24T21:41:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:25 smithi123 ceph-mon[37143]: osdmap e62: 10 total, 9 up, 10 in 2024-07-24T21:41:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:25 smithi123 ceph-mon[37143]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:41:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:25 smithi086 ceph-mon[37128]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:26.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:25 smithi086 ceph-mon[37128]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-24T21:41:26.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:25 smithi086 ceph-mon[37128]: osdmap e62: 10 total, 9 up, 10 in 2024-07-24T21:41:26.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:25 smithi086 ceph-mon[37128]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-07-24T21:41:26.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.422 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-24T21:41:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-24T21:41:26.528 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-07-24T21:41:26.528 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-07-24T21:41:26.528 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135523 s, 774 MB/s 2024-07-24T21:41:26.528 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-07-24T21:41:26.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-mon[33000]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:41:26.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-mon[33000]: osdmap e63: 10 total, 9 up, 10 in 2024-07-24T21:41:26.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-mon[33000]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' 2024-07-24T21:41:26.735 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T21:41:26.455+0000 7fe1f9605640 -1 osd.9 0 waiting for initial osdmap 2024-07-24T21:41:26.735 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 21:41:26 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T21:41:26.474+0000 7fe1f1424640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:41:26.902 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:26 smithi086 ceph-mon[37128]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:41:26.902 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:26 smithi086 ceph-mon[37128]: osdmap e63: 10 total, 9 up, 10 in 2024-07-24T21:41:26.902 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.902 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.902 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:26 smithi086 ceph-mon[37128]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' 2024-07-24T21:41:26.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:26 smithi123 ceph-mon[37143]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-07-24T21:41:26.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:26 smithi123 ceph-mon[37143]: osdmap e63: 10 total, 9 up, 10 in 2024-07-24T21:41:26.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:26.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:26 smithi123 ceph-mon[37143]: from='osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111]' entity='osd.9' 2024-07-24T21:41:27.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-07-24T21:41:27.449 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T21:41:27.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] boot 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: osdmap e64: 10 total, 10 up, 10 in 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:27.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:27 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] boot 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: osdmap e64: 10 total, 10 up, 10 in 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:27.814 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:27.814 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:27 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: osd.9 [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] boot 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: osdmap e64: 10 total, 10 up, 10 in 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:27.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:27.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:27.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:27 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:28.747 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:28 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi060 2024-07-24T21:41:28.747 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:28 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 4273M 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- 172.21.15.86:0/258970829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 msgr2=0x7f0dac1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 --2- 172.21.15.86:0/258970829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1074a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009900 tx=0x7f0da002a3f0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- 172.21.15.86:0/258970829 shutdown_connections 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 --2- 172.21.15.86:0/258970829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac1079e0 0x7f0dac10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 --2- 172.21.15.86:0/258970829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1074a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 --2- 172.21.15.86:0/258970829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac101650 0x7f0dac101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- 172.21.15.86:0/258970829 >> 172.21.15.86:0/258970829 conn(0x7f0dac0fd0c0 msgr2=0x7f0dac0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:41:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- 172.21.15.86:0/258970829 shutdown_connections 2024-07-24T21:41:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- 172.21.15.86:0/258970829 wait complete. 2024-07-24T21:41:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 Processor -- start 2024-07-24T21:41:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.934+0000 7f0db24a3640 1 -- start start 2024-07-24T21:41:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 0x7f0dac198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac1079e0 0x7f0dac19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1115e0 con 0x7f0dac101650 2024-07-24T21:41:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac111460 con 0x7f0dac107060 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db24a3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac111760 con 0x7f0dac1079e0 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38744/0 (socket says 172.21.15.86:38744) 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 -- 172.21.15.86:0/4096611198 learned_addr learned my addr 172.21.15.86:0/4096611198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dabfff640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 0x7f0dac198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0db0a40640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac1079e0 0x7f0dac19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 -- 172.21.15.86:0/4096611198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac1079e0 msgr2=0x7f0dac19e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac1079e0 0x7f0dac19e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 -- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 msgr2=0x7f0dac198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 0x7f0dac198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 -- 172.21.15.86:0/4096611198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19ec50 con 0x7f0dac107060 2024-07-24T21:41:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dabfff640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 0x7f0dac198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0dab7fe640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006010 tx=0x7f0da00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.935+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da003f030 con 0x7f0dac107060 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.936+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da0003800 con 0x7f0dac107060 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.936+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19eee0 con 0x7f0dac107060 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.936+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0035070 con 0x7f0dac107060 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.936+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a5600 con 0x7f0dac107060 2024-07-24T21:41:28.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.937+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac101a30 con 0x7f0dac107060 2024-07-24T21:41:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.938+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0da0003cb0 con 0x7f0dac107060 2024-07-24T21:41:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.938+0000 7f0da97fa640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 0x7f0d940b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.939+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6476+0+0 (secure 0 0 0) 0x7f0da0104170 con 0x7f0dac107060 2024-07-24T21:41:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.939+0000 7f0dabfff640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 0x7f0d940b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.939+0000 7f0dabfff640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 0x7f0d940b60e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00abe0 tx=0x7f0d9c00a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:28.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:28.941+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0da00cbef0 con 0x7f0dac107060 2024-07-24T21:41:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:28 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi060 2024-07-24T21:41:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:28 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 4273M 2024-07-24T21:41:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:28 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi060 2024-07-24T21:41:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:28 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 4273M 2024-07-24T21:41:29.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:29.136+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0dac0ffc90 con 0x7f0d940b3c20 2024-07-24T21:41:29.881 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:29 smithi086 ceph-mon[37128]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:29.881 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:29 smithi086 ceph-mon[37128]: osdmap e65: 10 total, 10 up, 10 in 2024-07-24T21:41:29.881 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:29.881 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:29.881 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:29 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:29 smithi123 ceph-mon[37143]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:29 smithi123 ceph-mon[37143]: osdmap e65: 10 total, 10 up, 10 in 2024-07-24T21:41:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:29 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:29 smithi060 ceph-mon[33000]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:29 smithi060 ceph-mon[33000]: osdmap e65: 10 total, 10 up, 10 in 2024-07-24T21:41:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:30.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:30.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:29 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:30.747 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:30 smithi086 ceph-mon[37128]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:30.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:30 smithi123 ceph-mon[37143]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:30 smithi060 ceph-mon[33000]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:31.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:31.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/517049458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:31.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]': finished 2024-07-24T21:41:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: osdmap e66: 11 total, 10 up, 11 in 2024-07-24T21:41:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:32.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:32.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/517049458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:32.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:32.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]': finished 2024-07-24T21:41:32.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: osdmap e66: 11 total, 10 up, 11 in 2024-07-24T21:41:32.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/517049458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]: dispatch 2024-07-24T21:41:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce9b27b5-9ff8-4004-b2b7-fa97986a66cd"}]': finished 2024-07-24T21:41:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: osdmap e66: 11 total, 10 up, 11 in 2024-07-24T21:41:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:32.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:32 smithi123 ceph-mon[37143]: from='client.? 172.21.15.86:0/2409942417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:33.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:32 smithi086 ceph-mon[37128]: from='client.? 172.21.15.86:0/2409942417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:33.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:32 smithi060 ceph-mon[33000]: from='client.? 172.21.15.86:0/2409942417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:33.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:33 smithi123 ceph-mon[37143]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:34.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:33 smithi086 ceph-mon[37128]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:34.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:33 smithi060 ceph-mon[33000]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:34.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:34 smithi123 ceph-mon[37143]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:35.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:34 smithi086 ceph-mon[37128]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:35.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:34 smithi060 ceph-mon[33000]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:37.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:37 smithi086 ceph-mon[37128]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:37.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:37 smithi060 ceph-mon[33000]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:37.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:37 smithi123 ceph-mon[37143]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:39.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:39 smithi060 ceph-mon[33000]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:39.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:39 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-24T21:41:39.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:39 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:39.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:39 smithi060 ceph-mon[33000]: Deploying daemon osd.10 on smithi086 2024-07-24T21:41:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:39 smithi123 ceph-mon[37143]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:39 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-24T21:41:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:39 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:39.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:39 smithi123 ceph-mon[37143]: Deploying daemon osd.10 on smithi086 2024-07-24T21:41:39.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:39 smithi086 ceph-mon[37128]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:39 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-24T21:41:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:39 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:39 smithi086 ceph-mon[37128]: Deploying daemon osd.10 on smithi086 2024-07-24T21:41:41.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:41 smithi086 ceph-mon[37128]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:41.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:41 smithi123 ceph-mon[37143]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:41.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:41 smithi060 ceph-mon[33000]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:43.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:43 smithi086 ceph-mon[37128]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:43 smithi123 ceph-mon[37143]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:43.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:43 smithi060 ceph-mon[33000]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:44.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:44 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:44.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:44 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:44.221 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:44 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:44 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:44 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:44 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:44.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:44 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:44.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:44 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:44.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:44 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:45.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:45 smithi060 ceph-mon[33000]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:45.885 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:45 smithi086 ceph-mon[37128]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:45.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:45 smithi123 ceph-mon[37143]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:46.940 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 10 on host 'smithi086' 2024-07-24T21:41:46.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.938+0000 7f0da97fa640 1 -- 172.21.15.86:0/4096611198 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0dac0ffc90 con 0x7f0d940b3c20 2024-07-24T21:41:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 msgr2=0x7f0d940b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 0x7f0d940b60e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00abe0 tx=0x7f0d9c00a5c0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 msgr2=0x7f0dac1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006010 tx=0x7f0da00042f0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 shutdown_connections 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d940b3c20 0x7f0d940b60e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dac1079e0 0x7f0dac19e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dac107060 0x7f0dac1990e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 --2- 172.21.15.86:0/4096611198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dac101650 0x7f0dac198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 >> 172.21.15.86:0/4096611198 conn(0x7f0dac0fd0c0 msgr2=0x7f0dac06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 shutdown_connections 2024-07-24T21:41:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-07-24T21:41:46.941+0000 7f0db24a3640 1 -- 172.21.15.86:0/4096611198 wait complete. 2024-07-24T21:41:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:47.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:47.612 DEBUG:teuthology.orchestra.run.smithi086:osd.10> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.10.service 2024-07-24T21:41:47.615 INFO:tasks.cephadm:Deploying osd.11 on smithi123 with /dev/vg_nvme/lv_1... 2024-07-24T21:41:47.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-24T21:41:47.874 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:41:48.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:48 smithi123 ceph-mon[37143]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:48.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:48 smithi123 ceph-mon[37143]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:48.537 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:48 smithi086 ceph-mon[37128]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:48.537 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:48 smithi086 ceph-mon[37128]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:48.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:48 smithi060 ceph-mon[33000]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:48.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:48 smithi060 ceph-mon[33000]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-24T21:41:49.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:49.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-24T21:41:49.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: osdmap e67: 11 total, 10 up, 11 in 2024-07-24T21:41:49.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:49.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:49 smithi123 ceph-mon[37143]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-24T21:41:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: osdmap e67: 11 total, 10 up, 11 in 2024-07-24T21:41:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-mon[37128]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T21:41:49.219+0000 7f3dab0dc640 -1 osd.10 0 waiting for initial osdmap 2024-07-24T21:41:49.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 21:41:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T21:41:49.232+0000 7f3da6f03640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: osdmap e67: 11 total, 10 up, 11 in 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: from='osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:49 smithi060 ceph-mon[33000]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-07-24T21:41:49.637 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-24T21:41:49.727 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-24T21:41:49.740 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-07-24T21:41:49.741 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-07-24T21:41:49.741 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114726 s, 914 MB/s 2024-07-24T21:41:49.741 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-07-24T21:41:50.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:50 smithi086 ceph-mon[37128]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:41:50.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:50 smithi086 ceph-mon[37128]: osdmap e68: 11 total, 10 up, 11 in 2024-07-24T21:41:50.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.327 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:50 smithi123 ceph-mon[37143]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:41:50.327 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:50 smithi123 ceph-mon[37143]: osdmap e68: 11 total, 10 up, 11 in 2024-07-24T21:41:50.328 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.328 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-07-24T21:41:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:50 smithi060 ceph-mon[33000]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-07-24T21:41:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:50 smithi060 ceph-mon[33000]: osdmap e68: 11 total, 10 up, 11 in 2024-07-24T21:41:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:50.620 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:41:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:41:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:41:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] boot 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: osdmap e69: 11 total, 11 up, 11 in 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi086 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 4272M 2024-07-24T21:41:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:51.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:51.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] boot 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: osdmap e69: 11 total, 11 up, 11 in 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi086 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 4272M 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 894 GiB / 894 GiB avail 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: osd.10 [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] boot 2024-07-24T21:41:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: osdmap e69: 11 total, 11 up, 11 in 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi086 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 4272M 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:41:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:41:52.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 -- 172.21.15.123:0/3843733028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54081036a0 msgr2=0x7f5408103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 --2- 172.21.15.123:0/3843733028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54081036a0 0x7f5408103a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5404009900 tx=0x7f540402a410 comp rx=0 tx=0).stop 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 -- 172.21.15.123:0/3843733028 shutdown_connections 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 --2- 172.21.15.123:0/3843733028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540810cb00 0x7f540810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 --2- 172.21.15.123:0/3843733028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 0x7f540810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 --2- 172.21.15.123:0/3843733028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54081036a0 0x7f5408103a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 -- 172.21.15.123:0/3843733028 >> 172.21.15.123:0/3843733028 conn(0x7f54080fd100 msgr2=0x7f54080ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 -- 172.21.15.123:0/3843733028 shutdown_connections 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.131+0000 7f54106e5640 1 -- 172.21.15.123:0/3843733028 wait complete. 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 Processor -- start 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 -- start start 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54081036a0 0x7f5408198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 0x7f5408199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:52.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5408112740 con 0x7f5408103fc0 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54081125c0 con 0x7f54081036a0 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f54106e5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54081128c0 con 0x7f540810cb00 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39218/0 (socket says 172.21.15.123:39218) 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 -- 172.21.15.123:0/3740856668 learned_addr learned my addr 172.21.15.123:0/3740856668 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540e481640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54081036a0 0x7f5408198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540dc80640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 0x7f5408199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 -- 172.21.15.123:0/3740856668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54081036a0 msgr2=0x7f5408198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54081036a0 0x7f5408198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 -- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 msgr2=0x7f5408199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:41:52.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 0x7f5408199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.132+0000 7f540ec82640 1 -- 172.21.15.123:0/3740856668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540819ed80 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f540ec82640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f540000cad0 tx=0x7f540000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540000ae20 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5400005020 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540000f6b0 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540819f070 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.133+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081a5730 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.134+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5408103a80 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.135+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f54000051d0 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.136+0000 7f53ff7fe640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 0x7f53f40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.136+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6909+0+0 (secure 0 0 0) 0x7f54000e3a70 con 0x7f540810cb00 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.136+0000 7f540e481640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 0x7f53f40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:41:52.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.137+0000 7f540e481640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 0x7f53f40b60e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5404031440 tx=0x7f5404005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:41:52.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.139+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5400014030 con 0x7f540810cb00 2024-07-24T21:41:52.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:41:52.338+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f54080ff420 con 0x7f53f40b3c20 2024-07-24T21:41:52.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:52 smithi123 ceph-mon[37143]: osdmap e70: 11 total, 11 up, 11 in 2024-07-24T21:41:52.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:52 smithi086 ceph-mon[37128]: osdmap e70: 11 total, 11 up, 11 in 2024-07-24T21:41:52.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:52 smithi060 ceph-mon[33000]: osdmap e70: 11 total, 11 up, 11 in 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: osdmap e71: 11 total, 11 up, 11 in 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:53.397 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:53.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: osdmap e71: 11 total, 11 up, 11 in 2024-07-24T21:41:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: osdmap e71: 11 total, 11 up, 11 in 2024-07-24T21:41:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:41:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-24T21:41:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-24T21:41:53.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:41:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/786550169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]': finished 2024-07-24T21:41:55.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: osdmap e72: 12 total, 11 up, 12 in 2024-07-24T21:41:55.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:55 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:41:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:55.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/786550169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]': finished 2024-07-24T21:41:55.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: osdmap e72: 12 total, 11 up, 12 in 2024-07-24T21:41:55.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:55 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:41:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/786550169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]: dispatch 2024-07-24T21:41:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "384e3184-0630-4bce-bb27-ca026eb0597d"}]': finished 2024-07-24T21:41:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: osdmap e72: 12 total, 11 up, 12 in 2024-07-24T21:41:55.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:55 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:41:56.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:56 smithi086 ceph-mon[37128]: from='client.? 172.21.15.123:0/2112914698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:56.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.123:0/2112914698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:56.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:56 smithi123 ceph-mon[37143]: from='client.? 172.21.15.123:0/2112914698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-24T21:41:57.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:57 smithi086 ceph-mon[37128]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:57.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:57 smithi060 ceph-mon[33000]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:57 smithi123 ceph-mon[37143]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 360 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:41:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:41:59 smithi123 ceph-mon[37143]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-07-24T21:41:59.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:41:59 smithi086 ceph-mon[37128]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-07-24T21:41:59.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:41:59 smithi060 ceph-mon[33000]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-07-24T21:42:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:01 smithi086 ceph-mon[37128]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:42:01.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:01 smithi060 ceph-mon[33000]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:42:01.635 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:01 smithi123 ceph-mon[37143]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-24T21:42:02.440 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-24T21:42:02.441 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:02.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-24T21:42:02.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:02.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-24T21:42:02.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:03.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:03 smithi123 ceph-mon[37143]: Deploying daemon osd.11 on smithi123 2024-07-24T21:42:03.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:03 smithi123 ceph-mon[37143]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-07-24T21:42:03.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:03 smithi086 ceph-mon[37128]: Deploying daemon osd.11 on smithi123 2024-07-24T21:42:03.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:03 smithi086 ceph-mon[37128]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-07-24T21:42:03.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:03 smithi060 ceph-mon[33000]: Deploying daemon osd.11 on smithi123 2024-07-24T21:42:03.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:03 smithi060 ceph-mon[33000]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-07-24T21:42:05.525 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:05 smithi123 ceph-mon[37143]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:42:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:05 smithi086 ceph-mon[37128]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:42:05.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:05 smithi060 ceph-mon[33000]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-24T21:42:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:07 smithi123 ceph-mon[37143]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:42:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:07 smithi086 ceph-mon[37128]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:42:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:07 smithi060 ceph-mon[33000]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-24T21:42:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:09.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:09 smithi086 ceph-mon[37128]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:42:09.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:09 smithi060 ceph-mon[33000]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:42:09.611 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:09 smithi123 ceph-mon[37143]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-24T21:42:09.996 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 11 on host 'smithi123' 2024-07-24T21:42:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.995+0000 7f53ff7fe640 1 -- 172.21.15.123:0/3740856668 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f54080ff420 con 0x7f53f40b3c20 2024-07-24T21:42:09.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 msgr2=0x7f53f40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 0x7f53f40b60e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5404031440 tx=0x7f5404005b50 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 msgr2=0x7f540819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f540000cad0 tx=0x7f540000cf90 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 shutdown_connections 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f40b3c20 0x7f53f40b60e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540810cb00 0x7f540819e5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5408103fc0 0x7f5408199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 --2- 172.21.15.123:0/3740856668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54081036a0 0x7f5408198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 >> 172.21.15.123:0/3740856668 conn(0x7f54080fd100 msgr2=0x7f54080fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 shutdown_connections 2024-07-24T21:42:09.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:09.997+0000 7f54106e5640 1 -- 172.21.15.123:0/3740856668 wait complete. 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.556 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.556 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.556 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:10.556 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:10.556 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:10.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:10.601 DEBUG:teuthology.orchestra.run.smithi123:osd.11> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.11.service 2024-07-24T21:42:10.603 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-24T21:42:10.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd stat -f json 2024-07-24T21:42:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:11 smithi123 ceph-mon[37143]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:11.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:11 smithi123 ceph-mon[37143]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:11.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:11 smithi123 ceph-mon[37143]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:11 smithi086 ceph-mon[37128]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:11 smithi086 ceph-mon[37128]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:11 smithi086 ceph-mon[37128]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:11 smithi060 ceph-mon[33000]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:11 smithi060 ceph-mon[33000]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:11 smithi060 ceph-mon[33000]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-24T21:42:12.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-mon[37143]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-24T21:42:12.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-mon[37143]: osdmap e73: 12 total, 11 up, 12 in 2024-07-24T21:42:12.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:12.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-mon[37143]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-mon[37143]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.315 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T21:42:12.293+0000 7fca3f410640 -1 osd.11 0 waiting for initial osdmap 2024-07-24T21:42:12.315 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 21:42:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T21:42:12.312+0000 7fca3b237640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-24T21:42:12.414 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:12 smithi060 ceph-mon[33000]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-24T21:42:12.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:12 smithi060 ceph-mon[33000]: osdmap e73: 12 total, 11 up, 12 in 2024-07-24T21:42:12.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:12.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:12 smithi060 ceph-mon[33000]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:12 smithi060 ceph-mon[33000]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:12 smithi086 ceph-mon[37128]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-24T21:42:12.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:12 smithi086 ceph-mon[37128]: osdmap e73: 12 total, 11 up, 12 in 2024-07-24T21:42:12.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:12.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:12 smithi086 ceph-mon[37128]: from='osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:12 smithi086 ceph-mon[37128]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-07-24T21:42:12.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/1100555297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8101650 msgr2=0x7f8fb8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/1100555297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8101650 0x7f8fb8101a30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009900 tx=0x7f8fb402a410 comp rx=0 tx=0).stop 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/1100555297 shutdown_connections 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/1100555297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 0x7f8fb810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/1100555297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8107060 0x7f8fb81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/1100555297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8101650 0x7f8fb8101a30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/1100555297 >> 172.21.15.60:0/1100555297 conn(0x7f8fb80fd0c0 msgr2=0x7f8fb80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/1100555297 shutdown_connections 2024-07-24T21:42:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.875+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/1100555297 wait complete. 2024-07-24T21:42:12.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 Processor -- start 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 -- start start 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 0x7f8fb8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 0x7f8fb819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8111610 con 0x7f8fb8107060 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8111490 con 0x7f8fb81079e0 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbfdc8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8111790 con 0x7f8fb8101650 2024-07-24T21:42:12.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.876+0000 7f8fbd363640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46638/0 (socket says 172.21.15.60:46638) 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbdb64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 0x7f8fb8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 -- 172.21.15.60:0/4209639828 learned_addr learned my addr 172.21.15.60:0/4209639828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbe365640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 0x7f8fb819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 -- 172.21.15.60:0/4209639828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 msgr2=0x7f8fb8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 0x7f8fb8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 -- 172.21.15.60:0/4209639828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 msgr2=0x7f8fb819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 0x7f8fb819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 -- 172.21.15.60:0/4209639828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819ed00 con 0x7f8fb8107060 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbdb64640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 0x7f8fb8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbd363640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9f0 tx=0x7f8fa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800cd80 con 0x7f8fb8107060 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb819eff0 con 0x7f8fb8107060 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa80054d0 con 0x7f8fb8107060 2024-07-24T21:42:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8010690 con 0x7f8fb8107060 2024-07-24T21:42:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.877+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a56b0 con 0x7f8fb8107060 2024-07-24T21:42:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.879+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8101a30 con 0x7f8fb8107060 2024-07-24T21:42:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.880+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f8fa8004e30 con 0x7f8fb8107060 2024-07-24T21:42:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.881+0000 7f8faeffd640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 0x7f8fa40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.881+0000 7f8fbdb64640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 0x7f8fa40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.881+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7062+0+0 (secure 0 0 0) 0x7f8fa80e3db0 con 0x7f8fb8107060 2024-07-24T21:42:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.881+0000 7f8fbdb64640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 0x7f8fa40b60e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4031440 tx=0x7f8fb4005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:12.884+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8fa80555e0 con 0x7f8fb8107060 2024-07-24T21:42:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.078+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8fb8199d80 con 0x7f8fb8107060 2024-07-24T21:42:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.079+0000 7f8faeffd640 1 -- 172.21.15.60:0/4209639828 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f8fb8199d80 con 0x7f8fb8107060 2024-07-24T21:42:13.080 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 msgr2=0x7f8fa40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 0x7f8fa40b60e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4031440 tx=0x7f8fb4005b50 comp rx=0 tx=0).stop 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 msgr2=0x7f8fb8199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9f0 tx=0x7f8fa800edf0 comp rx=0 tx=0).stop 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 shutdown_connections 2024-07-24T21:42:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8fa40b3c20 0x7f8fa40b60e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81079e0 0x7f8fb819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fb8107060 0x7f8fb8199190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 --2- 172.21.15.60:0/4209639828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fb8101650 0x7f8fb8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 >> 172.21.15.60:0/4209639828 conn(0x7f8fb80fd0c0 msgr2=0x7f8fb806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 shutdown_connections 2024-07-24T21:42:13.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:13.081+0000 7f8fbfdc8640 1 -- 172.21.15.60:0/4209639828 wait complete. 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: osdmap e74: 12 total, 11 up, 12 in 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4209639828' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:42:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: osdmap e74: 12 total, 11 up, 12 in 2024-07-24T21:42:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4209639828' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:13.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-07-24T21:42:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: osdmap e74: 12 total, 11 up, 12 in 2024-07-24T21:42:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4209639828' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:13.852 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1721857310,"num_in_osds":12,"osd_in_since":1721857314,"num_remapped_pgs":0} 2024-07-24T21:42:14.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: purged_snaps scrub starts 2024-07-24T21:42:14.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: purged_snaps scrub ok 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] boot 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: osdmap e75: 12 total, 12 up, 12 in 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: Detected new or changed devices on smithi123 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 4272M 2024-07-24T21:42:14.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:14.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:14.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: purged_snaps scrub starts 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: purged_snaps scrub ok 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] boot 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: osdmap e75: 12 total, 12 up, 12 in 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: Detected new or changed devices on smithi123 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 4272M 2024-07-24T21:42:14.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:14.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:14.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: purged_snaps scrub starts 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: purged_snaps scrub ok 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: osd.11 [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] boot 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: osdmap e75: 12 total, 12 up, 12 in 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: Detected new or changed devices on smithi123 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 4272M 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:14.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:14.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd stat -f json 2024-07-24T21:42:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:15.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:15 smithi123 ceph-mon[37143]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:15.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:15 smithi123 ceph-mon[37143]: osdmap e76: 12 total, 12 up, 12 in 2024-07-24T21:42:15.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:15 smithi060 ceph-mon[33000]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:15.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:15 smithi060 ceph-mon[33000]: osdmap e76: 12 total, 12 up, 12 in 2024-07-24T21:42:15.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:15 smithi086 ceph-mon[37128]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 983 GiB / 983 GiB avail 2024-07-24T21:42:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:15 smithi086 ceph-mon[37128]: osdmap e76: 12 total, 12 up, 12 in 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 -- 172.21.15.60:0/1421958592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8102790 msgr2=0x7fccb8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 --2- 172.21.15.60:0/1421958592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8102790 0x7fccb8102b70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009900 tx=0x7fcca802a410 comp rx=0 tx=0).stop 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 -- 172.21.15.60:0/1421958592 shutdown_connections 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 --2- 172.21.15.60:0/1421958592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb8108b20 0x7fccb810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 --2- 172.21.15.60:0/1421958592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb81081a0 0x7fccb81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 --2- 172.21.15.60:0/1421958592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8102790 0x7fccb8102b70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 -- 172.21.15.60:0/1421958592 >> 172.21.15.60:0/1421958592 conn(0x7fccb80fe200 msgr2=0x7fccb8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 -- 172.21.15.60:0/1421958592 shutdown_connections 2024-07-24T21:42:16.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.680+0000 7fccbe99e640 1 -- 172.21.15.60:0/1421958592 wait complete. 2024-07-24T21:42:16.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 Processor -- start 2024-07-24T21:42:16.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 -- start start 2024-07-24T21:42:16.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:16.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb81081a0 0x7fccb81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:16.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8112720 con 0x7fccb8108b20 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81125a0 con 0x7fccb8102790 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbe99e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81128a0 con 0x7fccb81081a0 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbcf3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49540/0 (socket says 172.21.15.60:49540) 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccbcf3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58220/0 (socket says 172.21.15.60:58220) 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.681+0000 7fccb7fff640 1 -- 172.21.15.60:0/3221445756 learned_addr learned my addr 172.21.15.60:0/3221445756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccb77fe640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb81081a0 0x7fccb81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 -- 172.21.15.60:0/3221445756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb81081a0 msgr2=0x7fccb81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb81081a0 0x7fccb81990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 -- 172.21.15.60:0/3221445756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 msgr2=0x7fccb8198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 -- 172.21.15.60:0/3221445756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb819ec60 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccb7fff640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbcf3b640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e940 tx=0x7fccb000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0004d20 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb819ef50 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb0005020 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00116d0 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.682+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81a5610 con 0x7fccb8108b20 2024-07-24T21:42:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.684+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb8102b70 con 0x7fccb8108b20 2024-07-24T21:42:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.685+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fccb0005460 con 0x7fccb8108b20 2024-07-24T21:42:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.685+0000 7fccb57fa640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 0x7fcca00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.686+0000 7fccb7fff640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 0x7fcca00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.686+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7fccb00e39b0 con 0x7fccb8108b20 2024-07-24T21:42:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.686+0000 7fccb7fff640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 0x7fcca00b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcca8031440 tx=0x7fcca80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:16.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.690+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fccb0016030 con 0x7fccb8108b20 2024-07-24T21:42:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.884+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fccb81a59b0 con 0x7fccb8108b20 2024-07-24T21:42:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.884+0000 7fccb57fa640 1 -- 172.21.15.60:0/3221445756 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fccb00ab3d0 con 0x7fccb8108b20 2024-07-24T21:42:16.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 msgr2=0x7fcca00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 0x7fcca00b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcca8031440 tx=0x7fcca80058d0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 msgr2=0x7fccb819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e940 tx=0x7fccb000ee00 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 shutdown_connections 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcca00b3c20 0x7fcca00b60e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb8108b20 0x7fccb819e490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb81081a0 0x7fccb81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 --2- 172.21.15.60:0/3221445756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccb8102790 0x7fccb8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.886+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 >> 172.21.15.60:0/3221445756 conn(0x7fccb80fe200 msgr2=0x7fccb80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.887+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 shutdown_connections 2024-07-24T21:42:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:16.887+0000 7fccbe99e640 1 -- 172.21.15.60:0/3221445756 wait complete. 2024-07-24T21:42:17.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:17 smithi060 ceph-mon[33000]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:17.564 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:17 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3221445756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:17.593 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1721857333,"num_in_osds":12,"osd_in_since":1721857314,"num_remapped_pgs":0} 2024-07-24T21:42:17.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:42:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:17 smithi123 ceph-mon[37143]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:17 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3221445756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:17 smithi086 ceph-mon[37128]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3221445756' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-24T21:42:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 -- 172.21.15.60:0/3133442502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f01036a0 msgr2=0x7f64f0103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/3133442502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f01036a0 0x7f64f0103a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009900 tx=0x7f64e002a410 comp rx=0 tx=0).stop 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 -- 172.21.15.60:0/3133442502 shutdown_connections 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/3133442502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 0x7f64f010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/3133442502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f0103fc0 0x7f64f010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/3133442502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f01036a0 0x7f64f0103a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 -- 172.21.15.60:0/3133442502 >> 172.21.15.60:0/3133442502 conn(0x7f64f00fd100 msgr2=0x7f64f00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 -- 172.21.15.60:0/3133442502 shutdown_connections 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.421+0000 7f64f5ae6640 1 -- 172.21.15.60:0/3133442502 wait complete. 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 Processor -- start 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 -- start start 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f01036a0 0x7f64f019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 0x7f64f01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01127d0 con 0x7f64f0103fc0 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0112650 con 0x7f64f01036a0 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64f5ae6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0112950 con 0x7f64f010cb00 2024-07-24T21:42:19.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58242/0 (socket says 172.21.15.60:58242) 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64eeffd640 1 -- 172.21.15.60:0/2081429668 learned_addr learned my addr 172.21.15.60:0/2081429668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64effff640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 0x7f64f01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.422+0000 7f64ef7fe640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f01036a0 0x7f64f019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 -- 172.21.15.60:0/2081429668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 msgr2=0x7f64f01a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 0x7f64f01a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 -- 172.21.15.60:0/2081429668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f01036a0 msgr2=0x7f64f019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f01036a0 0x7f64f019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 -- 172.21.15.60:0/2081429668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f01a1070 con 0x7f64f0103fc0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64eeffd640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b3c0 tx=0x7f64e400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011830 con 0x7f64f0103fc0 2024-07-24T21:42:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4016070 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f01a1360 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4010de0 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.423+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01a7a20 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.424+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f0103a80 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.425+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f64e40119d0 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.426+0000 7f64ecff9640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 0x7f64d80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.426+0000 7f64ef7fe640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 0x7f64d80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.426+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f64e40e3570 con 0x7f64f0103fc0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.427+0000 7f64ef7fe640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 0x7f64d80b60e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64e0031440 tx=0x7f64e0005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.429+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f64e40aaef0 con 0x7f64f0103fc0 2024-07-24T21:42:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:19 smithi060 ceph-mon[33000]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.623+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64f01a7d70 con 0x7f64f0103fc0 2024-07-24T21:42:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.624+0000 7f64ecff9640 1 -- 172.21.15.60:0/2081429668 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18717 (secure 0 0 0) 0x7f64e40b31c0 con 0x7f64f0103fc0 2024-07-24T21:42:19.625 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:19.626 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":76,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:42:14.331804+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:42:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 msgr2=0x7f64d80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 0x7f64d80b60e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64e0031440 tx=0x7f64e0005e80 comp rx=0 tx=0).stop 2024-07-24T21:42:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 msgr2=0x7f64f019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b3c0 tx=0x7f64e400b880 comp rx=0 tx=0).stop 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 shutdown_connections 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64d80b3c20 0x7f64d80b60e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f010cb00 0x7f64f01a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f0103fc0 0x7f64f019b500 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 --2- 172.21.15.60:0/2081429668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f01036a0 0x7f64f019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.626+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 >> 172.21.15.60:0/2081429668 conn(0x7f64f00fd100 msgr2=0x7f64f010ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.627+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 shutdown_connections 2024-07-24T21:42:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:19.627+0000 7f64f5ae6640 1 -- 172.21.15.60:0/2081429668 wait complete. 2024-07-24T21:42:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:19 smithi123 ceph-mon[37143]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:19 smithi086 ceph-mon[37128]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:20.310 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-24T21:38:46.032062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-24T21:42:20.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-24T21:42:20.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2081429668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:42:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:20.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2081429668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:42:20.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2081429668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:42:21.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:21 smithi060 ceph-mon[33000]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:21 smithi123 ceph-mon[37143]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:21.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:21 smithi086 ceph-mon[37128]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 -- 172.21.15.60:0/2066772247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa01036a0 msgr2=0x7f5fa0103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 --2- 172.21.15.60:0/2066772247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa01036a0 0x7f5fa0103a80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009900 tx=0x7f5f9002a410 comp rx=0 tx=0).stop 2024-07-24T21:42:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 -- 172.21.15.60:0/2066772247 shutdown_connections 2024-07-24T21:42:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 --2- 172.21.15.60:0/2066772247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 0x7f5fa010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 --2- 172.21.15.60:0/2066772247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa0103fc0 0x7f5fa010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 --2- 172.21.15.60:0/2066772247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa01036a0 0x7f5fa0103a80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.260+0000 7f5fa5647640 1 -- 172.21.15.60:0/2066772247 >> 172.21.15.60:0/2066772247 conn(0x7f5fa00fd100 msgr2=0x7f5fa00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.261+0000 7f5fa5647640 1 -- 172.21.15.60:0/2066772247 shutdown_connections 2024-07-24T21:42:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.261+0000 7f5fa5647640 1 -- 172.21.15.60:0/2066772247 wait complete. 2024-07-24T21:42:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.261+0000 7f5fa5647640 1 Processor -- start 2024-07-24T21:42:22.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 -- start start 2024-07-24T21:42:22.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa01036a0 0x7f5fa0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:22.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 0x7f5fa019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01127b0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0112630 con 0x7f5fa010cb00 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5fa5647640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0112930 con 0x7f5fa01036a0 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58264/0 (socket says 172.21.15.60:58264) 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 -- 172.21.15.60:0/42333957 learned_addr learned my addr 172.21.15.60:0/42333957 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9effd640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa01036a0 0x7f5fa0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9f7fe640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 0x7f5fa019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 -- 172.21.15.60:0/42333957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa01036a0 msgr2=0x7f5fa0198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa01036a0 0x7f5fa0198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 -- 172.21.15.60:0/42333957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 msgr2=0x7f5fa019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 0x7f5fa019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 -- 172.21.15.60:0/42333957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa019edd0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.262+0000 7f5f9e7fc640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e9c0 tx=0x7f5f9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.263+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400ccc0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.263+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f940035f0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.263+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa019f0c0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.263+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400ce60 con 0x7f5fa0103fc0 2024-07-24T21:42:22.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.263+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa01a5780 con 0x7f5fa0103fc0 2024-07-24T21:42:22.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.264+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa006b550 con 0x7f5fa0103fc0 2024-07-24T21:42:22.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.265+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f5f94004000 con 0x7f5fa0103fc0 2024-07-24T21:42:22.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.266+0000 7f5f87fff640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 0x7f5f880b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:22.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.266+0000 7f5f9effd640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 0x7f5f880b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:22.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.266+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f5f940e3aa0 con 0x7f5fa0103fc0 2024-07-24T21:42:22.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.267+0000 7f5f9effd640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 0x7f5f880b6030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:22.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.270+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5f940ab460 con 0x7f5fa0103fc0 2024-07-24T21:42:22.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.467+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fa0199e80 con 0x7f5fa0103fc0 2024-07-24T21:42:22.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.468+0000 7f5f87fff640 1 -- 172.21.15.60:0/42333957 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f5fa0199e80 con 0x7f5fa0103fc0 2024-07-24T21:42:22.469 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-07-24T21:42:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 msgr2=0x7f5f880b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 0x7f5f880b6030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90005a20 comp rx=0 tx=0).stop 2024-07-24T21:42:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 msgr2=0x7f5fa0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e9c0 tx=0x7f5f9400ee80 comp rx=0 tx=0).stop 2024-07-24T21:42:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 shutdown_connections 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f880b3b70 0x7f5f880b6030 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fa010cb00 0x7f5fa019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa0103fc0 0x7f5fa0199260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 --2- 172.21.15.60:0/42333957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa01036a0 0x7f5fa0198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 >> 172.21.15.60:0/42333957 conn(0x7f5fa00fd100 msgr2=0x7f5fa00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 shutdown_connections 2024-07-24T21:42:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:22.470+0000 7f5fa5647640 1 -- 172.21.15.60:0/42333957 wait complete. 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.a on smithi060 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.d on smithi060 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.g on smithi060 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.j on smithi060 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.b on smithi086 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.e on smithi086 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.h on smithi086 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.k on smithi086 2024-07-24T21:42:23.212 INFO:tasks.cephadm:Adding mds.c on smithi123 2024-07-24T21:42:23.213 INFO:tasks.cephadm:Adding mds.f on smithi123 2024-07-24T21:42:23.213 INFO:tasks.cephadm:Adding mds.i on smithi123 2024-07-24T21:42:23.213 INFO:tasks.cephadm:Adding mds.l on smithi123 2024-07-24T21:42:23.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph orch apply mds all '12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l' 2024-07-24T21:42:23.476 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T21:42:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:23 smithi123 ceph-mon[37143]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:23 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/42333957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:42:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:23 smithi060 ceph-mon[33000]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:23 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/42333957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:42:23.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:23 smithi086 ceph-mon[37128]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:23.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:23 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/42333957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:42:24.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 -- 172.21.15.123:0/4023459552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc101690 msgr2=0x7fc6cc101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 --2- 172.21.15.123:0/4023459552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc101690 0x7fc6cc101a70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc02a410 comp rx=0 tx=0).stop 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 -- 172.21.15.123:0/4023459552 shutdown_connections 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 --2- 172.21.15.123:0/4023459552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc107a20 0x7fc6cc10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 --2- 172.21.15.123:0/4023459552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 0x7fc6cc1074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 --2- 172.21.15.123:0/4023459552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc101690 0x7fc6cc101a70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 -- 172.21.15.123:0/4023459552 >> 172.21.15.123:0/4023459552 conn(0x7fc6cc0fd100 msgr2=0x7fc6cc0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 -- 172.21.15.123:0/4023459552 shutdown_connections 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.934+0000 7fc6d0895640 1 -- 172.21.15.123:0/4023459552 wait complete. 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 Processor -- start 2024-07-24T21:42:24.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 -- start start 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc101690 0x7fc6cc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 0x7fc6cc199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc111650 con 0x7fc6cc101690 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1114d0 con 0x7fc6cc1070a0 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6d0895640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1117d0 con 0x7fc6cc107a20 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54706/0 (socket says 172.21.15.123:54706) 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 -- 172.21.15.123:0/3947948528 learned_addr learned my addr 172.21.15.123:0/3947948528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6ca59c640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc101690 0x7fc6cc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6c3fff640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 0x7fc6cc199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:24.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 -- 172.21.15.123:0/3947948528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 msgr2=0x7fc6cc199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 0x7fc6cc199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.935+0000 7fc6cad9d640 1 -- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc101690 msgr2=0x7fc6cc198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6cad9d640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc101690 0x7fc6cc198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6cad9d640 1 -- 172.21.15.123:0/3947948528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc19ed00 con 0x7fc6cc107a20 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6cad9d640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009f60 tx=0x7fc6c4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400cd80 con 0x7fc6cc107a20 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc19eff0 con 0x7fc6cc107a20 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c4013770 con 0x7fc6cc107a20 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4005380 con 0x7fc6cc107a20 2024-07-24T21:42:24.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.936+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc0fd4e0 con 0x7fc6cc107a20 2024-07-24T21:42:24.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.937+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6cc101a70 con 0x7fc6cc107a20 2024-07-24T21:42:24.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.938+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc6c4013920 con 0x7fc6cc107a20 2024-07-24T21:42:24.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.939+0000 7fc6c37fe640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 0x7fc6b80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:24.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.939+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7fc6c40e7f80 con 0x7fc6cc107a20 2024-07-24T21:42:24.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.939+0000 7fc6ca59c640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 0x7fc6b80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:24.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.939+0000 7fc6ca59c640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 0x7fc6b80b60e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031440 tx=0x7fc6bc005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:24.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:24.942+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc6c40af9a0 con 0x7fc6cc107a20 2024-07-24T21:42:25.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.137+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}) -- 0x7fc6cc1067e0 con 0x7fc6b80b3c20 2024-07-24T21:42:25.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.145+0000 7fc6c37fe640 1 -- 172.21.15.123:0/3947948528 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc6cc1067e0 con 0x7fc6b80b3c20 2024-07-24T21:42:25.147 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mds.all update... 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 msgr2=0x7fc6b80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 0x7fc6b80b60e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031440 tx=0x7fc6bc005b50 comp rx=0 tx=0).stop 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 msgr2=0x7fc6cc19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009f60 tx=0x7fc6c4009f90 comp rx=0 tx=0).stop 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 shutdown_connections 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6b80b3c20 0x7fc6b80b60e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc107a20 0x7fc6cc19e530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:25.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc1070a0 0x7fc6cc199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:25.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 --2- 172.21.15.123:0/3947948528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6cc101690 0x7fc6cc198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:25.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 >> 172.21.15.123:0/3947948528 conn(0x7fc6cc0fd100 msgr2=0x7fc6cc06ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:25.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 shutdown_connections 2024-07-24T21:42:25.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-07-24T21:42:25.148+0000 7fc6d0895640 1 -- 172.21.15.123:0/3947948528 wait complete. 2024-07-24T21:42:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:25 smithi123 ceph-mon[37143]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:25 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:25.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:25 smithi086 ceph-mon[37128]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:25.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:25 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:25.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:25 smithi060 ceph-mon[33000]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:25.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:25.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:25 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:42:25.861 DEBUG:teuthology.orchestra.run.smithi060:mds.a> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.a.service 2024-07-24T21:42:25.864 DEBUG:teuthology.orchestra.run.smithi060:mds.d> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.d.service 2024-07-24T21:42:25.907 DEBUG:teuthology.orchestra.run.smithi060:mds.g> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.g.service 2024-07-24T21:42:25.950 DEBUG:teuthology.orchestra.run.smithi060:mds.j> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.j.service 2024-07-24T21:42:25.993 DEBUG:teuthology.orchestra.run.smithi086:mds.b> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.b.service 2024-07-24T21:42:25.996 DEBUG:teuthology.orchestra.run.smithi086:mds.e> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.e.service 2024-07-24T21:42:26.038 DEBUG:teuthology.orchestra.run.smithi086:mds.h> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.h.service 2024-07-24T21:42:26.081 DEBUG:teuthology.orchestra.run.smithi086:mds.k> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.k.service 2024-07-24T21:42:26.125 DEBUG:teuthology.orchestra.run.smithi123:mds.c> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.c.service 2024-07-24T21:42:26.127 DEBUG:teuthology.orchestra.run.smithi123:mds.f> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service 2024-07-24T21:42:26.169 DEBUG:teuthology.orchestra.run.smithi123:mds.i> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.i.service 2024-07-24T21:42:26.214 DEBUG:teuthology.orchestra.run.smithi123:mds.l> sudo journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service 2024-07-24T21:42:26.258 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-24T21:42:26.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:26.410 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:42:26.410 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-07-24T21:42:26.410 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.410 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:26.410 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:26.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:26.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:26.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:26 smithi123 ceph-mon[37143]: Deploying daemon mds.l on smithi123 2024-07-24T21:42:26.518 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:26.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:42:26.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-07-24T21:42:26.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:26.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:26.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:26.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:26.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:26 smithi060 ceph-mon[33000]: Deploying daemon mds.l on smithi123 2024-07-24T21:42:26.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:26.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:26 smithi086 ceph-mon[37128]: Deploying daemon mds.l on smithi123 2024-07-24T21:42:27.492 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 systemd[1]: Starting Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:27.860 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:27 smithi123 ceph-mon[37143]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:27.860 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 podman[68851]: 2024-07-24 21:42:27.532811865 +0000 UTC m=+0.017053482 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:27.860 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 podman[68851]: 2024-07-24 21:42:27.648965012 +0000 UTC m=+0.133206672 container create 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-86385b1, org.label-schema.build-date=20240716) 2024-07-24T21:42:27.860 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 podman[68851]: 2024-07-24 21:42:27.859761256 +0000 UTC m=+0.344002884 container init 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T21:42:27.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:27 smithi060 ceph-mon[33000]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:28.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:27 smithi086 ceph-mon[37128]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 -- 172.21.15.60:0/1776708327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144101690 msgr2=0x7f7144101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 --2- 172.21.15.60:0/1776708327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144101690 0x7f7144101a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c02a410 comp rx=0 tx=0).stop 2024-07-24T21:42:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 -- 172.21.15.60:0/1776708327 shutdown_connections 2024-07-24T21:42:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 --2- 172.21.15.60:0/1776708327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7144107a20 0x7f714410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 --2- 172.21.15.60:0/1776708327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71441070a0 0x7f71441074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 --2- 172.21.15.60:0/1776708327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144101690 0x7f7144101a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.096+0000 7f714afda640 1 -- 172.21.15.60:0/1776708327 >> 172.21.15.60:0/1776708327 conn(0x7f71440fd100 msgr2=0x7f71440ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.097+0000 7f714afda640 1 -- 172.21.15.60:0/1776708327 shutdown_connections 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.097+0000 7f714afda640 1 -- 172.21.15.60:0/1776708327 wait complete. 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.097+0000 7f714afda640 1 Processor -- start 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.097+0000 7f714afda640 1 -- start start 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7144101690 0x7f7144198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71441070a0 0x7f71441991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111680 con 0x7f7144107a20 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111500 con 0x7f7144101690 2024-07-24T21:42:28.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f714afda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7144111800 con 0x7f71441070a0 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36862/0 (socket says 172.21.15.60:36862) 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 -- 172.21.15.60:0/3393445504 learned_addr learned my addr 172.21.15.60:0/3393445504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7148d76640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7144101690 0x7f7144198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7143fff640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71441070a0 0x7f71441991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 -- 172.21.15.60:0/3393445504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71441070a0 msgr2=0x7f71441991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71441070a0 0x7f71441991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 -- 172.21.15.60:0/3393445504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7144101690 msgr2=0x7f7144198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7144101690 0x7f7144198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 -- 172.21.15.60:0/3393445504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714419ed10 con 0x7f7144107a20 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7149577640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f713c00cad0 tx=0x7f713c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:28.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.098+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00ae20 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.099+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714419f000 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.099+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f713c005020 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.099+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f6b0 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.099+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a56c0 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.100+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7144101a70 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.101+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f713c0051d0 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.102+0000 7f7141ffb640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 0x7f71300b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.102+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f713c0e3770 con 0x7f7144107a20 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.102+0000 7f7148d76640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 0x7f71300b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.102+0000 7f7148d76640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 0x7f71300b60e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f712c005ab0 tx=0x7f712c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.106+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f713c014030 con 0x7f7144107a20 2024-07-24T21:42:28.242 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 podman[68851]: 2024-07-24 21:42:27.862349306 +0000 UTC m=+0.346590932 container start 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0) 2024-07-24T21:42:28.242 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 ceph-mds[68870]: -- 172.21.15.123:0/3593464532 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642e0279860 con 0x5642e0256800 2024-07-24T21:42:28.243 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[68866]: starting mds.l at 2024-07-24T21:42:28.243 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 bash[68851]: 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed 2024-07-24T21:42:28.243 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:42:27 smithi123 systemd[1]: Started Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.342+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f714402e000 con 0x7f7144107a20 2024-07-24T21:42:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.343+0000 7f7141ffb640 1 -- 172.21.15.60:0/3393445504 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f713c0aab40 con 0x7f7144107a20 2024-07-24T21:42:28.344 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:28.344 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"btime":"2024-07-24T21:42:27:889685+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-24T21:42:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-07-24T21:42:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 msgr2=0x7f71300b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 0x7f71300b60e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f712c005ab0 tx=0x7f712c005a20 comp rx=0 tx=0).stop 2024-07-24T21:42:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 msgr2=0x7f714419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f713c00cad0 tx=0x7f713c00cf90 comp rx=0 tx=0).stop 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 shutdown_connections 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71300b3c20 0x7f71300b60e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7144107a20 0x7f714419e540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71441070a0 0x7f71441991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 --2- 172.21.15.60:0/3393445504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7144101690 0x7f7144198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 >> 172.21.15.60:0/3393445504 conn(0x7f71440fd100 msgr2=0x7f714406b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 shutdown_connections 2024-07-24T21:42:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:28.345+0000 7f714afda640 1 -- 172.21.15.60:0/3393445504 wait complete. 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6832/1405683512,v1:172.21.15.123:6833/1405683512] up:boot 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: fsmap 1 up:standby 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:29.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: Deploying daemon mds.k on smithi086 2024-07-24T21:42:29.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:29.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:28 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3393445504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6832/1405683512,v1:172.21.15.123:6833/1405683512] up:boot 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: fsmap 1 up:standby 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: Deploying daemon mds.k on smithi086 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:29.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:28 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3393445504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:29.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6832/1405683512,v1:172.21.15.123:6833/1405683512] up:boot 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: fsmap 1 up:standby 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: Deploying daemon mds.k on smithi086 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:28 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3393445504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:30.035 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:29 smithi086 systemd[1]: Starting Ceph mds.k for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:30.295 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 podman[68257]: 2024-07-24 21:42:30.034256578 +0000 UTC m=+0.104332016 container create 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T21:42:30.295 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 podman[68257]: 2024-07-24 21:42:29.947084623 +0000 UTC m=+0.017160072 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:30.295 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 podman[68257]: 2024-07-24 21:42:30.236335641 +0000 UTC m=+0.306411080 container init 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-07-24T21:42:30.296 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 podman[68257]: 2024-07-24 21:42:30.238846803 +0000 UTC m=+0.308922242 container start 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T21:42:30.296 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 ceph-mds[68276]: -- 172.21.15.86:0/1869195321 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610ca7d9860 con 0x5610ca7b7000 2024-07-24T21:42:30.296 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: starting mds.k at 2024-07-24T21:42:30.795 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 bash[68257]: 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d 2024-07-24T21:42:30.795 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 21:42:30 smithi086 systemd[1]: Started Ceph mds.k for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:31.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:31.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:31.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: Deploying daemon mds.j on smithi060 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6832/3131020456,v1:172.21.15.86:6833/3131020456] up:boot 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: fsmap 2 up:standby 2024-07-24T21:42:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:31.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:31.668 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:31.668 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: Deploying daemon mds.j on smithi060 2024-07-24T21:42:31.668 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6832/3131020456,v1:172.21.15.86:6833/3131020456] up:boot 2024-07-24T21:42:31.668 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: fsmap 2 up:standby 2024-07-24T21:42:31.668 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: Deploying daemon mds.j on smithi060 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6832/3131020456,v1:172.21.15.86:6833/3131020456] up:boot 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: fsmap 2 up:standby 2024-07-24T21:42:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T21:42:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:33.666 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:33 smithi060 ceph-mon[33000]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:33.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:33 smithi123 ceph-mon[37143]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:33.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:33 smithi086 ceph-mon[37128]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 -- 172.21.15.60:0/195699943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 msgr2=0x7feedc1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 --2- 172.21.15.60:0/195699943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc1074a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feed0009900 tx=0x7feed002a410 comp rx=0 tx=0).stop 2024-07-24T21:42:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 -- 172.21.15.60:0/195699943 shutdown_connections 2024-07-24T21:42:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 --2- 172.21.15.60:0/195699943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 --2- 172.21.15.60:0/195699943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc1074a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 --2- 172.21.15.60:0/195699943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 0x7feedc101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 -- 172.21.15.60:0/195699943 >> 172.21.15.60:0/195699943 conn(0x7feedc0fd0c0 msgr2=0x7feedc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 -- 172.21.15.60:0/195699943 shutdown_connections 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.861+0000 7feee28c2640 1 -- 172.21.15.60:0/195699943 wait complete. 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 Processor -- start 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 -- start start 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 0x7feedc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feedc111690 con 0x7feedc107060 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feedc111510 con 0x7feedc101650 2024-07-24T21:42:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feee28c2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feedc111810 con 0x7feedc1079e0 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feedb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feedb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36880/0 (socket says 172.21.15.60:36880) 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.862+0000 7feedb7fe640 1 -- 172.21.15.60:0/1419906417 learned_addr learned my addr 172.21.15.60:0/1419906417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedbfff640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 0x7feedc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feee0e5f640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 -- 172.21.15.60:0/1419906417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 msgr2=0x7feedc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 -- 172.21.15.60:0/1419906417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 msgr2=0x7feedc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 0x7feedc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 -- 172.21.15.60:0/1419906417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc1a4f20 con 0x7feedc107060 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feee0e5f640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feedb7fe640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feed0005940 tx=0x7feed00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed003f030 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0003cb0 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc1a51b0 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0035070 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.863+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a56e0 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.864+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc101a30 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.866+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7feed0042030 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.866+0000 7feed97fa640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 0x7feec40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.867+0000 7feedbfff640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 0x7feec40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.867+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7feed0104360 con 0x7feedc107060 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.867+0000 7feedbfff640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 0x7feec40b6160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feecc009990 tx=0x7feecc009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:33.869+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feed00cbd60 con 0x7feedc107060 2024-07-24T21:42:34.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.106+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feedc02e000 con 0x7feedc107060 2024-07-24T21:42:34.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.107+0000 7feed97fa640 1 -- 172.21.15.60:0/1419906417 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7feed00cb660 con 0x7feedc107060 2024-07-24T21:42:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:34.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"btime":"2024-07-24T21:42:31:078915+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-24T21:42:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.109+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 msgr2=0x7feec40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.109+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 0x7feec40b6160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feecc009990 tx=0x7feecc009110 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.109+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 msgr2=0x7feedc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feed0005940 tx=0x7feed00042f0 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 shutdown_connections 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feec40b3ca0 0x7feec40b6160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc1079e0 0x7feedc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feedc107060 0x7feedc19a080 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 --2- 172.21.15.60:0/1419906417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feedc101650 0x7feedc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 >> 172.21.15.60:0/1419906417 conn(0x7feedc0fd0c0 msgr2=0x7feedc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 shutdown_connections 2024-07-24T21:42:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:34.110+0000 7feee28c2640 1 -- 172.21.15.60:0/1419906417 wait complete. 2024-07-24T21:42:34.709 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 21:42:34 smithi060 systemd[1]: Started Ceph mds.j for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:34.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:34 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1419906417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:34.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:34 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1419906417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:34.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:34 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1419906417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: Deploying daemon mds.d on smithi060 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.60:6834/1887502837,v1:172.21.15.60:6835/1887502837] up:boot 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: fsmap 3 up:standby 2024-07-24T21:42:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-24T21:42:35.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: Deploying daemon mds.d on smithi060 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.60:6834/1887502837,v1:172.21.15.60:6835/1887502837] up:boot 2024-07-24T21:42:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: fsmap 3 up:standby 2024-07-24T21:42:35.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: Deploying daemon mds.d on smithi060 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.60:6834/1887502837,v1:172.21.15.60:6835/1887502837] up:boot 2024-07-24T21:42:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: fsmap 3 up:standby 2024-07-24T21:42:36.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-24T21:42:37.018 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:36 smithi060 systemd[1]: Starting Ceph mds.d for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:37.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:37.321 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 podman[79682]: 2024-07-24 21:42:36.920333418 +0000 UTC m=+0.017794974 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:37.322 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 podman[79682]: 2024-07-24 21:42:37.019944532 +0000 UTC m=+0.117406116 container create 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-86385b1, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-07-24T21:42:37.322 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 podman[79682]: 2024-07-24 21:42:37.231108186 +0000 UTC m=+0.328569739 container init 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, io.buildah.version=1.36.0, ceph=True) 2024-07-24T21:42:37.322 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 podman[79682]: 2024-07-24 21:42:37.234649747 +0000 UTC m=+0.332111298 container start 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T21:42:37.322 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mds[79731]: -- 172.21.15.60:0/2749717834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cce2a99860 con 0x55cce2a76800 2024-07-24T21:42:37.322 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d[79715]: starting mds.d at 2024-07-24T21:42:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:37.599 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 bash[79682]: 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 2024-07-24T21:42:37.599 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 21:42:37 smithi060 systemd[1]: Started Ceph mds.d for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:37.873 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:37.874 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:38.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:38.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:38 smithi086 ceph-mon[37128]: Deploying daemon mds.e on smithi086 2024-07-24T21:42:38.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:38 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.60:6836/2896838776,v1:172.21.15.60:6837/2896838776] up:boot 2024-07-24T21:42:38.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:38 smithi086 ceph-mon[37128]: fsmap 4 up:standby 2024-07-24T21:42:38.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:38 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-24T21:42:38.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:38 smithi060 ceph-mon[33000]: Deploying daemon mds.e on smithi086 2024-07-24T21:42:38.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:38 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.60:6836/2896838776,v1:172.21.15.60:6837/2896838776] up:boot 2024-07-24T21:42:38.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:38 smithi060 ceph-mon[33000]: fsmap 4 up:standby 2024-07-24T21:42:38.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:38 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-24T21:42:38.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:38 smithi123 ceph-mon[37143]: Deploying daemon mds.e on smithi086 2024-07-24T21:42:38.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:38 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.60:6836/2896838776,v1:172.21.15.60:6837/2896838776] up:boot 2024-07-24T21:42:38.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:38 smithi123 ceph-mon[37143]: fsmap 4 up:standby 2024-07-24T21:42:38.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:38 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-24T21:42:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3122857298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8108b20 msgr2=0x7f86e810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3122857298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8108b20 0x7f86e810f320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009ca0 tx=0x7f86e002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3122857298 shutdown_connections 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3122857298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8108b20 0x7f86e810f320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3122857298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e81081a0 0x7f86e81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3122857298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e8102790 0x7f86e8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3122857298 >> 172.21.15.60:0/3122857298 conn(0x7f86e80fe200 msgr2=0x7f86e8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3122857298 shutdown_connections 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.068+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3122857298 wait complete. 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 Processor -- start 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 -- start start 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 0x7f86e819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 0x7f86e81a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81127a0 con 0x7f86e8102790 2024-07-24T21:42:39.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8112620 con 0x7f86e81081a0 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86ed7f3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8112920 con 0x7f86e8108b20 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56314/0 (socket says 172.21.15.60:56314) 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e6ffd640 1 -- 172.21.15.60:0/3296738813 learned_addr learned my addr 172.21.15.60:0/3296738813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e67fc640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 0x7f86e819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e77fe640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 0x7f86e81a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e6ffd640 1 -- 172.21.15.60:0/3296738813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 msgr2=0x7f86e81a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.069+0000 7f86e6ffd640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 0x7f86e81a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e6ffd640 1 -- 172.21.15.60:0/3296738813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 msgr2=0x7f86e819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e6ffd640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 0x7f86e819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e6ffd640 1 -- 172.21.15.60:0/3296738813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81a0f70 con 0x7f86e8102790 2024-07-24T21:42:39.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e77fe640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 0x7f86e81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e67fc640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 0x7f86e819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86e6ffd640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86d800d950 tx=0x7f86d800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8017070 con 0x7f86e8102790 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d800de80 con 0x7f86e8102790 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81a1260 con 0x7f86e8102790 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d800f620 con 0x7f86e8102790 2024-07-24T21:42:39.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.070+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a79a0 con 0x7f86e8102790 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.071+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e8102b70 con 0x7f86e8102790 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.072+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f86d8009c30 con 0x7f86e8102790 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.073+0000 7f86cffff640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 0x7f86d00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.073+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f86d80e3540 con 0x7f86e8102790 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.073+0000 7f86e67fc640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 0x7f86d00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:39.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.074+0000 7f86e67fc640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 0x7f86d00b60e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86e8104c20 tx=0x7f86dc009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:39.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.076+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f86d80aaec0 con 0x7f86e8102790 2024-07-24T21:42:39.295 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 systemd[1]: Starting Ceph mds.e for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:39.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.307+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86e802e000 con 0x7f86e8102790 2024-07-24T21:42:39.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.308+0000 7f86cffff640 1 -- 172.21.15.60:0/3296738813 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f86d80b3190 con 0x7f86e8102790 2024-07-24T21:42:39.310 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:39.310 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"btime":"2024-07-24T21:42:37:592350+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-24T21:42:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 5 2024-07-24T21:42:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 msgr2=0x7f86d00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 0x7f86d00b60e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86e8104c20 tx=0x7f86dc009110 comp rx=0 tx=0).stop 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 msgr2=0x7f86e819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f86d800d950 tx=0x7f86d800dd50 comp rx=0 tx=0).stop 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 shutdown_connections 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f86d00b3c20 0x7f86d00b60e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e8108b20 0x7f86e81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e81081a0 0x7f86e819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 --2- 172.21.15.60:0/3296738813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86e8102790 0x7f86e819aec0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 >> 172.21.15.60:0/3296738813 conn(0x7f86e80fe200 msgr2=0x7f86e8105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 shutdown_connections 2024-07-24T21:42:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:39.310+0000 7f86ed7f3640 1 -- 172.21.15.60:0/3296738813 wait complete. 2024-07-24T21:42:39.714 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:39 smithi086 ceph-mon[37128]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:39.714 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3296738813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:39.714 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 podman[68572]: 2024-07-24 21:42:39.305477495 +0000 UTC m=+0.017990929 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:39.714 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 podman[68572]: 2024-07-24 21:42:39.40293182 +0000 UTC m=+0.115445245 container create 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-24T21:42:39.715 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 podman[68572]: 2024-07-24 21:42:39.622236938 +0000 UTC m=+0.334750367 container init 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240716, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-24T21:42:39.715 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 podman[68572]: 2024-07-24 21:42:39.625025646 +0000 UTC m=+0.337539072 container start 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T21:42:39.715 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 ceph-mds[68591]: -- 172.21.15.86:0/3760190009 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56048baa5860 con 0x56048ba83000 2024-07-24T21:42:39.715 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: starting mds.e at 2024-07-24T21:42:39.715 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 bash[68572]: 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 2024-07-24T21:42:39.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:39 smithi060 ceph-mon[33000]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:39.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3296738813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:39.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:39 smithi123 ceph-mon[37143]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:39.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3296738813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:40.044 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 21:42:39 smithi086 systemd[1]: Started Ceph mds.e for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: Deploying daemon mds.f on smithi123 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6834/1999114136,v1:172.21.15.86:6835/1999114136] up:boot 2024-07-24T21:42:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: fsmap 5 up:standby 2024-07-24T21:42:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:41.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: Deploying daemon mds.f on smithi123 2024-07-24T21:42:41.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:41.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6834/1999114136,v1:172.21.15.86:6835/1999114136] up:boot 2024-07-24T21:42:41.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: fsmap 5 up:standby 2024-07-24T21:42:41.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: Deploying daemon mds.f on smithi123 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6834/1999114136,v1:172.21.15.86:6835/1999114136] up:boot 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: fsmap 5 up:standby 2024-07-24T21:42:41.074 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T21:42:41.743 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:41 smithi123 systemd[1]: Starting Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:42.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:41 smithi123 podman[69152]: 2024-07-24 21:42:41.664333276 +0000 UTC m=+0.017070896 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:41 smithi123 podman[69152]: 2024-07-24 21:42:41.775411578 +0000 UTC m=+0.128149201 container create 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:41 smithi123 podman[69152]: 2024-07-24 21:42:41.978052515 +0000 UTC m=+0.330790136 container init 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:41 smithi123 podman[69152]: 2024-07-24 21:42:41.980423415 +0000 UTC m=+0.333161053 container start 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716) 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:42 smithi123 ceph-mds[69171]: -- 172.21.15.123:0/1232465655 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f43121860 con 0x561f430ff400 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[69167]: starting mds.f at 2024-07-24T21:42:42.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:42 smithi123 bash[69152]: 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd 2024-07-24T21:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:42.492 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:42:42 smithi123 systemd[1]: Started Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:43.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6834/3784109187,v1:172.21.15.123:6835/3784109187] up:boot 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: fsmap 6 up:standby 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:43 smithi123 ceph-mon[37143]: Deploying daemon mds.c on smithi123 2024-07-24T21:42:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6834/3784109187,v1:172.21.15.123:6835/3784109187] up:boot 2024-07-24T21:42:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: fsmap 6 up:standby 2024-07-24T21:42:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:43 smithi060 ceph-mon[33000]: Deploying daemon mds.c on smithi123 2024-07-24T21:42:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6834/3784109187,v1:172.21.15.123:6835/3784109187] up:boot 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: fsmap 6 up:standby 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:43 smithi086 ceph-mon[37128]: Deploying daemon mds.c on smithi123 2024-07-24T21:42:43.993 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:43 smithi123 systemd[1]: Starting Ceph mds.c for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 -- 172.21.15.60:0/4231875195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 msgr2=0x7febac10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 --2- 172.21.15.60:0/4231875195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac10f320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feba4009ca0 tx=0x7feba402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 -- 172.21.15.60:0/4231875195 shutdown_connections 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 --2- 172.21.15.60:0/4231875195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac10f320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 --2- 172.21.15.60:0/4231875195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac1081a0 0x7febac1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 --2- 172.21.15.60:0/4231875195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac102790 0x7febac102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 -- 172.21.15.60:0/4231875195 >> 172.21.15.60:0/4231875195 conn(0x7febac0fe200 msgr2=0x7febac100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 -- 172.21.15.60:0/4231875195 shutdown_connections 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.068+0000 7febb0f19640 1 -- 172.21.15.60:0/4231875195 wait complete. 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.069+0000 7febb0f19640 1 Processor -- start 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.069+0000 7febb0f19640 1 -- start start 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac102790 0x7febac198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac1081a0 0x7febac199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7febac112790 con 0x7febac108b20 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febac112610 con 0x7febac102790 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febac112910 con 0x7febac1081a0 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56342/0 (socket says 172.21.15.60:56342) 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 -- 172.21.15.60:0/252241104 learned_addr learned my addr 172.21.15.60:0/252241104 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaa59c640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac102790 0x7febac198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7feba9d9b640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac1081a0 0x7febac199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 -- 172.21.15.60:0/252241104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac1081a0 msgr2=0x7febac199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac1081a0 0x7febac199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 -- 172.21.15.60:0/252241104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac102790 msgr2=0x7febac198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac102790 0x7febac198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 -- 172.21.15.60:0/252241104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac19edd0 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febaad9d640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feba4009430 tx=0x7feba40317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4003b70 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac19f060 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba4003d10 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.070+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba402fa30 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.071+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1a5780 con 0x7febac108b20 2024-07-24T21:42:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.072+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac102b70 con 0x7febac108b20 2024-07-24T21:42:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.073+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7feba402fbd0 con 0x7febac108b20 2024-07-24T21:42:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.074+0000 7feb9b7fe640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 0x7feb940b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.074+0000 7febaa59c640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 0x7feb940b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.074+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7feba4104e20 con 0x7febac108b20 2024-07-24T21:42:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.074+0000 7febaa59c640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 0x7feb940b60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005fd0 tx=0x7feb9c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.078+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7feba40cc7a0 con 0x7febac108b20 2024-07-24T21:42:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.315+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febac10a900 con 0x7febac108b20 2024-07-24T21:42:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.315+0000 7feb9b7fe640 1 -- 172.21.15.60:0/252241104 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5566 (secure 0 0 0) 0x7feba40cc0a0 con 0x7febac108b20 2024-07-24T21:42:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:44.317 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":7,"btime":"2024-07-24T21:42:42:057985+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24560,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-07-24T21:42:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 7 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.317+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 msgr2=0x7feb940b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.317+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 0x7feb940b60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005fd0 tx=0x7feb9c00b390 comp rx=0 tx=0).stop 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.317+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 msgr2=0x7febac19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.317+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feba4009430 tx=0x7feba40317c0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 shutdown_connections 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feb940b3c20 0x7feb940b60e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febac108b20 0x7febac19e600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac1081a0 0x7febac199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 --2- 172.21.15.60:0/252241104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac102790 0x7febac198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 >> 172.21.15.60:0/252241104 conn(0x7febac0fe200 msgr2=0x7febac1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 shutdown_connections 2024-07-24T21:42:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:44.318+0000 7febb0f19640 1 -- 172.21.15.60:0/252241104 wait complete. 2024-07-24T21:42:44.364 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 podman[69456]: 2024-07-24 21:42:44.009214517 +0000 UTC m=+0.017307645 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:44.364 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 podman[69456]: 2024-07-24 21:42:44.110863398 +0000 UTC m=+0.118956512 container create 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_BRANCH=HEAD) 2024-07-24T21:42:44.364 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 podman[69456]: 2024-07-24 21:42:44.338225543 +0000 UTC m=+0.346318667 container init 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240716, RELEASE=main-86385b1) 2024-07-24T21:42:44.364 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 podman[69456]: 2024-07-24 21:42:44.340983481 +0000 UTC m=+0.349076602 container start 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS) 2024-07-24T21:42:44.742 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 ceph-mds[69475]: -- 172.21.15.123:0/2427634213 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561aeced860 con 0x5561aeccb000 2024-07-24T21:42:44.742 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: starting mds.c at 2024-07-24T21:42:44.742 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 bash[69456]: 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 2024-07-24T21:42:44.743 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 21:42:44 smithi123 systemd[1]: Started Ceph mds.c for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/252241104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:45 smithi086 ceph-mon[37128]: Deploying daemon mds.h on smithi086 2024-07-24T21:42:45.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:45.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/252241104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:45.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:45 smithi060 ceph-mon[33000]: Deploying daemon mds.h on smithi086 2024-07-24T21:42:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/252241104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:45.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:45 smithi123 ceph-mon[37143]: Deploying daemon mds.h on smithi086 2024-07-24T21:42:46.114 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:46 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:boot 2024-07-24T21:42:46.114 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:46 smithi086 ceph-mon[37128]: fsmap 7 up:standby 2024-07-24T21:42:46.114 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:46 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T21:42:46.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:46 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:boot 2024-07-24T21:42:46.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:46 smithi123 ceph-mon[37143]: fsmap 7 up:standby 2024-07-24T21:42:46.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:46 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T21:42:46.503 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 systemd[1]: Starting Ceph mds.h for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:46.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:46 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:boot 2024-07-24T21:42:46.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:46 smithi060 ceph-mon[33000]: fsmap 7 up:standby 2024-07-24T21:42:46.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:46 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T21:42:46.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 podman[68873]: 2024-07-24 21:42:46.405041617 +0000 UTC m=+0.017431877 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:46.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 podman[68873]: 2024-07-24 21:42:46.505912161 +0000 UTC m=+0.118302467 container create 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-24T21:42:46.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 podman[68873]: 2024-07-24 21:42:46.724850173 +0000 UTC m=+0.337240434 container init 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD) 2024-07-24T21:42:46.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 podman[68873]: 2024-07-24 21:42:46.72733102 +0000 UTC m=+0.339721281 container start 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h, RELEASE=main-86385b1, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-24T21:42:46.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 ceph-mds[68893]: -- 172.21.15.86:0/3832850735 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b27da7860 con 0x556b27d85000 2024-07-24T21:42:46.796 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: starting mds.h at 2024-07-24T21:42:46.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:47.077 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 bash[68873]: 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 2024-07-24T21:42:47.078 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 21:42:46 smithi086 systemd[1]: Started Ceph mds.h for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:47.259 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:47.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:47.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:47.331 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:47.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:47.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:48.386 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:48 smithi123 ceph-mon[37143]: Deploying daemon mds.i on smithi123 2024-07-24T21:42:48.387 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:48 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6836/549557535,v1:172.21.15.86:6837/549557535] up:boot 2024-07-24T21:42:48.387 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:48 smithi123 ceph-mon[37143]: fsmap 8 up:standby 2024-07-24T21:42:48.387 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:48 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T21:42:48.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:48 smithi060 ceph-mon[33000]: Deploying daemon mds.i on smithi123 2024-07-24T21:42:48.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:48 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6836/549557535,v1:172.21.15.86:6837/549557535] up:boot 2024-07-24T21:42:48.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:48 smithi060 ceph-mon[33000]: fsmap 8 up:standby 2024-07-24T21:42:48.460 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:48 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T21:42:48.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:48 smithi086 ceph-mon[37128]: Deploying daemon mds.i on smithi123 2024-07-24T21:42:48.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:48 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6836/549557535,v1:172.21.15.86:6837/549557535] up:boot 2024-07-24T21:42:48.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:48 smithi086 ceph-mon[37128]: fsmap 8 up:standby 2024-07-24T21:42:48.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:48 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 -- 172.21.15.60:0/514638667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998101650 msgr2=0x7fc998101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 --2- 172.21.15.60:0/514638667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998101650 0x7fc998101a30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc988009900 tx=0x7fc98802a410 comp rx=0 tx=0).stop 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 -- 172.21.15.60:0/514638667 shutdown_connections 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 --2- 172.21.15.60:0/514638667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9981079e0 0x7fc99810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 --2- 172.21.15.60:0/514638667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9981074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 --2- 172.21.15.60:0/514638667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998101650 0x7fc998101a30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 -- 172.21.15.60:0/514638667 >> 172.21.15.60:0/514638667 conn(0x7fc9980fd0c0 msgr2=0x7fc9980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 -- 172.21.15.60:0/514638667 shutdown_connections 2024-07-24T21:42:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.972+0000 7fc99dad0640 1 -- 172.21.15.60:0/514638667 wait complete. 2024-07-24T21:42:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.973+0000 7fc99dad0640 1 Processor -- start 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.973+0000 7fc99dad0640 1 -- start start 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc998101650 0x7fc9980fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9980fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 0x7fc9980fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc998111610 con 0x7fc9981079e0 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc998111490 con 0x7fc998101650 2024-07-24T21:42:48.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc99dad0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc998111790 con 0x7fc998107060 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc997fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 0x7fc9980fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc997fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 0x7fc9980fe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49794/0 (socket says 172.21.15.60:49794) 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc997fff640 1 -- 172.21.15.60:0/2607223184 learned_addr learned my addr 172.21.15.60:0/2607223184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9980fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc9977fe640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc998101650 0x7fc9980fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 -- 172.21.15.60:0/2607223184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc998101650 msgr2=0x7fc9980fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc998101650 0x7fc9980fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 -- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 msgr2=0x7fc9980fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 0x7fc9980fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 -- 172.21.15.60:0/2607223184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981a7170 con 0x7fc998107060 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.974+0000 7fc996ffd640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9980fdb00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.975+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00ccc0 con 0x7fc998107060 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.975+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981a7460 con 0x7fc998107060 2024-07-24T21:42:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.975+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981a79a0 con 0x7fc998107060 2024-07-24T21:42:48.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.975+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c0044d0 con 0x7fc998107060 2024-07-24T21:42:48.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.975+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01c070 con 0x7fc998107060 2024-07-24T21:42:48.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.976+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9840052c0 con 0x7fc998107060 2024-07-24T21:42:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.977+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc98c005020 con 0x7fc998107060 2024-07-24T21:42:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.978+0000 7fc994ff9640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 0x7fc9800b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.978+0000 7fc9977fe640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 0x7fc9800b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.978+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7fc98c0e3790 con 0x7fc998107060 2024-07-24T21:42:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.978+0000 7fc9977fe640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 0x7fc9800b60e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc988031440 tx=0x7fc988005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:48.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:48.980+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc98c0ab110 con 0x7fc998107060 2024-07-24T21:42:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.213+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9840055b0 con 0x7fc998107060 2024-07-24T21:42:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.214+0000 7fc994ff9640 1 -- 172.21.15.60:0/2607223184 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7fc98c0aaa10 con 0x7fc998107060 2024-07-24T21:42:49.215 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:49.215 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":9,"btime":"2024-07-24T21:42:47:072261+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24560,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24578,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-07-24T21:42:49.216 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 9 2024-07-24T21:42:49.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 msgr2=0x7fc9800b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:49.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 0x7fc9800b60e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc988031440 tx=0x7fc988005b50 comp rx=0 tx=0).stop 2024-07-24T21:42:49.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 msgr2=0x7fc9980fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:49.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9980fdb00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 shutdown_connections 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9800b3c20 0x7fc9800b60e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981079e0 0x7fc9980fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc998107060 0x7fc9980fdb00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 --2- 172.21.15.60:0/2607223184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc998101650 0x7fc9980fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 >> 172.21.15.60:0/2607223184 conn(0x7fc9980fd0c0 msgr2=0x7fc99806a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 shutdown_connections 2024-07-24T21:42:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:49.216+0000 7fc99dad0640 1 -- 172.21.15.60:0/2607223184 wait complete. 2024-07-24T21:42:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:49 smithi123 ceph-mon[37143]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:49.492 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:42:49 smithi123 systemd[1]: Started Ceph mds.i for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:49 smithi086 ceph-mon[37128]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:49 smithi060 ceph-mon[33000]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:50.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2607223184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:50.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:50.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:50.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:50.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:50 smithi060 ceph-mon[33000]: Deploying daemon mds.g on smithi060 2024-07-24T21:42:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2607223184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:50.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:50.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:50.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:50 smithi123 ceph-mon[37143]: Deploying daemon mds.g on smithi060 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2607223184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:50 smithi086 ceph-mon[37128]: Deploying daemon mds.g on smithi060 2024-07-24T21:42:51.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:51 smithi060 ceph-mon[33000]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:51.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:51 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:boot 2024-07-24T21:42:51.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:51 smithi060 ceph-mon[33000]: fsmap 9 up:standby 2024-07-24T21:42:51.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T21:42:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:51 smithi123 ceph-mon[37143]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:51 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:boot 2024-07-24T21:42:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:51 smithi123 ceph-mon[37143]: fsmap 9 up:standby 2024-07-24T21:42:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T21:42:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:51 smithi086 ceph-mon[37128]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:51 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:boot 2024-07-24T21:42:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:51 smithi086 ceph-mon[37128]: fsmap 9 up:standby 2024-07-24T21:42:51.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T21:42:51.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:52.022 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:51 smithi060 systemd[1]: Starting Ceph mds.g for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:52.321 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 podman[80472]: 2024-07-24 21:42:51.963392564 +0000 UTC m=+0.017729397 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:52.321 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 podman[80472]: 2024-07-24 21:42:52.092834715 +0000 UTC m=+0.147171534 container create c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g, io.buildah.version=1.36.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, ceph=True) 2024-07-24T21:42:52.601 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:52.728 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 podman[80472]: 2024-07-24 21:42:52.462721663 +0000 UTC m=+0.517058499 container init c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-24T21:42:52.728 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 podman[80472]: 2024-07-24 21:42:52.466091926 +0000 UTC m=+0.520428745 container start c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-24T21:42:52.728 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 ceph-mds[80535]: -- 172.21.15.60:0/776766308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f7fbdd860 con 0x562f7fbba800 2024-07-24T21:42:52.728 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g[80517]: starting mds.g at 2024-07-24T21:42:52.729 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 bash[80472]: c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 2024-07-24T21:42:52.729 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 21:42:52 smithi060 systemd[1]: Started Ceph mds.g for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: Deploying daemon mds.a on smithi060 2024-07-24T21:42:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.60:6838/3968790109,v1:172.21.15.60:6839/3968790109] up:boot 2024-07-24T21:42:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: fsmap 10 up:standby 2024-07-24T21:42:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:53 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-24T21:42:53.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:53.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:53.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:53.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:53.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:53.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: Deploying daemon mds.a on smithi060 2024-07-24T21:42:53.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.60:6838/3968790109,v1:172.21.15.60:6839/3968790109] up:boot 2024-07-24T21:42:53.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: fsmap 10 up:standby 2024-07-24T21:42:53.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:53 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: Deploying daemon mds.a on smithi060 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.60:6838/3968790109,v1:172.21.15.60:6839/3968790109] up:boot 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: fsmap 10 up:standby 2024-07-24T21:42:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:53 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-24T21:42:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 -- 172.21.15.60:0/4012937266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5198101650 msgr2=0x7f5198101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 --2- 172.21.15.60:0/4012937266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5198101650 0x7f5198101a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402a410 comp rx=0 tx=0).stop 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 -- 172.21.15.60:0/4012937266 shutdown_connections 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 --2- 172.21.15.60:0/4012937266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51981079e0 0x7f519810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 --2- 172.21.15.60:0/4012937266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198107060 0x7f51981074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 --2- 172.21.15.60:0/4012937266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5198101650 0x7f5198101a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 -- 172.21.15.60:0/4012937266 >> 172.21.15.60:0/4012937266 conn(0x7f51980fd0c0 msgr2=0x7f51980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 -- 172.21.15.60:0/4012937266 shutdown_connections 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.387+0000 7f519f5e8640 1 -- 172.21.15.60:0/4012937266 wait complete. 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 Processor -- start 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 -- start start 2024-07-24T21:42:55.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198101650 0x7f5198198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 0x7f51981991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5198111660 con 0x7f51981079e0 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51981114e0 con 0x7f5198101650 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519f5e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51981117e0 con 0x7f5198107060 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519db85640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519db85640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49830/0 (socket says 172.21.15.60:49830) 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519db85640 1 -- 172.21.15.60:0/1577757535 learned_addr learned my addr 172.21.15.60:0/1577757535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:42:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.388+0000 7f519d384640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198101650 0x7f5198198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519cb83640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 0x7f51981991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 -- 172.21.15.60:0/1577757535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 msgr2=0x7f51981991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 0x7f51981991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 -- 172.21.15.60:0/1577757535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198101650 msgr2=0x7f5198198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198101650 0x7f5198198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 -- 172.21.15.60:0/1577757535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519819ed10 con 0x7f51981079e0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519cb83640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 0x7f51981991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519db85640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f519000cad0 tx=0x7f519000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000ae20 con 0x7f51981079e0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519819f000 con 0x7f51981079e0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5190005020 con 0x7f51981079e0 2024-07-24T21:42:55.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000f6b0 con 0x7f51981079e0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.389+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981a56c0 con 0x7f51981079e0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.390+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5198101a30 con 0x7f51981079e0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.393+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f51900051d0 con 0x7f51981079e0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.394+0000 7f518e7fc640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 0x7f51840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.394+0000 7f519d384640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 0x7f51840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.394+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f51900e3ca0 con 0x7f51981079e0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.394+0000 7f519d384640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 0x7f51840b60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5194005ab0 tx=0x7f5194005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:42:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.395+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5190014030 con 0x7f51981079e0 2024-07-24T21:42:55.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:55 smithi060 ceph-mon[33000]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:55.648 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:55 smithi060 systemd[1]: Starting Ceph mds.a for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.646+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5198199d90 con 0x7f51981079e0 2024-07-24T21:42:55.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.647+0000 7f518e7fc640 1 -- 172.21.15.60:0/1577757535 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8872 (secure 0 0 0) 0x7f51900ab6c0 con 0x7f51981079e0 2024-07-24T21:42:55.649 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:42:55.649 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"btime":"2024-07-24T21:42:53:104174+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24560,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24578,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-24T21:42:55.649 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 msgr2=0x7f51840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 0x7f51840b60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5194005ab0 tx=0x7f5194005a20 comp rx=0 tx=0).stop 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 msgr2=0x7f519819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f519000cad0 tx=0x7f519000cf90 comp rx=0 tx=0).stop 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 shutdown_connections 2024-07-24T21:42:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51840b3c20 0x7f51840b60e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51981079e0 0x7f519819e540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5198107060 0x7f51981991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 --2- 172.21.15.60:0/1577757535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5198101650 0x7f5198198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 >> 172.21.15.60:0/1577757535 conn(0x7f51980fd0c0 msgr2=0x7f519806a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 shutdown_connections 2024-07-24T21:42:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:42:55.650+0000 7f519f5e8640 1 -- 172.21.15.60:0/1577757535 wait complete. 2024-07-24T21:42:55.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:55 smithi123 ceph-mon[37143]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:55 smithi086 ceph-mon[37128]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:56.612 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 podman[80929]: 2024-07-24 21:42:56.44346925 +0000 UTC m=+0.018063168 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:56.612 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 podman[80929]: 2024-07-24 21:42:56.564626448 +0000 UTC m=+0.139220360 container create 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-24T21:42:56.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1577757535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:56.901 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 podman[80929]: 2024-07-24 21:42:56.817164555 +0000 UTC m=+0.391758487 container init 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-07-24T21:42:56.901 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 podman[80929]: 2024-07-24 21:42:56.820325477 +0000 UTC m=+0.394919390 container start 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240716, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-24T21:42:56.901 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 ceph-mds[80948]: -- 172.21.15.60:0/1703763435 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56195e729860 con 0x56195e707000 2024-07-24T21:42:56.901 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a[80944]: starting mds.a at 2024-07-24T21:42:56.902 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 bash[80929]: 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 2024-07-24T21:42:56.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:56 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1577757535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:57.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:56 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1577757535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:42:57.321 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 21:42:56 smithi060 systemd[1]: Started Ceph mds.a for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:57.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:57.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:57.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:58.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:58.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:58.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:58.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:58.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-24T21:42:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-24T21:42:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:42:58.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:42:58.653 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:42:58.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:58 smithi060 ceph-mon[33000]: Deploying daemon mds.b on smithi086 2024-07-24T21:42:58.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:58 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.60:6840/2105555291,v1:172.21.15.60:6841/2105555291] up:boot 2024-07-24T21:42:58.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:58 smithi060 ceph-mon[33000]: fsmap 11 up:standby 2024-07-24T21:42:58.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:58 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-24T21:42:58.971 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:58 smithi086 ceph-mon[37128]: Deploying daemon mds.b on smithi086 2024-07-24T21:42:58.972 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:58 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.60:6840/2105555291,v1:172.21.15.60:6841/2105555291] up:boot 2024-07-24T21:42:58.972 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:58 smithi086 ceph-mon[37128]: fsmap 11 up:standby 2024-07-24T21:42:58.972 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:58 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-24T21:42:58.972 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:58 smithi086 systemd[1]: Starting Ceph mds.b for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:42:58.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:58 smithi123 ceph-mon[37143]: Deploying daemon mds.b on smithi086 2024-07-24T21:42:58.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:58 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.60:6840/2105555291,v1:172.21.15.60:6841/2105555291] up:boot 2024-07-24T21:42:58.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:58 smithi123 ceph-mon[37143]: fsmap 11 up:standby 2024-07-24T21:42:58.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:58 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-24T21:42:59.285 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:58 smithi086 podman[69180]: 2024-07-24 21:42:58.873127536 +0000 UTC m=+0.017368001 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:42:59.285 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:58 smithi086 podman[69180]: 2024-07-24 21:42:58.974614478 +0000 UTC m=+0.118854985 container create a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-24T21:42:59.286 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 podman[69180]: 2024-07-24 21:42:59.185092931 +0000 UTC m=+0.329333396 container init a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240716) 2024-07-24T21:42:59.286 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 podman[69180]: 2024-07-24 21:42:59.188022285 +0000 UTC m=+0.332262750 container start a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-24T21:42:59.286 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mds[69199]: -- 172.21.15.86:0/3326916048 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563850eff860 con 0x563850edc800 2024-07-24T21:42:59.286 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: starting mds.b at 2024-07-24T21:42:59.286 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 bash[69180]: a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e 2024-07-24T21:42:59.544 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:42:59 smithi086 systemd[1]: Started Ceph mds.b for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:42:59.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:42:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:42:59 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:00.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:42:59 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:00.040 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:00.040 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.041 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.041 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.041 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.041 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:00.041 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:42:59 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 -- 172.21.15.60:0/787750507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103f80 msgr2=0x7f5e2010c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 --2- 172.21.15.60:0/787750507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103f80 0x7f5e2010c470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009900 tx=0x7f5e1402a410 comp rx=0 tx=0).stop 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 -- 172.21.15.60:0/787750507 shutdown_connections 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 --2- 172.21.15.60:0/787750507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 0x7f5e2010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 --2- 172.21.15.60:0/787750507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103f80 0x7f5e2010c470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 --2- 172.21.15.60:0/787750507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103660 0x7f5e20103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 -- 172.21.15.60:0/787750507 >> 172.21.15.60:0/787750507 conn(0x7f5e200fd0c0 msgr2=0x7f5e200ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 -- 172.21.15.60:0/787750507 shutdown_connections 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 -- 172.21.15.60:0/787750507 wait complete. 2024-07-24T21:43:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.344+0000 7f5e268a5640 1 Processor -- start 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 -- start start 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103f80 0x7f5e20199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 0x7f5e2019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e20112700 con 0x7f5e20103660 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e20112580 con 0x7f5e20103f80 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e268a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e20112880 con 0x7f5e2010cac0 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60710/0 (socket says 172.21.15.60:60710) 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 -- 172.21.15.60:0/3925957870 learned_addr learned my addr 172.21.15.60:0/3925957870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e24e42640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 0x7f5e2019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1f7fe640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103f80 0x7f5e20199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 -- 172.21.15.60:0/3925957870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 msgr2=0x7f5e2019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 0x7f5e2019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 -- 172.21.15.60:0/3925957870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103f80 msgr2=0x7f5e20199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103f80 0x7f5e20199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.345+0000 7f5e1ffff640 1 -- 172.21.15.60:0/3925957870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2019ece0 con 0x7f5e20103660 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e1ffff640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c890 tx=0x7f5e1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004220 con 0x7f5e20103660 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e100043c0 con 0x7f5e20103660 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2019efd0 con 0x7f5e20103660 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004a00 con 0x7f5e20103660 2024-07-24T21:43:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.346+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e201a5860 con 0x7f5e20103660 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.347+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e20103a40 con 0x7f5e20103660 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.348+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f5e100079b0 con 0x7f5e20103660 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.349+0000 7f5e1d7fa640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 0x7f5e080b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.349+0000 7f5e1f7fe640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 0x7f5e080b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.349+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7f5e100e3a40 con 0x7f5e20103660 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.349+0000 7f5e1f7fe640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 0x7f5e080b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e14000c00 tx=0x7f5e14031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.352+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5e10014030 con 0x7f5e20103660 2024-07-24T21:43:00.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.605+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e20199d60 con 0x7f5e20103660 2024-07-24T21:43:00.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.606+0000 7f5e1d7fa640 1 -- 172.21.15.60:0/3925957870 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7f5e20199d60 con 0x7f5e20103660 2024-07-24T21:43:00.607 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:00.607 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"btime":"2024-07-24T21:42:59:623876+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24560,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24578,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-24T21:43:00.608 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2024-07-24T21:43:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 msgr2=0x7f5e080b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 0x7f5e080b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e14000c00 tx=0x7f5e14031400 comp rx=0 tx=0).stop 2024-07-24T21:43:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 msgr2=0x7f5e20198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c890 tx=0x7f5e1000cd50 comp rx=0 tx=0).stop 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 shutdown_connections 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5e080b3ca0 0x7f5e080b6160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e2010cac0 0x7f5e2019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e20103f80 0x7f5e20199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 --2- 172.21.15.60:0/3925957870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20103660 0x7f5e20198c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 >> 172.21.15.60:0/3925957870 conn(0x7f5e200fd0c0 msgr2=0x7f5e200fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 shutdown_connections 2024-07-24T21:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:00.608+0000 7f5e268a5640 1 -- 172.21.15.60:0/3925957870 wait complete. 2024-07-24T21:43:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:00 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:boot 2024-07-24T21:43:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:00 smithi060 ceph-mon[33000]: fsmap 12 up:standby 2024-07-24T21:43:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:00 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T21:43:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:00 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3925957870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:00 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:boot 2024-07-24T21:43:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:00 smithi123 ceph-mon[37143]: fsmap 12 up:standby 2024-07-24T21:43:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:00 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T21:43:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:00 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3925957870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:00 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:boot 2024-07-24T21:43:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:00 smithi086 ceph-mon[37128]: fsmap 12 up:standby 2024-07-24T21:43:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:00 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T21:43:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:00 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3925957870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:01.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:01.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:01 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-24T21:43:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:01 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:01 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:02.133 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.545 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:03.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:43:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 -- 172.21.15.60:0/604215705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda810caa0 msgr2=0x7fcda810ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/604215705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda810caa0 0x7fcda810ef10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009ca0 tx=0x7fcda002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 -- 172.21.15.60:0/604215705 shutdown_connections 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/604215705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda810caa0 0x7fcda810ef10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/604215705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda8103ef0 0x7fcda810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/604215705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 0x7fcda81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 -- 172.21.15.60:0/604215705 >> 172.21.15.60:0/604215705 conn(0x7fcda80fd0c0 msgr2=0x7fcda80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 -- 172.21.15.60:0/604215705 shutdown_connections 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.891+0000 7fcdadbb9640 1 -- 172.21.15.60:0/604215705 wait complete. 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 Processor -- start 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 -- start start 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 0x7fcda8101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda810caa0 0x7fcda8101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81126f0 con 0x7fcda8103ef0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112570 con 0x7fcda810caa0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcdadbb9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112870 con 0x7fcda81034f0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcda6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcda6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60720/0 (socket says 172.21.15.60:60720) 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcda6ffd640 1 -- 172.21.15.60:0/1626273856 learned_addr learned my addr 172.21.15.60:0/1626273856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcda77fe640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 0x7fcda8101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.892+0000 7fcda7fff640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda810caa0 0x7fcda8101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 -- 172.21.15.60:0/1626273856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 msgr2=0x7fcda8101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 0x7fcda8101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 -- 172.21.15.60:0/1626273856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda810caa0 msgr2=0x7fcda8101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda810caa0 0x7fcda8101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 -- 172.21.15.60:0/1626273856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda81a7020 con 0x7fcda8103ef0 2024-07-24T21:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda6ffd640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00e9f0 tx=0x7fcd9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00cd80 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9c0108e0 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda81a7310 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c018980 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.893+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a78a0 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.894+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda81038d0 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.896+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fcd9c010430 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.896+0000 7fcda4ff9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 0x7fcd900b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.896+0000 7fcda77fe640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 0x7fcd900b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.896+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7fcd9c0e3a80 con 0x7fcda8103ef0 2024-07-24T21:43:03.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.897+0000 7fcda77fe640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 0x7fcd900b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005d00 tx=0x7fcd9800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:03.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:03.900+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcd9c0aae50 con 0x7fcda8103ef0 2024-07-24T21:43:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.117+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcda8102670 con 0x7fcda8103ef0 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.118+0000 7fcda4ff9640 1 -- 172.21.15.60:0/1626273856 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fcd9c0af3a0 con 0x7fcda8103ef0 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:43:04.119 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 msgr2=0x7fcd900b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 0x7fcd900b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd98005d00 tx=0x7fcd9800b410 comp rx=0 tx=0).stop 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 msgr2=0x7fcda81018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00e9f0 tx=0x7fcd9c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 shutdown_connections 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd900b3ca0 0x7fcd900b6160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcda810caa0 0x7fcda8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8103ef0 0x7fcda81018b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 --2- 172.21.15.60:0/1626273856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcda81034f0 0x7fcda8101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 >> 172.21.15.60:0/1626273856 conn(0x7fcda80fd0c0 msgr2=0x7fcda80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 shutdown_connections 2024-07-24T21:43:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:04.120+0000 7fcdadbb9640 1 -- 172.21.15.60:0/1626273856 wait complete. 2024-07-24T21:43:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:04.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:04 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1626273856' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.165 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:04 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1626273856' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-24T21:43:05.198 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-24T21:43:05.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:05.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:04 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1626273856' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-24T21:43:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.095+0000 7fab6ca2e640 1 -- 172.21.15.60:0/3687043571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68107080 msgr2=0x7fab681074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.095+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/3687043571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68107080 0x7fab681074c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009900 tx=0x7fab5c02a410 comp rx=0 tx=0).stop 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 -- 172.21.15.60:0/3687043571 shutdown_connections 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/3687043571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107a00 0x7fab6810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/3687043571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68107080 0x7fab681074c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/3687043571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68101670 0x7fab68101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 -- 172.21.15.60:0/3687043571 >> 172.21.15.60:0/3687043571 conn(0x7fab680fd0e0 msgr2=0x7fab680ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 -- 172.21.15.60:0/3687043571 shutdown_connections 2024-07-24T21:43:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.096+0000 7fab6ca2e640 1 -- 172.21.15.60:0/3687043571 wait complete. 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 Processor -- start 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 -- start start 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107080 0x7fab68199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 0x7fab6819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fab68111680 con 0x7fab68101670 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab68111500 con 0x7fab68107a00 2024-07-24T21:43:07.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6ca2e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab68111800 con 0x7fab68107080 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46262/0 (socket says 172.21.15.60:46262) 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 -- 172.21.15.60:0/789711968 learned_addr learned my addr 172.21.15.60:0/789711968 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab65d9b640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107080 0x7fab68199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab66d9d640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 0x7fab6819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 -- 172.21.15.60:0/789711968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107080 msgr2=0x7fab68199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107080 0x7fab68199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 -- 172.21.15.60:0/789711968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 msgr2=0x7fab6819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 0x7fab6819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab6659c640 1 -- 172.21.15.60:0/789711968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab681a4e90 con 0x7fab68101670 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.097+0000 7fab66d9d640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 0x7fab6819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:43:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab6659c640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab5800eab0 tx=0x7fab5800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800ce40 con 0x7fab68101670 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab58011920 con 0x7fab68101670 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab681a5180 con 0x7fab68101670 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58010a30 con 0x7fab68101670 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.098+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab681a5710 con 0x7fab68101670 2024-07-24T21:43:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.099+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68101a50 con 0x7fab68101670 2024-07-24T21:43:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.100+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fab58010bd0 con 0x7fab68101670 2024-07-24T21:43:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.101+0000 7fab577fe640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 0x7fab500b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.101+0000 7fab65d9b640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 0x7fab500b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.101+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7342+0+0 (secure 0 0 0) 0x7fab580e3f20 con 0x7fab68101670 2024-07-24T21:43:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.102+0000 7fab65d9b640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 0x7fab500b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab5c031440 tx=0x7fab5c005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:07.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.104+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fab58016030 con 0x7fab68101670 2024-07-24T21:43:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:07.299+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fab6806b260 con 0x7fab68101670 2024-07-24T21:43:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:43:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:43:08.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.238+0000 7fab577fe640 1 -- 172.21.15.60:0/789711968 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fab580ab8a0 con 0x7fab68101670 2024-07-24T21:43:08.240 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-07-24T21:43:08.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 msgr2=0x7fab500b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 0x7fab500b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab5c031440 tx=0x7fab5c005b50 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 msgr2=0x7fab68199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab5800eab0 tx=0x7fab5800ef70 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 shutdown_connections 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab500b3c20 0x7fab500b60e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab68107a00 0x7fab6819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab68107080 0x7fab68199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 --2- 172.21.15.60:0/789711968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab68101670 0x7fab68199ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.242+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 >> 172.21.15.60:0/789711968 conn(0x7fab680fd0e0 msgr2=0x7fab68104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.243+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 shutdown_connections 2024-07-24T21:43:08.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:08.243+0000 7fab6ca2e640 1 -- 172.21.15.60:0/789711968 wait complete. 2024-07-24T21:43:08.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:08 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-24T21:43:08.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:08 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-24T21:43:08.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:08 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-24T21:43:08.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-24T21:43:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:09.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:09 smithi060 ceph-mon[33000]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:09.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:09 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-24T21:43:09.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:09 smithi060 ceph-mon[33000]: osdmap e77: 12 total, 12 up, 12 in 2024-07-24T21:43:09.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:09 smithi086 ceph-mon[37128]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:09.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:09 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-24T21:43:09.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:09 smithi086 ceph-mon[37128]: osdmap e77: 12 total, 12 up, 12 in 2024-07-24T21:43:09.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:09 smithi123 ceph-mon[37143]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:09.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:09 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/789711968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-24T21:43:09.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:09 smithi123 ceph-mon[37143]: osdmap e77: 12 total, 12 up, 12 in 2024-07-24T21:43:10.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:10 smithi060 ceph-mon[33000]: osdmap e78: 12 total, 12 up, 12 in 2024-07-24T21:43:10.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:10 smithi060 ceph-mon[33000]: osdmap e79: 12 total, 12 up, 12 in 2024-07-24T21:43:10.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:10 smithi086 ceph-mon[37128]: osdmap e78: 12 total, 12 up, 12 in 2024-07-24T21:43:10.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:10 smithi086 ceph-mon[37128]: osdmap e79: 12 total, 12 up, 12 in 2024-07-24T21:43:10.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:10 smithi123 ceph-mon[37143]: osdmap e78: 12 total, 12 up, 12 in 2024-07-24T21:43:10.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:10 smithi123 ceph-mon[37143]: osdmap e79: 12 total, 12 up, 12 in 2024-07-24T21:43:10.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 -- 172.21.15.60:0/989827256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608101650 msgr2=0x7fd608101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:10.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 --2- 172.21.15.60:0/989827256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608101650 0x7fd608101a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009900 tx=0x7fd5f802a410 comp rx=0 tx=0).stop 2024-07-24T21:43:10.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 -- 172.21.15.60:0/989827256 shutdown_connections 2024-07-24T21:43:10.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 --2- 172.21.15.60:0/989827256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 0x7fd60810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 --2- 172.21.15.60:0/989827256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608107060 0x7fd6081074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 --2- 172.21.15.60:0/989827256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608101650 0x7fd608101a30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 -- 172.21.15.60:0/989827256 >> 172.21.15.60:0/989827256 conn(0x7fd6080fd0c0 msgr2=0x7fd6080ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 -- 172.21.15.60:0/989827256 shutdown_connections 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.756+0000 7fd60e49a640 1 -- 172.21.15.60:0/989827256 wait complete. 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 Processor -- start 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 -- start start 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608101650 0x7fd6081977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 0x7fd608198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081115f0 con 0x7fd608107060 2024-07-24T21:43:10.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd608111470 con 0x7fd6081079e0 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60e49a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd608111770 con 0x7fd608101650 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46278/0 (socket says 172.21.15.60:46278) 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd6077fe640 1 -- 172.21.15.60:0/1068984251 learned_addr learned my addr 172.21.15.60:0/1068984251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd60ca37640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 0x7fd608198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.757+0000 7fd607fff640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608101650 0x7fd6081977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 -- 172.21.15.60:0/1068984251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608101650 msgr2=0x7fd6081977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608101650 0x7fd6081977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 -- 172.21.15.60:0/1068984251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 msgr2=0x7fd608198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 0x7fd608198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 -- 172.21.15.60:0/1068984251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6081a6fb0 con 0x7fd608107060 2024-07-24T21:43:10.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6077fe640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c920 tx=0x7fd5fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:10.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc007c70 con 0x7fd608107060 2024-07-24T21:43:10.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a72a0 con 0x7fd608107060 2024-07-24T21:43:10.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc007e10 con 0x7fd608107060 2024-07-24T21:43:10.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc018820 con 0x7fd608107060 2024-07-24T21:43:10.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.758+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a77e0 con 0x7fd608107060 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.759+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd608101a30 con 0x7fd608107060 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.761+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fd5fc00f450 con 0x7fd608107060 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.761+0000 7fd6057fa640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 0x7fd5f00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.762+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7712+0+0 (secure 0 0 0) 0x7fd5fc0e3d20 con 0x7fd608107060 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.762+0000 7fd607fff640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 0x7fd5f00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.762+0000 7fd607fff640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 0x7fd5f00b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8031440 tx=0x7fd5f8005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:10.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.764+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd5fc014030 con 0x7fd608107060 2024-07-24T21:43:10.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:10.961+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd60806a880 con 0x7fd608107060 2024-07-24T21:43:11.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.255+0000 7fd6057fa640 1 -- 172.21.15.60:0/1068984251 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fd5fc0ab660 con 0x7fd608107060 2024-07-24T21:43:11.256 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-07-24T21:43:11.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.257+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 msgr2=0x7fd5f00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.257+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 0x7fd5f00b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8031440 tx=0x7fd5f8005b50 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.257+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 msgr2=0x7fd608197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.257+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c920 tx=0x7fd5fc00cde0 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 shutdown_connections 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd5f00b3c20 0x7fd5f00b60e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6081079e0 0x7fd608198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd608107060 0x7fd608197d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 --2- 172.21.15.60:0/1068984251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd608101650 0x7fd6081977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:11.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 >> 172.21.15.60:0/1068984251 conn(0x7fd6080fd0c0 msgr2=0x7fd6080fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:11.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.258+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 shutdown_connections 2024-07-24T21:43:11.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:11.259+0000 7fd60e49a640 1 -- 172.21.15.60:0/1068984251 wait complete. 2024-07-24T21:43:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:11 smithi086 ceph-mon[37128]: pgmap v263: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:11 smithi086 ceph-mon[37128]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-24T21:43:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:11 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-24T21:43:11.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:11 smithi060 ceph-mon[33000]: pgmap v263: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:11.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:11 smithi060 ceph-mon[33000]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-24T21:43:11.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:11 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-24T21:43:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:11 smithi123 ceph-mon[37143]: pgmap v263: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:11 smithi123 ceph-mon[37143]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-24T21:43:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:11 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-24T21:43:11.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-24T21:43:12.192 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:12.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:12 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-24T21:43:12.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:12 smithi086 ceph-mon[37128]: osdmap e80: 12 total, 12 up, 12 in 2024-07-24T21:43:12.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:12 smithi086 ceph-mon[37128]: osdmap e81: 12 total, 12 up, 12 in 2024-07-24T21:43:12.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:12 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-24T21:43:12.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:12 smithi060 ceph-mon[33000]: osdmap e80: 12 total, 12 up, 12 in 2024-07-24T21:43:12.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:12 smithi060 ceph-mon[33000]: osdmap e81: 12 total, 12 up, 12 in 2024-07-24T21:43:12.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:12 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1068984251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-24T21:43:12.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:12 smithi123 ceph-mon[37143]: osdmap e80: 12 total, 12 up, 12 in 2024-07-24T21:43:12.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:12 smithi123 ceph-mon[37143]: osdmap e81: 12 total, 12 up, 12 in 2024-07-24T21:43:13.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:13 smithi086 ceph-mon[37128]: pgmap v266: 97 pgs: 23 creating+peering, 52 unknown, 22 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:13.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:13 smithi086 ceph-mon[37128]: osdmap e82: 12 total, 12 up, 12 in 2024-07-24T21:43:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:13 smithi060 ceph-mon[33000]: pgmap v266: 97 pgs: 23 creating+peering, 52 unknown, 22 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:13 smithi060 ceph-mon[33000]: osdmap e82: 12 total, 12 up, 12 in 2024-07-24T21:43:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:13 smithi123 ceph-mon[37143]: pgmap v266: 97 pgs: 23 creating+peering, 52 unknown, 22 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:13 smithi123 ceph-mon[37143]: osdmap e82: 12 total, 12 up, 12 in 2024-07-24T21:43:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 -- 172.21.15.60:0/467971686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1428101650 msgr2=0x7f1428101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 --2- 172.21.15.60:0/467971686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1428101650 0x7f1428101a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 -- 172.21.15.60:0/467971686 shutdown_connections 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 --2- 172.21.15.60:0/467971686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14281079e0 0x7f142810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 --2- 172.21.15.60:0/467971686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 0x7f14281074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 --2- 172.21.15.60:0/467971686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1428101650 0x7f1428101a30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 -- 172.21.15.60:0/467971686 >> 172.21.15.60:0/467971686 conn(0x7f14280fd0c0 msgr2=0x7f14280ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 -- 172.21.15.60:0/467971686 shutdown_connections 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.834+0000 7f142d629640 1 -- 172.21.15.60:0/467971686 wait complete. 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 Processor -- start 2024-07-24T21:43:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 -- start start 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 0x7f1428198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 0x7f14281992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f14281127e0 con 0x7f14281079e0 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1428112660 con 0x7f1428101650 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f142d629640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1428112960 con 0x7f1428107060 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f14277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f14277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46292/0 (socket says 172.21.15.60:46292) 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f14277fe640 1 -- 172.21.15.60:0/2146857475 learned_addr learned my addr 172.21.15.60:0/2146857475 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f1426ffd640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 0x7f1428198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.835+0000 7f14267fc640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 0x7f14281992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 -- 172.21.15.60:0/2146857475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 msgr2=0x7f14281992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 0x7f14281992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 -- 172.21.15.60:0/2146857475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 msgr2=0x7f1428198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 0x7f1428198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 -- 172.21.15.60:0/2146857475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142819ee40 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f1426ffd640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 0x7f1428198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f14277fe640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f141800b970 tx=0x7f141800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418019070 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1418014440 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142819f130 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180075d0 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.836+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281a5870 con 0x7f14281079e0 2024-07-24T21:43:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.837+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1428101ab0 con 0x7f14281079e0 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.839+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f141801c070 con 0x7f14281079e0 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.839+0000 7f141ffff640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 0x7f14140b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.839+0000 7f1426ffd640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 0x7f14140b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.840+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8094+0+0 (secure 0 0 0) 0x7f14180e3650 con 0x7f14281079e0 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.840+0000 7f1426ffd640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 0x7f14140b6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1420031440 tx=0x7f14200095f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:13.842+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f14180aad80 con 0x7f14281079e0 2024-07-24T21:43:14.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.078+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1428199ec0 con 0x7f14281079e0 2024-07-24T21:43:14.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.288+0000 7f141ffff640 1 -- 172.21.15.60:0/2146857475 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1428199ec0 con 0x7f14281079e0 2024-07-24T21:43:14.290 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 msgr2=0x7f14140b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 0x7f14140b6160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1420031440 tx=0x7f14200095f0 comp rx=0 tx=0).stop 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 msgr2=0x7f142819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f141800b970 tx=0x7f141800be30 comp rx=0 tx=0).stop 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 shutdown_connections 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f14140b3ca0 0x7f14140b6160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14281079e0 0x7f142819e670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1428107060 0x7f14281992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 --2- 172.21.15.60:0/2146857475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428101650 0x7f1428198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 >> 172.21.15.60:0/2146857475 conn(0x7f14280fd0c0 msgr2=0x7f14280ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 shutdown_connections 2024-07-24T21:43:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:14.291+0000 7f142d629640 1 -- 172.21.15.60:0/2146857475 wait complete. 2024-07-24T21:43:14.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:14 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-24T21:43:14.332 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:14 smithi060 ceph-mon[33000]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-24T21:43:14.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:14 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-24T21:43:14.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:14 smithi123 ceph-mon[37143]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-24T21:43:14.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:14 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-24T21:43:14.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:14 smithi086 ceph-mon[37128]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-24T21:43:14.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:43:15.215 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:15.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:15 smithi086 ceph-mon[37128]: pgmap v269: 97 pgs: 18 creating+peering, 33 unknown, 46 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:15 smithi086 ceph-mon[37128]: osdmap e83: 12 total, 12 up, 12 in 2024-07-24T21:43:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:15 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-24T21:43:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:15 smithi086 ceph-mon[37128]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-24T21:43:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:15 smithi086 ceph-mon[37128]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-24T21:43:15.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:15 smithi060 ceph-mon[33000]: pgmap v269: 97 pgs: 18 creating+peering, 33 unknown, 46 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:15.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:15 smithi060 ceph-mon[33000]: osdmap e83: 12 total, 12 up, 12 in 2024-07-24T21:43:15.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:15 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-24T21:43:15.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:15 smithi060 ceph-mon[33000]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-24T21:43:15.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:15 smithi060 ceph-mon[33000]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-24T21:43:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:15 smithi123 ceph-mon[37143]: pgmap v269: 97 pgs: 18 creating+peering, 33 unknown, 46 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T21:43:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:15 smithi123 ceph-mon[37143]: osdmap e83: 12 total, 12 up, 12 in 2024-07-24T21:43:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:15 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2146857475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-24T21:43:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:15 smithi123 ceph-mon[37143]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-24T21:43:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:15 smithi123 ceph-mon[37143]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-24T21:43:16.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:16 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:active 2024-07-24T21:43:16.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:16 smithi086 ceph-mon[37128]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:16 smithi086 ceph-mon[37128]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.550 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:16 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:active 2024-07-24T21:43:16.550 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:16 smithi060 ceph-mon[33000]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.551 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:16 smithi060 ceph-mon[33000]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:16 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] up:active 2024-07-24T21:43:16.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:16 smithi123 ceph-mon[37143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:16 smithi123 ceph-mon[37143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 -- 172.21.15.60:0/3509671031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 msgr2=0x7f69ac1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 --2- 172.21.15.60:0/3509671031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1111d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009ca0 tx=0x7f69a402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 -- 172.21.15.60:0/3509671031 shutdown_connections 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 --2- 172.21.15.60:0/3509671031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1111d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 --2- 172.21.15.60:0/3509671031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 0x7f69ac10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 --2- 172.21.15.60:0/3509671031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 -- 172.21.15.60:0/3509671031 >> 172.21.15.60:0/3509671031 conn(0x7f69ac0fd100 msgr2=0x7f69ac0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.897+0000 7f69b1569640 1 -- 172.21.15.60:0/3509671031 shutdown_connections 2024-07-24T21:43:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.898+0000 7f69b1569640 1 -- 172.21.15.60:0/3509671031 wait complete. 2024-07-24T21:43:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.898+0000 7f69b1569640 1 Processor -- start 2024-07-24T21:43:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.898+0000 7f69b1569640 1 -- start start 2024-07-24T21:43:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 0x7f69ac19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac114960 con 0x7f69ac10ede0 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac1147e0 con 0x7f69ac101230 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69b1569640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac114ae0 con 0x7f69ac101b50 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40882/0 (socket says 172.21.15.60:40882) 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 -- 172.21.15.60:0/3049171578 learned_addr learned my addr 172.21.15.60:0/3049171578 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69aa7fc640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 0x7f69ac19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69aaffd640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 -- 172.21.15.60:0/3049171578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 msgr2=0x7f69ac19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 0x7f69ac19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 -- 172.21.15.60:0/3049171578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 msgr2=0x7f69ac19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 -- 172.21.15.60:0/3049171578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac1a0e40 con 0x7f69ac10ede0 2024-07-24T21:43:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69aaffd640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69a4031a20 tx=0x7f69a4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.899+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4003cf0 con 0x7f69ac10ede0 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.900+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a40044f0 con 0x7f69ac10ede0 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.900+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1a10d0 con 0x7f69ac10ede0 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.900+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a402fce0 con 0x7f69ac10ede0 2024-07-24T21:43:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.900+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1a77f0 con 0x7f69ac10ede0 2024-07-24T21:43:16.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.901+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac101610 con 0x7f69ac10ede0 2024-07-24T21:43:16.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.902+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f69a4004040 con 0x7f69ac10ede0 2024-07-24T21:43:16.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.903+0000 7f6993fff640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 0x7f69940b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:16.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.903+0000 7f69aaffd640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 0x7f69940b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:16.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.903+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7f69a4105430 con 0x7f69ac10ede0 2024-07-24T21:43:16.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.903+0000 7f69aaffd640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 0x7f69940b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f699c00abe0 tx=0x7f699c00a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:16.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:16.906+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f69a40ccaa0 con 0x7f69ac10ede0 2024-07-24T21:43:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.105+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69ac1a7ab0 con 0x7f69ac10ede0 2024-07-24T21:43:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.106+0000 7f6993fff640 1 -- 172.21.15.60:0/3049171578 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22480 (secure 0 0 0) 0x7f69a40cc3a0 con 0x7f69ac10ede0 2024-07-24T21:43:17.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:17.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:43:14.079522+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 msgr2=0x7f69940b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 0x7f69940b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f699c00abe0 tx=0x7f699c00a5c0 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 msgr2=0x7f69ac1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69a4031a20 tx=0x7f69a4031a50 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 shutdown_connections 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69940b3c20 0x7f69940b60e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac10ede0 0x7f69ac1a0670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ac101b50 0x7f69ac19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 --2- 172.21.15.60:0/3049171578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69ac101230 0x7f69ac19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 >> 172.21.15.60:0/3049171578 conn(0x7f69ac0fd100 msgr2=0x7f69ac110f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 shutdown_connections 2024-07-24T21:43:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:17.109+0000 7f69b1569640 1 -- 172.21.15.60:0/3049171578 wait complete. 2024-07-24T21:43:17.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:17 smithi060 ceph-mon[33000]: pgmap v271: 97 pgs: 18 creating+peering, 4 unknown, 75 active+clean; 577 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-07-24T21:43:17.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:17 smithi060 ceph-mon[33000]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-24T21:43:17.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:17 smithi060 ceph-mon[33000]: Cluster is now healthy 2024-07-24T21:43:17.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:17 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3049171578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:17 smithi086 ceph-mon[37128]: pgmap v271: 97 pgs: 18 creating+peering, 4 unknown, 75 active+clean; 577 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-07-24T21:43:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:17 smithi086 ceph-mon[37128]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-24T21:43:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:17 smithi086 ceph-mon[37128]: Cluster is now healthy 2024-07-24T21:43:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3049171578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:17 smithi123 ceph-mon[37143]: pgmap v271: 97 pgs: 18 creating+peering, 4 unknown, 75 active+clean; 577 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-07-24T21:43:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:17 smithi123 ceph-mon[37143]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-24T21:43:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:17 smithi123 ceph-mon[37143]: Cluster is now healthy 2024-07-24T21:43:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:17 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3049171578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:17.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:43:18.056 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:19.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:19 smithi060 ceph-mon[33000]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.668+0000 7fc747811640 1 -- 172.21.15.60:0/910458916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 msgr2=0x7fc7401085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.668+0000 7fc747811640 1 --2- 172.21.15.60:0/910458916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc7401085e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc734009900 tx=0x7fc73402a410 comp rx=0 tx=0).stop 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 -- 172.21.15.60:0/910458916 shutdown_connections 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 --2- 172.21.15.60:0/910458916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 0x7fc74010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 --2- 172.21.15.60:0/910458916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc7401085e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 --2- 172.21.15.60:0/910458916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 0x7fc740102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 -- 172.21.15.60:0/910458916 >> 172.21.15.60:0/910458916 conn(0x7fc7400fe200 msgr2=0x7fc740100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 -- 172.21.15.60:0/910458916 shutdown_connections 2024-07-24T21:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 -- 172.21.15.60:0/910458916 wait complete. 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 Processor -- start 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 -- start start 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 0x7fc740198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.669+0000 7fc747811640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc747811640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 0x7fc74019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc747811640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc740112870 con 0x7fc7401081a0 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc747811640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401126f0 con 0x7fc740102790 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc747811640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401129f0 con 0x7fc740108b20 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40908/0 (socket says 172.21.15.60:40908) 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 -- 172.21.15.60:0/28921175 learned_addr learned my addr 172.21.15.60:0/28921175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc7455ad640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 0x7fc740198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc745dae640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 0x7fc74019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 -- 172.21.15.60:0/28921175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 msgr2=0x7fc74019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 0x7fc74019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 -- 172.21.15.60:0/28921175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 msgr2=0x7fc740198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 0x7fc740198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 -- 172.21.15.60:0/28921175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74019eeb0 con 0x7fc7401081a0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc744dac640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc734005ab0 tx=0x7fc734003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73403f070 con 0x7fc7401081a0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc734031d60 con 0x7fc7401081a0 2024-07-24T21:43:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734035030 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.670+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74019f140 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.671+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401a5860 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.672+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc740102bf0 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.674+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc734042070 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.674+0000 7fc73e7fc640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 0x7fc7280b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.675+0000 7fc7455ad640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 0x7fc7280b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.675+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7fc734104cc0 con 0x7fc7401081a0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.675+0000 7fc7455ad640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 0x7fc7280b60e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc730009990 tx=0x7fc730009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.678+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc7340cc330 con 0x7fc7401081a0 2024-07-24T21:43:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:19 smithi123 ceph-mon[37143]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-24T21:43:19.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:19 smithi086 ceph-mon[37128]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-24T21:43:19.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.876+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc740199f30 con 0x7fc7401081a0 2024-07-24T21:43:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.878+0000 7fc73e7fc640 1 -- 172.21.15.60:0/28921175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22480 (secure 0 0 0) 0x7fc7340cbc30 con 0x7fc7401081a0 2024-07-24T21:43:19.879 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:19.879 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:43:14.079522+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:43:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 msgr2=0x7fc7280b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 0x7fc7280b60e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc730009990 tx=0x7fc730009110 comp rx=0 tx=0).stop 2024-07-24T21:43:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 msgr2=0x7fc740199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc734005ab0 tx=0x7fc734003d50 comp rx=0 tx=0).stop 2024-07-24T21:43:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 shutdown_connections 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7280b3c20 0x7fc7280b60e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc740108b20 0x7fc74019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7401081a0 0x7fc740199340 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 --2- 172.21.15.60:0/28921175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc740102790 0x7fc740198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 >> 172.21.15.60:0/28921175 conn(0x7fc7400fe200 msgr2=0x7fc7401005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 shutdown_connections 2024-07-24T21:43:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:19.880+0000 7fc747811640 1 -- 172.21.15.60:0/28921175 wait complete. 2024-07-24T21:43:20.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/28921175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:20.493 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:43:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:43:20.108+0000 7f827c594640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-24T21:43:20.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/28921175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:20.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-24T21:43:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/28921175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:21.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:21 smithi060 ceph-mon[33000]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:21 smithi123 ceph-mon[37143]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:21.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:21 smithi086 ceph-mon[37128]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:22.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- 172.21.15.60:0/1783380424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8101650 msgr2=0x7f08c8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 --2- 172.21.15.60:0/1783380424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8101650 0x7f08c8101a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009900 tx=0x7f08b802a410 comp rx=0 tx=0).stop 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- 172.21.15.60:0/1783380424 shutdown_connections 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 --2- 172.21.15.60:0/1783380424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c81079e0 0x7f08c810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 --2- 172.21.15.60:0/1783380424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c8107060 0x7f08c81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 --2- 172.21.15.60:0/1783380424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8101650 0x7f08c8101a30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- 172.21.15.60:0/1783380424 >> 172.21.15.60:0/1783380424 conn(0x7f08c80fd0c0 msgr2=0x7f08c80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- 172.21.15.60:0/1783380424 shutdown_connections 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- 172.21.15.60:0/1783380424 wait complete. 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 Processor -- start 2024-07-24T21:43:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.495+0000 7f08ce621640 1 -- start start 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 0x7f08c8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c81079e0 0x7f08c819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8111610 con 0x7f08c8107060 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8111490 con 0x7f08c8101650 2024-07-24T21:43:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ce621640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8111790 con 0x7f08c81079e0 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40928/0 (socket says 172.21.15.60:40928) 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 -- 172.21.15.60:0/3053186250 learned_addr learned my addr 172.21.15.60:0/3053186250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08ccbbe640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c81079e0 0x7f08c819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c7fff640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 0x7f08c8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 -- 172.21.15.60:0/3053186250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c81079e0 msgr2=0x7f08c819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c81079e0 0x7f08c819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 -- 172.21.15.60:0/3053186250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 msgr2=0x7f08c8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 0x7f08c8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c77fe640 1 -- 172.21.15.60:0/3053186250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c819dbd0 con 0x7f08c8107060 2024-07-24T21:43:22.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.496+0000 7f08c7fff640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 0x7f08c8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08c77fe640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009860 tx=0x7f08bc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc005020 con 0x7f08c8107060 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c819dec0 con 0x7f08c8107060 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08bc0047a0 con 0x7f08c8107060 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc007b00 con 0x7f08c8107060 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.497+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c806a470 con 0x7f08c8107060 2024-07-24T21:43:22.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.498+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c8101a30 con 0x7f08c8107060 2024-07-24T21:43:22.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.499+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f08bc004950 con 0x7f08c8107060 2024-07-24T21:43:22.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.500+0000 7f08c57fa640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 0x7f08b00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:22.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.500+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7f08bc0e5980 con 0x7f08c8107060 2024-07-24T21:43:22.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.500+0000 7f08c7fff640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 0x7f08b00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:22.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.501+0000 7f08c7fff640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 0x7f08b00b6160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.503+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f08bc0acff0 con 0x7f08c8107060 2024-07-24T21:43:22.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:22.741+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f08c806afa0 con 0x7f08c8107060 2024-07-24T21:43:23.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.327+0000 7f08c57fa640 1 -- 172.21.15.60:0/3053186250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f08bc0ac8f0 con 0x7f08c8107060 2024-07-24T21:43:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 msgr2=0x7f08b00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 0x7f08b00b6160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8031400 comp rx=0 tx=0).stop 2024-07-24T21:43:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 msgr2=0x7f08c8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009860 tx=0x7f08bc009d20 comp rx=0 tx=0).stop 2024-07-24T21:43:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 shutdown_connections 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f08b00b3ca0 0x7f08b00b6160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08c81079e0 0x7f08c819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08c8107060 0x7f08c8198060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 --2- 172.21.15.60:0/3053186250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c8101650 0x7f08c8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 >> 172.21.15.60:0/3053186250 conn(0x7f08c80fd0c0 msgr2=0x7f08c80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 shutdown_connections 2024-07-24T21:43:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:23.330+0000 7f08ce621640 1 -- 172.21.15.60:0/3053186250 wait complete. 2024-07-24T21:43:23.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:23 smithi060 ceph-mon[33000]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-07-24T21:43:23.712 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:23 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-24T21:43:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:23 smithi123 ceph-mon[37143]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-07-24T21:43:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:23 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-24T21:43:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:23 smithi086 ceph-mon[37128]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-07-24T21:43:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:23 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-24T21:43:24.057 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-07-24T21:43:24.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-07-24T21:43:24.330 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:24.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:24 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-24T21:43:24.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:24 smithi060 ceph-mon[33000]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:24.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:24 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-24T21:43:24.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:24 smithi123 ceph-mon[37143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:24.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:24 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3053186250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-24T21:43:24.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:24 smithi086 ceph-mon[37128]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:25.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:25 smithi060 ceph-mon[33000]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-24T21:43:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:25 smithi123 ceph-mon[37143]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-24T21:43:25.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:25 smithi086 ceph-mon[37128]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-24T21:43:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 -- 172.21.15.60:0/1043157728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04102770 msgr2=0x7ffa04102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/1043157728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04102770 0x7ffa04102b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009900 tx=0x7ff9f402a410 comp rx=0 tx=0).stop 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 -- 172.21.15.60:0/1043157728 shutdown_connections 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/1043157728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108b00 0x7ffa0410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/1043157728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04108180 0x7ffa041085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/1043157728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04102770 0x7ffa04102b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 -- 172.21.15.60:0/1043157728 >> 172.21.15.60:0/1043157728 conn(0x7ffa040fe1e0 msgr2=0x7ffa04100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 -- 172.21.15.60:0/1043157728 shutdown_connections 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.983+0000 7ffa0a39c640 1 -- 172.21.15.60:0/1043157728 wait complete. 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 Processor -- start 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 -- start start 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102770 0x7ffa04198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108180 0x7ffa04199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04112730 con 0x7ffa04108b00 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041125b0 con 0x7ffa04108180 2024-07-24T21:43:26.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa0a39c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041128b0 con 0x7ffa04102770 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37372/0 (socket says 172.21.15.60:37372) 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa08939640 1 -- 172.21.15.60:0/3264675557 learned_addr learned my addr 172.21.15.60:0/3264675557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa03fff640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102770 0x7ffa04198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.984+0000 7ffa037fe640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108180 0x7ffa04199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 -- 172.21.15.60:0/3264675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102770 msgr2=0x7ffa04198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102770 0x7ffa04198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 -- 172.21.15.60:0/3264675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108180 msgr2=0x7ffa04199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108180 0x7ffa04199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 -- 172.21.15.60:0/3264675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0419ecd0 con 0x7ffa04108b00 2024-07-24T21:43:26.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa08939640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00eaa0 tx=0x7ff9fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00cdb0 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0419efc0 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc005020 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0116d0 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.985+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041a5680 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.986+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa04102b50 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.987+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7ff9fc0051d0 con 0x7ffa04108b00 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.988+0000 7ffa017fa640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 0x7ff9ec0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.988+0000 7ffa03fff640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 0x7ff9ec0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:26.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.988+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7ff9fc0e4260 con 0x7ffa04108b00 2024-07-24T21:43:26.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.989+0000 7ffa03fff640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 0x7ff9ec0b60e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4031440 tx=0x7ff9f40095f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:26.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:25.991+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff9fc016030 con 0x7ffa04108b00 2024-07-24T21:43:26.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.226+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffa0410a350 con 0x7ffa04108b00 2024-07-24T21:43:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.346+0000 7ffa017fa640 1 -- 172.21.15.60:0/3264675557 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff9fc0ab8d0 con 0x7ffa04108b00 2024-07-24T21:43:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.348+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 msgr2=0x7ff9ec0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 0x7ff9ec0b60e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4031440 tx=0x7ff9f40095f0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 msgr2=0x7ffa0419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00eaa0 tx=0x7ff9fc00ef60 comp rx=0 tx=0).stop 2024-07-24T21:43:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 shutdown_connections 2024-07-24T21:43:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9ec0b3c20 0x7ff9ec0b60e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa04108b00 0x7ffa0419e500 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa04108180 0x7ffa04199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 --2- 172.21.15.60:0/3264675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa04102770 0x7ffa04198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 >> 172.21.15.60:0/3264675557 conn(0x7ffa040fe1e0 msgr2=0x7ffa041005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 shutdown_connections 2024-07-24T21:43:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:26.349+0000 7ffa0a39c640 1 -- 172.21.15.60:0/3264675557 wait complete. 2024-07-24T21:43:26.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:26 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-24T21:43:26.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:26 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-24T21:43:26.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:26 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-24T21:43:27.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-24T21:43:27.272 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-24T21:43:27.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:27 smithi060 ceph-mon[33000]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-24T21:43:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-07-24T21:43:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-24T21:43:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-24T21:43:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-24T21:43:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:27.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-24T21:43:27.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:27 smithi123 ceph-mon[37143]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-24T21:43:27.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3264675557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-24T21:43:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:27 smithi086 ceph-mon[37128]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:active 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: Cluster is now healthy 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-24T21:43:28.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:28 smithi060 ceph-mon[33000]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-24T21:43:28.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:active 2024-07-24T21:43:28.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-24T21:43:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-24T21:43:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: Cluster is now healthy 2024-07-24T21:43:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-24T21:43:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-24T21:43:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:28 smithi123 ceph-mon[37143]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] up:active 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: Cluster is now healthy 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-24T21:43:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:28 smithi086 ceph-mon[37128]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 -- 172.21.15.60:0/956976240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 msgr2=0x7f2d0810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/956976240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d0810c4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009900 tx=0x7f2cfc02a410 comp rx=0 tx=0).stop 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 -- 172.21.15.60:0/956976240 shutdown_connections 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/956976240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/956976240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d0810c4b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/956976240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 0x7f2d08103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.906+0000 7f2d0cc15640 1 -- 172.21.15.60:0/956976240 >> 172.21.15.60:0/956976240 conn(0x7f2d080fd100 msgr2=0x7f2d080ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.907+0000 7f2d0cc15640 1 -- 172.21.15.60:0/956976240 shutdown_connections 2024-07-24T21:43:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.907+0000 7f2d0cc15640 1 -- 172.21.15.60:0/956976240 wait complete. 2024-07-24T21:43:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.907+0000 7f2d0cc15640 1 Processor -- start 2024-07-24T21:43:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.907+0000 7f2d0cc15640 1 -- start start 2024-07-24T21:43:28.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 0x7f2d08198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:28.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081127b0 con 0x7f2d08103fc0 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08112630 con 0x7f2d0810cb00 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08112930 con 0x7f2d081036a0 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37388/0 (socket says 172.21.15.60:37388) 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 -- 172.21.15.60:0/1244932644 learned_addr learned my addr 172.21.15.60:0/1244932644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d06d9d640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0659c640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 0x7f2d08198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 -- 172.21.15.60:0/1244932644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 msgr2=0x7f2d08198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 0x7f2d08198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 -- 172.21.15.60:0/1244932644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 msgr2=0x7f2d0819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 -- 172.21.15.60:0/1244932644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0819ee10 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d06d9d640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d05d9b640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc03f030 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc003aa0 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0819f0a0 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.908+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc035070 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.909+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d081a57c0 con 0x7f2d08103fc0 2024-07-24T21:43:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.910+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d08103a80 con 0x7f2d08103fc0 2024-07-24T21:43:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.911+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f2cfc042030 con 0x7f2d08103fc0 2024-07-24T21:43:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.912+0000 7f2cf77fe640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 0x7f2cf00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.912+0000 7f2d0659c640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 0x7f2cf00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.912+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7f2cfc104160 con 0x7f2d08103fc0 2024-07-24T21:43:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.912+0000 7f2d0659c640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 0x7f2cf00b60e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009890 tx=0x7f2cf8008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:28.916+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2cfc0cb7d0 con 0x7f2d08103fc0 2024-07-24T21:43:29.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.150+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2d08107b70 con 0x7f2d08103fc0 2024-07-24T21:43:29.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.367+0000 7f2cf77fe640 1 -- 172.21.15.60:0/1244932644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f2cfc0cb0d0 con 0x7f2d08103fc0 2024-07-24T21:43:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 msgr2=0x7f2cf00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 0x7f2cf00b60e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009890 tx=0x7f2cf8008040 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 msgr2=0x7f2d081992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc004060 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 shutdown_connections 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cf00b3c20 0x7f2cf00b60e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0810cb00 0x7f2d0819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d08103fc0 0x7f2d081992a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 --2- 172.21.15.60:0/1244932644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d081036a0 0x7f2d08198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 >> 172.21.15.60:0/1244932644 conn(0x7f2d080fd100 msgr2=0x7f2d080fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 shutdown_connections 2024-07-24T21:43:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:29.370+0000 7f2d0cc15640 1 -- 172.21.15.60:0/1244932644 wait complete. 2024-07-24T21:43:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:29 smithi060 ceph-mon[33000]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-07-24T21:43:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:29 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:active 2024-07-24T21:43:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:29 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:29.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-24T21:43:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:29 smithi123 ceph-mon[37143]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-07-24T21:43:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:29 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:active 2024-07-24T21:43:29.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:29 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:29.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-24T21:43:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:29 smithi086 ceph-mon[37128]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-07-24T21:43:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:29 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] up:active 2024-07-24T21:43:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:29 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-24T21:43:30.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-24T21:43:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:30.371 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:43:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:43:30.109+0000 7f827c594640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-24T21:43:30.371 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:43:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:43:30.109+0000 7f827c594640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-24T21:43:30.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:30 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-24T21:43:30.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:30 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:30.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:30 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:30.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:30 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-24T21:43:30.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:30 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:30.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:30 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:30.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:30 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1244932644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-24T21:43:30.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:30 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-24T21:43:30.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:30 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:31.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:31 smithi060 ceph-mon[33000]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:31 smithi123 ceph-mon[37143]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:31.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:31 smithi086 ceph-mon[37128]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-24T21:43:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 -- 172.21.15.60:0/1426845815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 msgr2=0x7f9770101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 --2- 172.21.15.60:0/1426845815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770101a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9760009900 tx=0x7f976002a410 comp rx=0 tx=0).stop 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 -- 172.21.15.60:0/1426845815 shutdown_connections 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 --2- 172.21.15.60:0/1426845815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 0x7f977010e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 --2- 172.21.15.60:0/1426845815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f97701074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 --2- 172.21.15.60:0/1426845815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770101a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 -- 172.21.15.60:0/1426845815 >> 172.21.15.60:0/1426845815 conn(0x7f97700fd0d0 msgr2=0x7f97700ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 -- 172.21.15.60:0/1426845815 shutdown_connections 2024-07-24T21:43:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.946+0000 7f97759cf640 1 -- 172.21.15.60:0/1426845815 wait complete. 2024-07-24T21:43:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 Processor -- start 2024-07-24T21:43:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 -- start start 2024-07-24T21:43:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f9770198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 0x7f977019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9770111620 con 0x7f9770101680 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97701114a0 con 0x7f9770107a10 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f97759cf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97701117a0 con 0x7f9770107090 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f976effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f976effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37406/0 (socket says 172.21.15.60:37406) 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f976effd640 1 -- 172.21.15.60:0/450234019 learned_addr learned my addr 172.21.15.60:0/450234019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f976f7fe640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 0x7f977019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.947+0000 7f9767fff640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f9770198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 -- 172.21.15.60:0/450234019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 msgr2=0x7f9770198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f9770198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 -- 172.21.15.60:0/450234019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 msgr2=0x7f977019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 0x7f977019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 -- 172.21.15.60:0/450234019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977019dee0 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f9767fff640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f9770198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f976effd640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9760005940 tx=0x7f97600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976003f030 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977019e170 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9760003cb0 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760035070 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.948+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977006ae70 con 0x7f9770101680 2024-07-24T21:43:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.949+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770101a60 con 0x7f9770101680 2024-07-24T21:43:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.950+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f9760042030 con 0x7f9770101680 2024-07-24T21:43:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.951+0000 7f97749cd640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 0x7f975c0b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.952+0000 7f9767fff640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 0x7f975c0b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.952+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7f9760104810 con 0x7f9770101680 2024-07-24T21:43:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.952+0000 7f9767fff640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 0x7f975c0b82f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9770106c30 tx=0x7f9758005d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:31.955+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f97600cbe20 con 0x7f9770101680 2024-07-24T21:43:32.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.189+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f97700008d0 con 0x7f9770101680 2024-07-24T21:43:32.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.382+0000 7f97749cd640 1 -- 172.21.15.60:0/450234019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f97600cb720 con 0x7f9770101680 2024-07-24T21:43:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 msgr2=0x7f975c0b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 0x7f975c0b82f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9770106c30 tx=0x7f9758005d90 comp rx=0 tx=0).stop 2024-07-24T21:43:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 msgr2=0x7f9770197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9760005940 tx=0x7f97600042f0 comp rx=0 tx=0).stop 2024-07-24T21:43:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 shutdown_connections 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f975c0b5e30 0x7f975c0b82f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9770107a10 0x7f977019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770107090 0x7f9770198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 --2- 172.21.15.60:0/450234019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9770101680 0x7f9770197e30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 >> 172.21.15.60:0/450234019 conn(0x7f97700fd0d0 msgr2=0x7f97700fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 shutdown_connections 2024-07-24T21:43:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:32.386+0000 7f97759cf640 1 -- 172.21.15.60:0/450234019 wait complete. 2024-07-24T21:43:32.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:32 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-24T21:43:32.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:32 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-24T21:43:32.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:32 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-24T21:43:33.031 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-07-24T21:43:33.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-24T21:43:33.303 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:33.643 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:33 smithi123 ceph-mon[37143]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-07-24T21:43:33.644 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:33 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-24T21:43:33.644 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:33 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:33.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:33 smithi060 ceph-mon[33000]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-07-24T21:43:33.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:33 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-24T21:43:33.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:33 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:33.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:33 smithi086 ceph-mon[37128]: pgmap v279: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-07-24T21:43:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:33 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/450234019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-24T21:43:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:33 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 -- 172.21.15.60:0/3807871268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107080 msgr2=0x7f69101074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 --2- 172.21.15.60:0/3807871268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107080 0x7f69101074c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6900009900 tx=0x7f690002a410 comp rx=0 tx=0).stop 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 -- 172.21.15.60:0/3807871268 shutdown_connections 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 --2- 172.21.15.60:0/3807871268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910107a00 0x7f691010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 --2- 172.21.15.60:0/3807871268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107080 0x7f69101074c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 --2- 172.21.15.60:0/3807871268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910101670 0x7f6910101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.935+0000 7f69174db640 1 -- 172.21.15.60:0/3807871268 >> 172.21.15.60:0/3807871268 conn(0x7f69100fd0e0 msgr2=0x7f69100ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- 172.21.15.60:0/3807871268 shutdown_connections 2024-07-24T21:43:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- 172.21.15.60:0/3807871268 wait complete. 2024-07-24T21:43:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 Processor -- start 2024-07-24T21:43:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- start start 2024-07-24T21:43:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910101670 0x7f6910198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910107080 0x7f69101991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111600 con 0x7f6910107a00 2024-07-24T21:43:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111480 con 0x7f6910101670 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.936+0000 7f69174db640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111780 con 0x7f6910107080 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37422/0 (socket says 172.21.15.60:37422) 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915277640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910101670 0x7f6910198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 -- 172.21.15.60:0/2769195504 learned_addr learned my addr 172.21.15.60:0/2769195504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6914a76640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910107080 0x7f69101991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 -- 172.21.15.60:0/2769195504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910107080 msgr2=0x7f69101991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910107080 0x7f69101991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 -- 172.21.15.60:0/2769195504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910101670 msgr2=0x7f6910198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910101670 0x7f6910198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 -- 172.21.15.60:0/2769195504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691019ed20 con 0x7f6910107a00 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f6915a78640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f690800eaa0 tx=0x7f690800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690800cdb0 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6908005020 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691019f010 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080116d0 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.937+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69101a56d0 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.938+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910101a50 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.940+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f69080051d0 con 0x7f6910107a00 2024-07-24T21:43:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.940+0000 7f69067fc640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 0x7f68fc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.941+0000 7f6915277640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 0x7f68fc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.941+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8194+0+0 (secure 0 0 0) 0x7f69080e4100 con 0x7f6910107a00 2024-07-24T21:43:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.941+0000 7f6915277640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 0x7f68fc0b60e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f690c00bbe0 tx=0x7f690c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:34.943+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6908016030 con 0x7f6910107a00 2024-07-24T21:43:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.165+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f69100ffcb0 con 0x7f68fc0b3c20 2024-07-24T21:43:35.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.189+0000 7f69067fc640 1 -- 172.21.15.60:0/2769195504 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69100ffcb0 con 0x7f68fc0b3c20 2024-07-24T21:43:35.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 msgr2=0x7f68fc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 0x7f68fc0b60e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f690c00bbe0 tx=0x7f690c00b390 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 msgr2=0x7f691019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f690800eaa0 tx=0x7f690800ef60 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 shutdown_connections 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68fc0b3c20 0x7f68fc0b60e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6910107a00 0x7f691019e550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6910107080 0x7f69101991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 --2- 172.21.15.60:0/2769195504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6910101670 0x7f6910198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 >> 172.21.15.60:0/2769195504 conn(0x7f69100fd0e0 msgr2=0x7f691006b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 shutdown_connections 2024-07-24T21:43:35.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:35.191+0000 7f69174db640 1 -- 172.21.15.60:0/2769195504 wait complete. 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-mon[33000]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:43:35.170+0000 7f88fcd22640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:43:35.170+0000 7f88fcd22640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:43:35.170+0000 7f88fcd22640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:43:35.170+0000 7f88fcd22640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:43:35.552 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:43:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:43:35.170+0000 7f88fcd22640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:43:35.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:35 smithi123 ceph-mon[37143]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:35.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:35 smithi086 ceph-mon[37128]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:35.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-24T21:43:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-24T21:43:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-24T21:43:36.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-24T21:43:36.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-24T21:43:36.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:36.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-24T21:43:36.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-24T21:43:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-24T21:43:36.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-07-24T21:43:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-24T21:43:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-24T21:43:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-24T21:43:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: osdmap e84: 12 total, 12 up, 12 in 2024-07-24T21:43:37.529 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:37 smithi060 ceph-mon[33000]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-24T21:43:37.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:37.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-24T21:43:37.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: osdmap e84: 12 total, 12 up, 12 in 2024-07-24T21:43:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:37 smithi086 ceph-mon[37128]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-24T21:43:37.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-24T21:43:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: osdmap e84: 12 total, 12 up, 12 in 2024-07-24T21:43:37.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:37 smithi123 ceph-mon[37143]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-07-24T21:43:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 -- 172.21.15.60:0/1870678892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b41079e0 msgr2=0x7fc9b410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1870678892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b41079e0 0x7fc9b410e1e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009ca0 tx=0x7fc9ac02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 -- 172.21.15.60:0/1870678892 shutdown_connections 2024-07-24T21:43:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1870678892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b41079e0 0x7fc9b410e1e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1870678892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4107060 0x7fc9b41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1870678892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b4101650 0x7fc9b4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 -- 172.21.15.60:0/1870678892 >> 172.21.15.60:0/1870678892 conn(0x7fc9b40fd0c0 msgr2=0x7fc9b40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 -- 172.21.15.60:0/1870678892 shutdown_connections 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.751+0000 7fc9bc268640 1 -- 172.21.15.60:0/1870678892 wait complete. 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 Processor -- start 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 -- start start 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4101650 0x7fc9b4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 0x7fc9b419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4111670 con 0x7fc9b4107060 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.752+0000 7fc9bc268640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41114f0 con 0x7fc9b4101650 2024-07-24T21:43:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9bc268640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41117f0 con 0x7fc9b41079e0 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52588/0 (socket says 172.21.15.60:52588) 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 -- 172.21.15.60:0/1339751731 learned_addr learned my addr 172.21.15.60:0/1339751731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ba004640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4101650 0x7fc9b4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ba805640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 0x7fc9b419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 -- 172.21.15.60:0/1339751731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 msgr2=0x7fc9b419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 0x7fc9b419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 -- 172.21.15.60:0/1339751731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4101650 msgr2=0x7fc9b4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4101650 0x7fc9b4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 -- 172.21.15.60:0/1339751731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b419db50 con 0x7fc9b4107060 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ba805640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 0x7fc9b419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9b9803640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c920 tx=0x7fc9a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007c70 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a4007e10 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b419de40 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4018820 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.753+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41a4580 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.754+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b4101a30 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.757+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fc9a400f450 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.757+0000 7fc9ab7fe640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 0x7fc9a00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.757+0000 7fc9ba004640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 0x7fc9a00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.757+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7fc9a40e40d0 con 0x7fc9b4107060 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.757+0000 7fc9ba004640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 0x7fc9a00b6170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009890 tx=0x7fc9b0008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.761+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc9a4014030 con 0x7fc9b4107060 2024-07-24T21:43:37.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:37.986+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc9b4062f80 con 0x7fc9a00b3cb0 2024-07-24T21:43:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.011+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/1339751731 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9b4062f80 con 0x7fc9a00b3cb0 2024-07-24T21:43:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 msgr2=0x7fc9a00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 0x7fc9a00b6170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009890 tx=0x7fc9b0008040 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 msgr2=0x7fc9b4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c920 tx=0x7fc9a400cde0 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 shutdown_connections 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc9a00b3cb0 0x7fc9a00b6170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b41079e0 0x7fc9b419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4107060 0x7fc9b4197fe0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 --2- 172.21.15.60:0/1339751731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b4101650 0x7fc9b4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 >> 172.21.15.60:0/1339751731 conn(0x7fc9b40fd0c0 msgr2=0x7fc9b40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 shutdown_connections 2024-07-24T21:43:38.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:38.013+0000 7fc9bc268640 1 -- 172.21.15.60:0/1339751731 wait complete. 2024-07-24T21:43:38.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-24T21:43:38.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:38 smithi060 ceph-mon[33000]: osdmap e85: 12 total, 12 up, 12 in 2024-07-24T21:43:38.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:38 smithi123 ceph-mon[37143]: osdmap e85: 12 total, 12 up, 12 in 2024-07-24T21:43:39.164 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:39.164 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:38 smithi086 ceph-mon[37128]: osdmap e85: 12 total, 12 up, 12 in 2024-07-24T21:43:39.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:39 smithi060 ceph-mon[33000]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:39.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:39 smithi060 ceph-mon[33000]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-24T21:43:39.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:39 smithi123 ceph-mon[37143]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:39.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:39 smithi123 ceph-mon[37143]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-24T21:43:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:39 smithi086 ceph-mon[37128]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:39 smithi086 ceph-mon[37128]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-24T21:43:40.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 -- 172.21.15.60:0/2608359569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce01070a0 msgr2=0x7f2ce01074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/2608359569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce01070a0 0x7f2ce01074e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009900 tx=0x7f2cd402a410 comp rx=0 tx=0).stop 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 -- 172.21.15.60:0/2608359569 shutdown_connections 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/2608359569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 0x7f2ce010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/2608359569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce01070a0 0x7f2ce01074e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/2608359569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce0101690 0x7f2ce0101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 -- 172.21.15.60:0/2608359569 >> 172.21.15.60:0/2608359569 conn(0x7f2ce00fd100 msgr2=0x7f2ce00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 -- 172.21.15.60:0/2608359569 shutdown_connections 2024-07-24T21:43:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.625+0000 7f2ce5a30640 1 -- 172.21.15.60:0/2608359569 wait complete. 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.626+0000 7f2ce5a30640 1 Processor -- start 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.626+0000 7f2ce5a30640 1 -- start start 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 0x7f2ce019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 0x7f2ce019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0111620 con 0x7f2ce0101690 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01114a0 con 0x7f2ce01070a0 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce5a30640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01117a0 con 0x7f2ce0107a20 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52594/0 (socket says 172.21.15.60:52594) 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 -- 172.21.15.60:0/4201054611 learned_addr learned my addr 172.21.15.60:0/4201054611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cde7fc640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 0x7f2ce019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdf7fe640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 0x7f2ce019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 -- 172.21.15.60:0/4201054611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 msgr2=0x7f2ce019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 0x7f2ce019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 -- 172.21.15.60:0/4201054611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 msgr2=0x7f2ce019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 0x7f2ce019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 -- 172.21.15.60:0/4201054611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce01a4eb0 con 0x7f2ce0101690 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cde7fc640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 0x7f2ce019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2cdeffd640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000ebc0 tx=0x7f2cd000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.627+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0010630 con 0x7f2ce0101690 2024-07-24T21:43:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.628+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd0010c80 con 0x7f2ce0101690 2024-07-24T21:43:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.628+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce01a51a0 con 0x7f2ce0101690 2024-07-24T21:43:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.628+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0018b60 con 0x7f2ce0101690 2024-07-24T21:43:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.628+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a5700 con 0x7f2ce0101690 2024-07-24T21:43:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.629+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce0101a70 con 0x7f2ce0101690 2024-07-24T21:43:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.630+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2cd0018d00 con 0x7f2ce0101690 2024-07-24T21:43:40.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.631+0000 7f2ce4a2e640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 0x7f2cc80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:40.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.631+0000 7f2cde7fc640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 0x7f2cc80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:40.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.631+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f2cd00e4a30 con 0x7f2ce0101690 2024-07-24T21:43:40.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.631+0000 7f2cde7fc640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 0x7f2cc80b61f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4031440 tx=0x7f2cd4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:40.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.634+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f2cd00abe40 con 0x7f2ce0101690 2024-07-24T21:43:40.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.858+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2ce0062f80 con 0x7f2cc80b3d30 2024-07-24T21:43:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.863+0000 7f2ce4a2e640 1 -- 172.21.15.60:0/4201054611 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2ce0062f80 con 0x7f2cc80b3d30 2024-07-24T21:43:40.864 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_0/0bb2916b-c016-4680-92b2-7c502842687b 2024-07-24T21:43:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 msgr2=0x7f2cc80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 0x7f2cc80b61f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4031440 tx=0x7f2cd4005a20 comp rx=0 tx=0).stop 2024-07-24T21:43:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 msgr2=0x7f2ce0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000ebc0 tx=0x7f2cd000c6a0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 shutdown_connections 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2cc80b3d30 0x7f2cc80b61f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce0107a20 0x7f2ce019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ce01070a0 0x7f2ce019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 --2- 172.21.15.60:0/4201054611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce0101690 0x7f2ce0199ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 >> 172.21.15.60:0/4201054611 conn(0x7f2ce00fd100 msgr2=0x7f2ce00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 shutdown_connections 2024-07-24T21:43:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:40.865+0000 7f2ce5a30640 1 -- 172.21.15.60:0/4201054611 wait complete. 2024-07-24T21:43:41.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:41 smithi060 ceph-mon[33000]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:41.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-24T21:43:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:41 smithi123 ceph-mon[37143]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:41 smithi086 ceph-mon[37128]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 588 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:41.849 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:42.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:42 smithi060 ceph-mon[33000]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:42.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:42 smithi123 ceph-mon[37143]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:42.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:42 smithi086 ceph-mon[37128]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 -- 172.21.15.60:0/1362338605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108180 msgr2=0x7f11d01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 --2- 172.21.15.60:0/1362338605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108180 0x7f11d01085c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-07-24T21:43:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 -- 172.21.15.60:0/1362338605 shutdown_connections 2024-07-24T21:43:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 --2- 172.21.15.60:0/1362338605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0108b00 0x7f11d010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 --2- 172.21.15.60:0/1362338605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108180 0x7f11d01085c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 --2- 172.21.15.60:0/1362338605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0102770 0x7f11d0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 -- 172.21.15.60:0/1362338605 >> 172.21.15.60:0/1362338605 conn(0x7f11d00fe1c0 msgr2=0x7f11d01005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 -- 172.21.15.60:0/1362338605 shutdown_connections 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 -- 172.21.15.60:0/1362338605 wait complete. 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.575+0000 7f11d7fce640 1 Processor -- start 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 -- start start 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0102770 0x7f11d0074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 0x7f11d00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112780 con 0x7f11d0108b00 2024-07-24T21:43:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112600 con 0x7f11d0108180 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d7fce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112900 con 0x7f11d0102770 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52606/0 (socket says 172.21.15.60:52606) 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 -- 172.21.15.60:0/2273211219 learned_addr learned my addr 172.21.15.60:0/2273211219 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d5d6a640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0102770 0x7f11d0074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d5569640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 0x7f11d00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 -- 172.21.15.60:0/2273211219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0102770 msgr2=0x7f11d0074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0102770 0x7f11d0074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 -- 172.21.15.60:0/2273211219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 msgr2=0x7f11d00753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 0x7f11d00753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 -- 172.21.15.60:0/2273211219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a70d0 con 0x7f11d0108b00 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d5569640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 0x7f11d00753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:43:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.576+0000 7f11d656b640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11c800eaa0 tx=0x7f11c800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.577+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004d20 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.577+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a73c0 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.577+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8005020 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.577+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c80116b0 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.577+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a7930 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.578+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0102b50 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.579+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f11c80051d0 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.580+0000 7f11c6ffd640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 0x7f11bc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.580+0000 7f11d5d6a640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 0x7f11bc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.580+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f11c80e40e0 con 0x7f11d0108b00 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.580+0000 7f11d5d6a640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 0x7f11bc0b6170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.583+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f11c8016030 con 0x7f11d0108b00 2024-07-24T21:43:43.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:43 smithi123 ceph-mon[37143]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:43.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:43 smithi086 ceph-mon[37128]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:43.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.809+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f11d0062f80 con 0x7f11bc0b3cb0 2024-07-24T21:43:43.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:43 smithi060 ceph-mon[33000]: pgmap v286: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.833+0000 7f11c6ffd640 1 -- 172.21.15.60:0/2273211219 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11d0062f80 con 0x7f11bc0b3cb0 2024-07-24T21:43:43.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 msgr2=0x7f11bc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:43.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 0x7f11bc0b6170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc009110 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 msgr2=0x7f11d0075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11c800eaa0 tx=0x7f11c800eea0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 shutdown_connections 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11bc0b3cb0 0x7f11bc0b6170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11d0108b00 0x7f11d0075910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d0108180 0x7f11d00753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 --2- 172.21.15.60:0/2273211219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11d0102770 0x7f11d0074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.835+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 >> 172.21.15.60:0/2273211219 conn(0x7f11d00fe1c0 msgr2=0x7f11d01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.836+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 shutdown_connections 2024-07-24T21:43:43.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:43.836+0000 7f11d7fce640 1 -- 172.21.15.60:0/2273211219 wait complete. 2024-07-24T21:43:44.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-24T21:43:44.891 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:45 smithi060 ceph-mon[33000]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:45 smithi060 ceph-mon[33000]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:45.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:45 smithi123 ceph-mon[37143]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:45.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:45 smithi123 ceph-mon[37143]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:45.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:45 smithi086 ceph-mon[37128]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:45 smithi086 ceph-mon[37128]: pgmap v287: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 -- 172.21.15.60:0/722431441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 msgr2=0x7f5a7810ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/722431441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a7810ef10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a7002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 -- 172.21.15.60:0/722431441 shutdown_connections 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/722431441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a7810ef10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/722431441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 0x7f5a7810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/722431441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 0x7f5a781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 -- 172.21.15.60:0/722431441 >> 172.21.15.60:0/722431441 conn(0x7f5a780fd0c0 msgr2=0x7f5a780ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 -- 172.21.15.60:0/722431441 shutdown_connections 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.611+0000 7f5a7ed18640 1 -- 172.21.15.60:0/722431441 wait complete. 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 Processor -- start 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 -- start start 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 0x7f5a78101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 0x7f5a781018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781127e0 con 0x7f5a7810cac0 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78112660 con 0x7f5a78103f10 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7ed18640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78112960 con 0x7f5a78103510 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36082/0 (socket says 172.21.15.60:36082) 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 -- 172.21.15.60:0/2067168899 learned_addr learned my addr 172.21.15.60:0/2067168899 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7cab4640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 0x7f5a78101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a77fff640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 0x7f5a781018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 -- 172.21.15.60:0/2067168899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 msgr2=0x7f5a78101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 0x7f5a78101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 -- 172.21.15.60:0/2067168899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 msgr2=0x7f5a781018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 0x7f5a781018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.612+0000 7f5a7d2b5640 1 -- 172.21.15.60:0/2067168899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a781a7150 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a7d2b5640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5a70005cc0 tx=0x7f5a7002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70003cf0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a700044f0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a781a73e0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7002fc30 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.613+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a781a79a0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.614+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a78103970 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.615+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f5a700046a0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.616+0000 7f5a75ffb640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 0x7f5a600b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.616+0000 7f5a7cab4640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 0x7f5a600b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.616+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f5a701054b0 con 0x7f5a7810cac0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.616+0000 7f5a7cab4640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 0x7f5a600b6170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009990 tx=0x7f5a68009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.620+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5a700cc970 con 0x7f5a7810cac0 2024-07-24T21:43:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.843+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5a780fecd0 con 0x7f5a600b3cb0 2024-07-24T21:43:46.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.847+0000 7f5a75ffb640 1 -- 172.21.15.60:0/2067168899 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5a780fecd0 con 0x7f5a600b3cb0 2024-07-24T21:43:46.848 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b 2024-07-24T21:43:46.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 msgr2=0x7f5a600b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:46.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 0x7f5a600b6170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009990 tx=0x7f5a68009110 comp rx=0 tx=0).stop 2024-07-24T21:43:46.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 msgr2=0x7f5a78101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5a70005cc0 tx=0x7f5a7002f470 comp rx=0 tx=0).stop 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 shutdown_connections 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a600b3cb0 0x7f5a600b6170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a7810cac0 0x7f5a78101e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a78103f10 0x7f5a781018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 --2- 172.21.15.60:0/2067168899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a78103510 0x7f5a78101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.849+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 >> 172.21.15.60:0/2067168899 conn(0x7f5a780fd0c0 msgr2=0x7f5a7810ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.850+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 shutdown_connections 2024-07-24T21:43:46.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:46.850+0000 7f5a7ed18640 1 -- 172.21.15.60:0/2067168899 wait complete. 2024-07-24T21:43:47.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:47 smithi060 ceph-mon[33000]: pgmap v288: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:47.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:43:47.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:47 smithi123 ceph-mon[37143]: pgmap v288: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:47 smithi086 ceph-mon[37128]: pgmap v288: 97 pgs: 97 active+clean; 607 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-07-24T21:43:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:48.717 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:48 smithi060 ceph-mon[33000]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:48 smithi123 ceph-mon[37143]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:48.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:48 smithi086 ceph-mon[37128]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:43:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 -- 172.21.15.60:0/382514213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d54101650 msgr2=0x7f9d54101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/382514213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d54101650 0x7f9d54101a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009900 tx=0x7f9d4402a410 comp rx=0 tx=0).stop 2024-07-24T21:43:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 -- 172.21.15.60:0/382514213 shutdown_connections 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/382514213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d541079e0 0x7f9d5410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/382514213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 0x7f9d541074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/382514213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d54101650 0x7f9d54101a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 -- 172.21.15.60:0/382514213 >> 172.21.15.60:0/382514213 conn(0x7f9d540fd0c0 msgr2=0x7f9d540ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 -- 172.21.15.60:0/382514213 shutdown_connections 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.497+0000 7f9d58ccb640 1 -- 172.21.15.60:0/382514213 wait complete. 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 Processor -- start 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 -- start start 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 0x7f9d54197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 0x7f9d54198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54111610 con 0x7f9d541079e0 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54111490 con 0x7f9d54107060 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d58ccb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54111790 con 0x7f9d54101650 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d52d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d52d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36112/0 (socket says 172.21.15.60:36112) 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d52d9d640 1 -- 172.21.15.60:0/3392100880 learned_addr learned my addr 172.21.15.60:0/3392100880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d51d9b640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 0x7f9d54198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d5259c640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 0x7f9d54197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.498+0000 7f9d52d9d640 1 -- 172.21.15.60:0/3392100880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 msgr2=0x7f9d54197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d52d9d640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 0x7f9d54197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d52d9d640 1 -- 172.21.15.60:0/3392100880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 msgr2=0x7f9d54198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d52d9d640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 0x7f9d54198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d52d9d640 1 -- 172.21.15.60:0/3392100880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5419dbf0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d5259c640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 0x7f9d54197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d52d9d640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00eaa0 tx=0x7f9d4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00cdb0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5419dee0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d4c005020 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0116d0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.499+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541a45a0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.500+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54101a30 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.501+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f9d4c0051d0 con 0x7f9d541079e0 2024-07-24T21:43:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.502+0000 7f9d437fe640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 0x7f9d3c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:49.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.502+0000 7f9d5259c640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 0x7f9d3c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:49.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.502+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f9d4c0e4480 con 0x7f9d541079e0 2024-07-24T21:43:49.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.503+0000 7f9d5259c640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 0x7f9d3c0b6170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d44031440 tx=0x7f9d44005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:49.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.505+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9d4c016030 con 0x7f9d541079e0 2024-07-24T21:43:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:49 smithi060 ceph-mon[33000]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:49.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.739+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d54198c70 con 0x7f9d541079e0 2024-07-24T21:43:49.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.740+0000 7f9d437fe640 1 -- 172.21.15.60:0/3392100880 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f9d4c0ab830 con 0x7f9d541079e0 2024-07-24T21:43:49.741 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:49.741 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:43:49.742 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:43:49.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:49 smithi123 ceph-mon[37143]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 msgr2=0x7f9d3c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 0x7f9d3c0b6170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9d44031440 tx=0x7f9d44005a20 comp rx=0 tx=0).stop 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 msgr2=0x7f9d5419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00eaa0 tx=0x7f9d4c00ef60 comp rx=0 tx=0).stop 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 shutdown_connections 2024-07-24T21:43:49.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d3c0b3cb0 0x7f9d3c0b6170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d541079e0 0x7f9d5419d420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d54107060 0x7f9d54198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 --2- 172.21.15.60:0/3392100880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101650 0x7f9d54197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 >> 172.21.15.60:0/3392100880 conn(0x7f9d540fd0c0 msgr2=0x7f9d540fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 shutdown_connections 2024-07-24T21:43:49.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:49.742+0000 7f9d58ccb640 1 -- 172.21.15.60:0/3392100880 wait complete. 2024-07-24T21:43:49.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:49 smithi086 ceph-mon[37128]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:50.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:43:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:50.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3392100880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:50.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3392100880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:50.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3392100880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:43:51.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:51 smithi123 ceph-mon[37143]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.1 KiB/s wr, 7 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:51.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:51 smithi086 ceph-mon[37128]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.1 KiB/s wr, 7 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:51 smithi060 ceph-mon[33000]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.1 KiB/s wr, 7 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-07-24T21:43:52.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 -- 172.21.15.60:0/1978504464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42801079e0 msgr2=0x7f428010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:52.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 --2- 172.21.15.60:0/1978504464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42801079e0 0x7f428010e1e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4278009ca0 tx=0x7f427802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 -- 172.21.15.60:0/1978504464 shutdown_connections 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 --2- 172.21.15.60:0/1978504464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42801079e0 0x7f428010e1e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 --2- 172.21.15.60:0/1978504464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4280107060 0x7f42801074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 --2- 172.21.15.60:0/1978504464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4280101650 0x7f4280101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 -- 172.21.15.60:0/1978504464 >> 172.21.15.60:0/1978504464 conn(0x7f42800fd0c0 msgr2=0x7f42800ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 -- 172.21.15.60:0/1978504464 shutdown_connections 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.363+0000 7f428867e640 1 -- 172.21.15.60:0/1978504464 wait complete. 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 Processor -- start 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 -- start start 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:52.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4280107060 0x7f42800fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42801079e0 0x7f42800fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4280111670 con 0x7f4280101650 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42801114f0 con 0x7f42801079e0 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428867e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42801117f0 con 0x7f4280107060 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36130/0 (socket says 172.21.15.60:36130) 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 -- 172.21.15.60:0/3641690465 learned_addr learned my addr 172.21.15.60:0/3641690465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 -- 172.21.15.60:0/3641690465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4280107060 msgr2=0x7f42800fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4280107060 0x7f42800fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 -- 172.21.15.60:0/3641690465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42801079e0 msgr2=0x7f42800fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42801079e0 0x7f42800fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 -- 172.21.15.60:0/3641690465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42801a7100 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.364+0000 7f428641a640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b4a0 tx=0x7f427c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.365+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c014070 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.365+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c010ac0 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.365+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42801a73f0 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.365+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c017c30 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.365+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801a7980 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.366+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4280101a30 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.367+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f427c017070 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.368+0000 7f42777fe640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 0x7f426c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.368+0000 7f4285c19640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 0x7f426c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.368+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f427c0e3530 con 0x7f4280101650 2024-07-24T21:43:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.368+0000 7f4285c19640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 0x7f426c0b61f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4270005fd0 tx=0x7f4270005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:52.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.371+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f427c0aa9f0 con 0x7f4280101650 2024-07-24T21:43:52.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.565+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f428002e000 con 0x7f4280101650 2024-07-24T21:43:52.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.567+0000 7f42777fe640 1 -- 172.21.15.60:0/3641690465 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22978 (secure 0 0 0) 0x7f427c0aa2f0 con 0x7f4280101650 2024-07-24T21:43:52.568 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:52.568 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:43:37.396360+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 msgr2=0x7f426c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 0x7f426c0b61f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4270005fd0 tx=0x7f4270005850 comp rx=0 tx=0).stop 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 msgr2=0x7f42800fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b4a0 tx=0x7f427c00b960 comp rx=0 tx=0).stop 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 shutdown_connections 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f426c0b3d30 0x7f426c0b61f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42801079e0 0x7f42800fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4280107060 0x7f42800fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 --2- 172.21.15.60:0/3641690465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4280101650 0x7f42800fd5c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 >> 172.21.15.60:0/3641690465 conn(0x7f42800fd0c0 msgr2=0x7f428006a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 shutdown_connections 2024-07-24T21:43:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:52.569+0000 7f428867e640 1 -- 172.21.15.60:0/3641690465 wait complete. 2024-07-24T21:43:53.403 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-24T21:43:53.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:43:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:53.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:53 smithi123 ceph-mon[37143]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-07-24T21:43:53.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:53 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3641690465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:53.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:53 smithi086 ceph-mon[37128]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-07-24T21:43:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:53 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3641690465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:53.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:53 smithi060 ceph-mon[33000]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-07-24T21:43:53.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:53 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3641690465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:43:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 -- 172.21.15.60:0/561995838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920102790 msgr2=0x7ff920102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 --2- 172.21.15.60:0/561995838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920102790 0x7ff920102b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff910009ca0 tx=0x7ff91002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 -- 172.21.15.60:0/561995838 shutdown_connections 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 --2- 172.21.15.60:0/561995838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 0x7ff92010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 --2- 172.21.15.60:0/561995838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9201081a0 0x7ff9201085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 --2- 172.21.15.60:0/561995838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920102790 0x7ff920102b70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 -- 172.21.15.60:0/561995838 >> 172.21.15.60:0/561995838 conn(0x7ff9200fe200 msgr2=0x7ff920100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 -- 172.21.15.60:0/561995838 shutdown_connections 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.402+0000 7ff92757f640 1 -- 172.21.15.60:0/561995838 wait complete. 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 Processor -- start 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 -- start start 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920102790 0x7ff920198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 0x7ff92019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff920112720 con 0x7ff9201081a0 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201125a0 con 0x7ff920102790 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff92757f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201128a0 con 0x7ff920108b20 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff924b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff924b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36152/0 (socket says 172.21.15.60:36152) 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.403+0000 7ff924b1a640 1 -- 172.21.15.60:0/1266275917 learned_addr learned my addr 172.21.15.60:0/1266275917 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff92531b640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920102790 0x7ff920198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff925b1c640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 0x7ff92019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 -- 172.21.15.60:0/1266275917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 msgr2=0x7ff92019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 0x7ff92019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 -- 172.21.15.60:0/1266275917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920102790 msgr2=0x7ff920198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920102790 0x7ff920198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 -- 172.21.15.60:0/1266275917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92019ece0 con 0x7ff9201081a0 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff924b1a640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d950 tx=0x7ff91400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914017070 con 0x7ff9201081a0 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff914009c30 con 0x7ff9201081a0 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92019efd0 con 0x7ff9201081a0 2024-07-24T21:43:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400f620 con 0x7ff9201081a0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.404+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9201a5690 con 0x7ff9201081a0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.405+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff920102b70 con 0x7ff9201081a0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.407+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7ff914010460 con 0x7ff9201081a0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.407+0000 7ff91e7fc640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 0x7ff9080b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.408+0000 7ff92531b640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 0x7ff9080b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.408+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7ff9140e3a50 con 0x7ff9201081a0 2024-07-24T21:43:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.408+0000 7ff92531b640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 0x7ff9080b6170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff910009430 tx=0x7ff910009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.411+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff914009de0 con 0x7ff9201081a0 2024-07-24T21:43:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.601+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff920062f80 con 0x7ff9080b3cb0 2024-07-24T21:43:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.606+0000 7ff91e7fc640 1 -- 172.21.15.60:0/1266275917 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258699 (secure 0 0 0) 0x7ff920062f80 con 0x7ff9080b3cb0 2024-07-24T21:43:55.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 msgr2=0x7ff9080b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 0x7ff9080b6170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff910009430 tx=0x7ff910009360 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 msgr2=0x7ff920199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d950 tx=0x7ff91400de10 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 shutdown_connections 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff9080b3cb0 0x7ff9080b6170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff920108b20 0x7ff92019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9201081a0 0x7ff920199170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 --2- 172.21.15.60:0/1266275917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920102790 0x7ff920198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:55.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 >> 172.21.15.60:0/1266275917 conn(0x7ff9200fe200 msgr2=0x7ff9200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 shutdown_connections 2024-07-24T21:43:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:55.610+0000 7ff92757f640 1 -- 172.21.15.60:0/1266275917 wait complete. 2024-07-24T21:43:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:55 smithi123 ceph-mon[37143]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T21:43:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:55 smithi086 ceph-mon[37128]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T21:43:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:55 smithi060 ceph-mon[33000]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T21:43:56.378 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-07-24T21:43:54.045937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648970,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":244,"num_read_kb":211,"num_write":285,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403832,"kb_used_data":13052,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537992,"statfs":{"total":1151940427776,"available":1151526903808,"internally_reserved":0,"allocated":13365248,"data_stored":7497741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27539,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":72,"num_read_kb":63,"num_write":24,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003139"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404780+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:43:37.404780+0000","last_peered":"2024-07-24T21:43:37.404780+0000","last_clean":"2024-07-24T21:43:37.404780+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:43:37.404780+0000","last_undegraded":"2024-07-24T21:43:37.404780+0000","last_fullsized":"2024-07-24T21:43:37.404780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402109+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:43:37.402109+0000","last_peered":"2024-07-24T21:43:37.402109+0000","last_clean":"2024-07-24T21:43:37.402109+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:43:37.402109+0000","last_undegraded":"2024-07-24T21:43:37.402109+0000","last_fullsized":"2024-07-24T21:43:37.402109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402092+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:43:37.402092+0000","last_peered":"2024-07-24T21:43:37.402092+0000","last_clean":"2024-07-24T21:43:37.402092+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:43:37.402092+0000","last_undegraded":"2024-07-24T21:43:37.402092+0000","last_fullsized":"2024-07-24T21:43:37.402092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874608+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:43:37.874608+0000","last_peered":"2024-07-24T21:43:37.874608+0000","last_clean":"2024-07-24T21:43:37.874608+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:43:37.874608+0000","last_undegraded":"2024-07-24T21:43:37.874608+0000","last_fullsized":"2024-07-24T21:43:37.874608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401096+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:43:37.401096+0000","last_peered":"2024-07-24T21:43:37.401096+0000","last_clean":"2024-07-24T21:43:37.401096+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:43:37.401096+0000","last_undegraded":"2024-07-24T21:43:37.401096+0000","last_fullsized":"2024-07-24T21:43:37.401096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409471+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:43:37.409471+0000","last_peered":"2024-07-24T21:43:37.409471+0000","last_clean":"2024-07-24T21:43:37.409471+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:43:37.409471+0000","last_undegraded":"2024-07-24T21:43:37.409471+0000","last_fullsized":"2024-07-24T21:43:37.409471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400290+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:43:37.400290+0000","last_peered":"2024-07-24T21:43:37.400290+0000","last_clean":"2024-07-24T21:43:37.400290+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:43:37.400290+0000","last_undegraded":"2024-07-24T21:43:37.400290+0000","last_fullsized":"2024-07-24T21:43:37.400290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873092+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:43:37.873092+0000","last_peered":"2024-07-24T21:43:37.873092+0000","last_clean":"2024-07-24T21:43:37.873092+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:43:37.873092+0000","last_undegraded":"2024-07-24T21:43:37.873092+0000","last_fullsized":"2024-07-24T21:43:37.873092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874364+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:43:37.874364+0000","last_peered":"2024-07-24T21:43:37.874364+0000","last_clean":"2024-07-24T21:43:37.874364+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:43:37.874364+0000","last_undegraded":"2024-07-24T21:43:37.874364+0000","last_fullsized":"2024-07-24T21:43:37.874364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409431+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:43:37.409431+0000","last_peered":"2024-07-24T21:43:37.409431+0000","last_clean":"2024-07-24T21:43:37.409431+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:43:37.409431+0000","last_undegraded":"2024-07-24T21:43:37.409431+0000","last_fullsized":"2024-07-24T21:43:37.409431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400770+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:43:37.400770+0000","last_peered":"2024-07-24T21:43:37.400770+0000","last_clean":"2024-07-24T21:43:37.400770+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:43:37.400770+0000","last_undegraded":"2024-07-24T21:43:37.400770+0000","last_fullsized":"2024-07-24T21:43:37.400770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874612+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:43:37.874612+0000","last_peered":"2024-07-24T21:43:37.874612+0000","last_clean":"2024-07-24T21:43:37.874612+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:43:37.874612+0000","last_undegraded":"2024-07-24T21:43:37.874612+0000","last_fullsized":"2024-07-24T21:43:37.874612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402759+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:43:37.402759+0000","last_peered":"2024-07-24T21:43:37.402759+0000","last_clean":"2024-07-24T21:43:37.402759+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:43:37.402759+0000","last_undegraded":"2024-07-24T21:43:37.402759+0000","last_fullsized":"2024-07-24T21:43:37.402759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402021+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:43:37.402021+0000","last_peered":"2024-07-24T21:43:37.402021+0000","last_clean":"2024-07-24T21:43:37.402021+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:43:37.402021+0000","last_undegraded":"2024-07-24T21:43:37.402021+0000","last_fullsized":"2024-07-24T21:43:37.402021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401947+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:43:37.401947+0000","last_peered":"2024-07-24T21:43:37.401947+0000","last_clean":"2024-07-24T21:43:37.401947+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:43:37.401947+0000","last_undegraded":"2024-07-24T21:43:37.401947+0000","last_fullsized":"2024-07-24T21:43:37.401947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873505+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:43:37.873505+0000","last_peered":"2024-07-24T21:43:37.873505+0000","last_clean":"2024-07-24T21:43:37.873505+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:43:37.873505+0000","last_undegraded":"2024-07-24T21:43:37.873505+0000","last_fullsized":"2024-07-24T21:43:37.873505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404863+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:43:37.404863+0000","last_peered":"2024-07-24T21:43:37.404863+0000","last_clean":"2024-07-24T21:43:37.404863+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:43:37.404863+0000","last_undegraded":"2024-07-24T21:43:37.404863+0000","last_fullsized":"2024-07-24T21:43:37.404863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.407884+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:43:37.407884+0000","last_peered":"2024-07-24T21:43:37.407884+0000","last_clean":"2024-07-24T21:43:37.407884+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:43:37.407884+0000","last_undegraded":"2024-07-24T21:43:37.407884+0000","last_fullsized":"2024-07-24T21:43:37.407884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404348+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:43:37.404348+0000","last_peered":"2024-07-24T21:43:37.404348+0000","last_clean":"2024-07-24T21:43:37.404348+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:43:37.404348+0000","last_undegraded":"2024-07-24T21:43:37.404348+0000","last_fullsized":"2024-07-24T21:43:37.404348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404775+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:43:37.404775+0000","last_peered":"2024-07-24T21:43:37.404775+0000","last_clean":"2024-07-24T21:43:37.404775+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:43:37.404775+0000","last_undegraded":"2024-07-24T21:43:37.404775+0000","last_fullsized":"2024-07-24T21:43:37.404775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402794+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:43:37.402794+0000","last_peered":"2024-07-24T21:43:37.402794+0000","last_clean":"2024-07-24T21:43:37.402794+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:43:37.402794+0000","last_undegraded":"2024-07-24T21:43:37.402794+0000","last_fullsized":"2024-07-24T21:43:37.402794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873680+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:43:37.873680+0000","last_peered":"2024-07-24T21:43:37.873680+0000","last_clean":"2024-07-24T21:43:37.873680+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:43:37.873680+0000","last_undegraded":"2024-07-24T21:43:37.873680+0000","last_fullsized":"2024-07-24T21:43:37.873680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872834+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:43:37.872834+0000","last_peered":"2024-07-24T21:43:37.872834+0000","last_clean":"2024-07-24T21:43:37.872834+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:43:37.872834+0000","last_undegraded":"2024-07-24T21:43:37.872834+0000","last_fullsized":"2024-07-24T21:43:37.872834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409464+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:43:37.409464+0000","last_peered":"2024-07-24T21:43:37.409464+0000","last_clean":"2024-07-24T21:43:37.409464+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:43:37.409464+0000","last_undegraded":"2024-07-24T21:43:37.409464+0000","last_fullsized":"2024-07-24T21:43:37.409464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873302+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:43:37.873302+0000","last_peered":"2024-07-24T21:43:37.873302+0000","last_clean":"2024-07-24T21:43:37.873302+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:43:37.873302+0000","last_undegraded":"2024-07-24T21:43:37.873302+0000","last_fullsized":"2024-07-24T21:43:37.873302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404573+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:43:37.404573+0000","last_peered":"2024-07-24T21:43:37.404573+0000","last_clean":"2024-07-24T21:43:37.404573+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:43:37.404573+0000","last_undegraded":"2024-07-24T21:43:37.404573+0000","last_fullsized":"2024-07-24T21:43:37.404573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.405266+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:43:37.405266+0000","last_peered":"2024-07-24T21:43:37.405266+0000","last_clean":"2024-07-24T21:43:37.405266+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:43:37.405266+0000","last_undegraded":"2024-07-24T21:43:37.405266+0000","last_fullsized":"2024-07-24T21:43:37.405266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402909+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:43:37.402909+0000","last_peered":"2024-07-24T21:43:37.402909+0000","last_clean":"2024-07-24T21:43:37.402909+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:43:37.402909+0000","last_undegraded":"2024-07-24T21:43:37.402909+0000","last_fullsized":"2024-07-24T21:43:37.402909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402344+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:43:37.402344+0000","last_peered":"2024-07-24T21:43:37.402344+0000","last_clean":"2024-07-24T21:43:37.402344+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:43:37.402344+0000","last_undegraded":"2024-07-24T21:43:37.402344+0000","last_fullsized":"2024-07-24T21:43:37.402344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403560+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:43:37.403560+0000","last_peered":"2024-07-24T21:43:37.403560+0000","last_clean":"2024-07-24T21:43:37.403560+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:43:37.403560+0000","last_undegraded":"2024-07-24T21:43:37.403560+0000","last_fullsized":"2024-07-24T21:43:37.403560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.875378+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:43:37.875378+0000","last_peered":"2024-07-24T21:43:37.875378+0000","last_clean":"2024-07-24T21:43:37.875378+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:43:37.875378+0000","last_undegraded":"2024-07-24T21:43:37.875378+0000","last_fullsized":"2024-07-24T21:43:37.875378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404358+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:43:37.404358+0000","last_peered":"2024-07-24T21:43:37.404358+0000","last_clean":"2024-07-24T21:43:37.404358+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:43:37.404358+0000","last_undegraded":"2024-07-24T21:43:37.404358+0000","last_fullsized":"2024-07-24T21:43:37.404358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401138+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:43:37.401138+0000","last_peered":"2024-07-24T21:43:37.401138+0000","last_clean":"2024-07-24T21:43:37.401138+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:43:37.401138+0000","last_undegraded":"2024-07-24T21:43:37.401138+0000","last_fullsized":"2024-07-24T21:43:37.401138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:52.413864+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:43:52.413864+0000","last_peered":"2024-07-24T21:43:52.413864+0000","last_clean":"2024-07-24T21:43:52.413864+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:43:52.413864+0000","last_undegraded":"2024-07-24T21:43:52.413864+0000","last_fullsized":"2024-07-24T21:43:52.413864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399997+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:43:37.399997+0000","last_peered":"2024-07-24T21:43:37.399997+0000","last_clean":"2024-07-24T21:43:37.399997+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:43:37.399997+0000","last_undegraded":"2024-07-24T21:43:37.399997+0000","last_fullsized":"2024-07-24T21:43:37.399997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403801+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:43:37.403801+0000","last_peered":"2024-07-24T21:43:37.403801+0000","last_clean":"2024-07-24T21:43:37.403801+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:43:37.403801+0000","last_undegraded":"2024-07-24T21:43:37.403801+0000","last_fullsized":"2024-07-24T21:43:37.403801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402472+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:43:37.402472+0000","last_peered":"2024-07-24T21:43:37.402472+0000","last_clean":"2024-07-24T21:43:37.402472+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:43:37.402472+0000","last_undegraded":"2024-07-24T21:43:37.402472+0000","last_fullsized":"2024-07-24T21:43:37.402472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872913+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:43:37.872913+0000","last_peered":"2024-07-24T21:43:37.872913+0000","last_clean":"2024-07-24T21:43:37.872913+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:43:37.872913+0000","last_undegraded":"2024-07-24T21:43:37.872913+0000","last_fullsized":"2024-07-24T21:43:37.872913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400758+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:43:37.400758+0000","last_peered":"2024-07-24T21:43:37.400758+0000","last_clean":"2024-07-24T21:43:37.400758+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:43:37.400758+0000","last_undegraded":"2024-07-24T21:43:37.400758+0000","last_fullsized":"2024-07-24T21:43:37.400758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.407915+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:43:37.407915+0000","last_peered":"2024-07-24T21:43:37.407915+0000","last_clean":"2024-07-24T21:43:37.407915+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:43:37.407915+0000","last_undegraded":"2024-07-24T21:43:37.407915+0000","last_fullsized":"2024-07-24T21:43:37.407915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:49.408503+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:43:49.408503+0000","last_peered":"2024-07-24T21:43:49.408503+0000","last_clean":"2024-07-24T21:43:49.408503+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:43:49.408503+0000","last_undegraded":"2024-07-24T21:43:49.408503+0000","last_fullsized":"2024-07-24T21:43:49.408503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873995+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:43:37.873995+0000","last_peered":"2024-07-24T21:43:37.873995+0000","last_clean":"2024-07-24T21:43:37.873995+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:43:37.873995+0000","last_undegraded":"2024-07-24T21:43:37.873995+0000","last_fullsized":"2024-07-24T21:43:37.873995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400808+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:43:37.400808+0000","last_peered":"2024-07-24T21:43:37.400808+0000","last_clean":"2024-07-24T21:43:37.400808+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:43:37.400808+0000","last_undegraded":"2024-07-24T21:43:37.400808+0000","last_fullsized":"2024-07-24T21:43:37.400808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404982+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:43:37.404982+0000","last_peered":"2024-07-24T21:43:37.404982+0000","last_clean":"2024-07-24T21:43:37.404982+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:43:37.404982+0000","last_undegraded":"2024-07-24T21:43:37.404982+0000","last_fullsized":"2024-07-24T21:43:37.404982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400259+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:43:37.400259+0000","last_peered":"2024-07-24T21:43:37.400259+0000","last_clean":"2024-07-24T21:43:37.400259+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:43:37.400259+0000","last_undegraded":"2024-07-24T21:43:37.400259+0000","last_fullsized":"2024-07-24T21:43:37.400259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874272+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:43:37.874272+0000","last_peered":"2024-07-24T21:43:37.874272+0000","last_clean":"2024-07-24T21:43:37.874272+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:43:37.874272+0000","last_undegraded":"2024-07-24T21:43:37.874272+0000","last_fullsized":"2024-07-24T21:43:37.874272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'26","reported_seq":103,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:50.414461+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:43:50.414461+0000","last_peered":"2024-07-24T21:43:50.414461+0000","last_clean":"2024-07-24T21:43:50.414461+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:43:50.414461+0000","last_undegraded":"2024-07-24T21:43:50.414461+0000","last_fullsized":"2024-07-24T21:43:50.414461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":78,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401614+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:43:37.401614+0000","last_peered":"2024-07-24T21:43:37.401614+0000","last_clean":"2024-07-24T21:43:37.401614+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:43:37.401614+0000","last_undegraded":"2024-07-24T21:43:37.401614+0000","last_fullsized":"2024-07-24T21:43:37.401614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403946+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:43:37.403946+0000","last_peered":"2024-07-24T21:43:37.403946+0000","last_clean":"2024-07-24T21:43:37.403946+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:43:37.403946+0000","last_undegraded":"2024-07-24T21:43:37.403946+0000","last_fullsized":"2024-07-24T21:43:37.403946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409460+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:43:37.409460+0000","last_peered":"2024-07-24T21:43:37.409460+0000","last_clean":"2024-07-24T21:43:37.409460+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:43:37.409460+0000","last_undegraded":"2024-07-24T21:43:37.409460+0000","last_fullsized":"2024-07-24T21:43:37.409460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399962+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:43:37.399962+0000","last_peered":"2024-07-24T21:43:37.399962+0000","last_clean":"2024-07-24T21:43:37.399962+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:43:37.399962+0000","last_undegraded":"2024-07-24T21:43:37.399962+0000","last_fullsized":"2024-07-24T21:43:37.399962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400532+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:43:37.400532+0000","last_peered":"2024-07-24T21:43:37.400532+0000","last_clean":"2024-07-24T21:43:37.400532+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:43:37.400532+0000","last_undegraded":"2024-07-24T21:43:37.400532+0000","last_fullsized":"2024-07-24T21:43:37.400532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:50.424183+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:43:50.424183+0000","last_peered":"2024-07-24T21:43:50.424183+0000","last_clean":"2024-07-24T21:43:50.424183+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:43:50.424183+0000","last_undegraded":"2024-07-24T21:43:50.424183+0000","last_fullsized":"2024-07-24T21:43:50.424183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400223+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:43:37.400223+0000","last_peered":"2024-07-24T21:43:37.400223+0000","last_clean":"2024-07-24T21:43:37.400223+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:43:37.400223+0000","last_undegraded":"2024-07-24T21:43:37.400223+0000","last_fullsized":"2024-07-24T21:43:37.400223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401671+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:43:37.401671+0000","last_peered":"2024-07-24T21:43:37.401671+0000","last_clean":"2024-07-24T21:43:37.401671+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:43:37.401671+0000","last_undegraded":"2024-07-24T21:43:37.401671+0000","last_fullsized":"2024-07-24T21:43:37.401671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399907+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:43:37.399907+0000","last_peered":"2024-07-24T21:43:37.399907+0000","last_clean":"2024-07-24T21:43:37.399907+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:43:37.399907+0000","last_undegraded":"2024-07-24T21:43:37.399907+0000","last_fullsized":"2024-07-24T21:43:37.399907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401927+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:43:37.401927+0000","last_peered":"2024-07-24T21:43:37.401927+0000","last_clean":"2024-07-24T21:43:37.401927+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:43:37.401927+0000","last_undegraded":"2024-07-24T21:43:37.401927+0000","last_fullsized":"2024-07-24T21:43:37.401927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404643+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:43:37.404643+0000","last_peered":"2024-07-24T21:43:37.404643+0000","last_clean":"2024-07-24T21:43:37.404643+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:43:37.404643+0000","last_undegraded":"2024-07-24T21:43:37.404643+0000","last_fullsized":"2024-07-24T21:43:37.404643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403044+0000","last_change":"2024-07-24T21:43:09.255779+0000","last_active":"2024-07-24T21:43:37.403044+0000","last_peered":"2024-07-24T21:43:37.403044+0000","last_clean":"2024-07-24T21:43:37.403044+0000","last_became_active":"2024-07-24T21:43:09.255448+0000","last_became_peered":"2024-07-24T21:43:09.255448+0000","last_unstale":"2024-07-24T21:43:37.403044+0000","last_undegraded":"2024-07-24T21:43:37.403044+0000","last_fullsized":"2024-07-24T21:43:37.403044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:13:05.781739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400808+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:43:37.400808+0000","last_peered":"2024-07-24T21:43:37.400808+0000","last_clean":"2024-07-24T21:43:37.400808+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:43:37.400808+0000","last_undegraded":"2024-07-24T21:43:37.400808+0000","last_fullsized":"2024-07-24T21:43:37.400808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400390+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:43:37.400390+0000","last_peered":"2024-07-24T21:43:37.400390+0000","last_clean":"2024-07-24T21:43:37.400390+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:43:37.400390+0000","last_undegraded":"2024-07-24T21:43:37.400390+0000","last_fullsized":"2024-07-24T21:43:37.400390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400668+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:43:37.400668+0000","last_peered":"2024-07-24T21:43:37.400668+0000","last_clean":"2024-07-24T21:43:37.400668+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:43:37.400668+0000","last_undegraded":"2024-07-24T21:43:37.400668+0000","last_fullsized":"2024-07-24T21:43:37.400668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404031+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:43:37.404031+0000","last_peered":"2024-07-24T21:43:37.404031+0000","last_clean":"2024-07-24T21:43:37.404031+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:43:37.404031+0000","last_undegraded":"2024-07-24T21:43:37.404031+0000","last_fullsized":"2024-07-24T21:43:37.404031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400776+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:43:37.400776+0000","last_peered":"2024-07-24T21:43:37.400776+0000","last_clean":"2024-07-24T21:43:37.400776+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:43:37.400776+0000","last_undegraded":"2024-07-24T21:43:37.400776+0000","last_fullsized":"2024-07-24T21:43:37.400776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400392+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:43:37.400392+0000","last_peered":"2024-07-24T21:43:37.400392+0000","last_clean":"2024-07-24T21:43:37.400392+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:43:37.400392+0000","last_undegraded":"2024-07-24T21:43:37.400392+0000","last_fullsized":"2024-07-24T21:43:37.400392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874347+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:43:37.874347+0000","last_peered":"2024-07-24T21:43:37.874347+0000","last_clean":"2024-07-24T21:43:37.874347+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:43:37.874347+0000","last_undegraded":"2024-07-24T21:43:37.874347+0000","last_fullsized":"2024-07-24T21:43:37.874347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873011+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:43:37.873011+0000","last_peered":"2024-07-24T21:43:37.873011+0000","last_clean":"2024-07-24T21:43:37.873011+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:43:37.873011+0000","last_undegraded":"2024-07-24T21:43:37.873011+0000","last_fullsized":"2024-07-24T21:43:37.873011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400387+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:43:37.400387+0000","last_peered":"2024-07-24T21:43:37.400387+0000","last_clean":"2024-07-24T21:43:37.400387+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:43:37.400387+0000","last_undegraded":"2024-07-24T21:43:37.400387+0000","last_fullsized":"2024-07-24T21:43:37.400387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402349+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:43:37.402349+0000","last_peered":"2024-07-24T21:43:37.402349+0000","last_clean":"2024-07-24T21:43:37.402349+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:43:37.402349+0000","last_undegraded":"2024-07-24T21:43:37.402349+0000","last_fullsized":"2024-07-24T21:43:37.402349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:51.424997+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:43:51.424997+0000","last_peered":"2024-07-24T21:43:51.424997+0000","last_clean":"2024-07-24T21:43:51.424997+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:43:51.424997+0000","last_undegraded":"2024-07-24T21:43:51.424997+0000","last_fullsized":"2024-07-24T21:43:51.424997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.405723+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:43:37.405723+0000","last_peered":"2024-07-24T21:43:37.405723+0000","last_clean":"2024-07-24T21:43:37.405723+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:43:37.405723+0000","last_undegraded":"2024-07-24T21:43:37.405723+0000","last_fullsized":"2024-07-24T21:43:37.405723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400582+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:43:37.400582+0000","last_peered":"2024-07-24T21:43:37.400582+0000","last_clean":"2024-07-24T21:43:37.400582+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:43:37.400582+0000","last_undegraded":"2024-07-24T21:43:37.400582+0000","last_fullsized":"2024-07-24T21:43:37.400582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403924+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:43:37.403924+0000","last_peered":"2024-07-24T21:43:37.403924+0000","last_clean":"2024-07-24T21:43:37.403924+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:43:37.403924+0000","last_undegraded":"2024-07-24T21:43:37.403924+0000","last_fullsized":"2024-07-24T21:43:37.403924+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.408073+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:43:37.408073+0000","last_peered":"2024-07-24T21:43:37.408073+0000","last_clean":"2024-07-24T21:43:37.408073+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:43:37.408073+0000","last_undegraded":"2024-07-24T21:43:37.408073+0000","last_fullsized":"2024-07-24T21:43:37.408073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404089+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:43:37.404089+0000","last_peered":"2024-07-24T21:43:37.404089+0000","last_clean":"2024-07-24T21:43:37.404089+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:43:37.404089+0000","last_undegraded":"2024-07-24T21:43:37.404089+0000","last_fullsized":"2024-07-24T21:43:37.404089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.412278+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:43:37.412278+0000","last_peered":"2024-07-24T21:43:37.412278+0000","last_clean":"2024-07-24T21:43:37.412278+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:43:37.412278+0000","last_undegraded":"2024-07-24T21:43:37.412278+0000","last_fullsized":"2024-07-24T21:43:37.412278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402915+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:43:37.402915+0000","last_peered":"2024-07-24T21:43:37.402915+0000","last_clean":"2024-07-24T21:43:37.402915+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:43:37.402915+0000","last_undegraded":"2024-07-24T21:43:37.402915+0000","last_fullsized":"2024-07-24T21:43:37.402915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:49.301112+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:43:49.301112+0000","last_peered":"2024-07-24T21:43:49.301112+0000","last_clean":"2024-07-24T21:43:49.301112+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:43:49.301112+0000","last_undegraded":"2024-07-24T21:43:49.301112+0000","last_fullsized":"2024-07-24T21:43:49.301112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873389+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:43:37.873389+0000","last_peered":"2024-07-24T21:43:37.873389+0000","last_clean":"2024-07-24T21:43:37.873389+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:43:37.873389+0000","last_undegraded":"2024-07-24T21:43:37.873389+0000","last_fullsized":"2024-07-24T21:43:37.873389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874876+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:43:37.874876+0000","last_peered":"2024-07-24T21:43:37.874876+0000","last_clean":"2024-07-24T21:43:37.874876+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:43:37.874876+0000","last_undegraded":"2024-07-24T21:43:37.874876+0000","last_fullsized":"2024-07-24T21:43:37.874876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402421+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:43:37.402421+0000","last_peered":"2024-07-24T21:43:37.402421+0000","last_clean":"2024-07-24T21:43:37.402421+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:43:37.402421+0000","last_undegraded":"2024-07-24T21:43:37.402421+0000","last_fullsized":"2024-07-24T21:43:37.402421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873657+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:43:37.873657+0000","last_peered":"2024-07-24T21:43:37.873657+0000","last_clean":"2024-07-24T21:43:37.873657+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:43:37.873657+0000","last_undegraded":"2024-07-24T21:43:37.873657+0000","last_fullsized":"2024-07-24T21:43:37.873657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400895+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:43:37.400895+0000","last_peered":"2024-07-24T21:43:37.400895+0000","last_clean":"2024-07-24T21:43:37.400895+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:43:37.400895+0000","last_undegraded":"2024-07-24T21:43:37.400895+0000","last_fullsized":"2024-07-24T21:43:37.400895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872966+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:43:37.872966+0000","last_peered":"2024-07-24T21:43:37.872966+0000","last_clean":"2024-07-24T21:43:37.872966+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:43:37.872966+0000","last_undegraded":"2024-07-24T21:43:37.872966+0000","last_fullsized":"2024-07-24T21:43:37.872966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404498+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:43:37.404498+0000","last_peered":"2024-07-24T21:43:37.404498+0000","last_clean":"2024-07-24T21:43:37.404498+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:43:37.404498+0000","last_undegraded":"2024-07-24T21:43:37.404498+0000","last_fullsized":"2024-07-24T21:43:37.404498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403660+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:43:37.403660+0000","last_peered":"2024-07-24T21:43:37.403660+0000","last_clean":"2024-07-24T21:43:37.403660+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:43:37.403660+0000","last_undegraded":"2024-07-24T21:43:37.403660+0000","last_fullsized":"2024-07-24T21:43:37.403660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872939+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:43:37.872939+0000","last_peered":"2024-07-24T21:43:37.872939+0000","last_clean":"2024-07-24T21:43:37.872939+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:43:37.872939+0000","last_undegraded":"2024-07-24T21:43:37.872939+0000","last_fullsized":"2024-07-24T21:43:37.872939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401194+0000","last_change":"2024-07-24T21:43:09.256542+0000","last_active":"2024-07-24T21:43:37.401194+0000","last_peered":"2024-07-24T21:43:37.401194+0000","last_clean":"2024-07-24T21:43:37.401194+0000","last_became_active":"2024-07-24T21:43:09.256187+0000","last_became_peered":"2024-07-24T21:43:09.256187+0000","last_unstale":"2024-07-24T21:43:37.401194+0000","last_undegraded":"2024-07-24T21:43:37.401194+0000","last_fullsized":"2024-07-24T21:43:37.401194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:20:32.874780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400438+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:43:37.400438+0000","last_peered":"2024-07-24T21:43:37.400438+0000","last_clean":"2024-07-24T21:43:37.400438+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:43:37.400438+0000","last_undegraded":"2024-07-24T21:43:37.400438+0000","last_fullsized":"2024-07-24T21:43:37.400438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400118+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:43:37.400118+0000","last_peered":"2024-07-24T21:43:37.400118+0000","last_clean":"2024-07-24T21:43:37.400118+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:43:37.400118+0000","last_undegraded":"2024-07-24T21:43:37.400118+0000","last_fullsized":"2024-07-24T21:43:37.400118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401641+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:43:37.401641+0000","last_peered":"2024-07-24T21:43:37.401641+0000","last_clean":"2024-07-24T21:43:37.401641+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:43:37.401641+0000","last_undegraded":"2024-07-24T21:43:37.401641+0000","last_fullsized":"2024-07-24T21:43:37.401641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874339+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:43:37.874339+0000","last_peered":"2024-07-24T21:43:37.874339+0000","last_clean":"2024-07-24T21:43:37.874339+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:43:37.874339+0000","last_undegraded":"2024-07-24T21:43:37.874339+0000","last_fullsized":"2024-07-24T21:43:37.874339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.880007+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:43:37.880007+0000","last_peered":"2024-07-24T21:43:37.880007+0000","last_clean":"2024-07-24T21:43:37.880007+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:43:37.880007+0000","last_undegraded":"2024-07-24T21:43:37.880007+0000","last_fullsized":"2024-07-24T21:43:37.880007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402353+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:43:37.402353+0000","last_peered":"2024-07-24T21:43:37.402353+0000","last_clean":"2024-07-24T21:43:37.402353+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:43:37.402353+0000","last_undegraded":"2024-07-24T21:43:37.402353+0000","last_fullsized":"2024-07-24T21:43:37.402353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:43:37.873499+0000","last_peered":"2024-07-24T21:43:37.873499+0000","last_clean":"2024-07-24T21:43:37.873499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:43:37.873499+0000","last_undegraded":"2024-07-24T21:43:37.873499+0000","last_fullsized":"2024-07-24T21:43:37.873499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873272+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:43:37.873272+0000","last_peered":"2024-07-24T21:43:37.873272+0000","last_clean":"2024-07-24T21:43:37.873272+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:43:37.873272+0000","last_undegraded":"2024-07-24T21:43:37.873272+0000","last_fullsized":"2024-07-24T21:43:37.873272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.408020+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:43:37.408020+0000","last_peered":"2024-07-24T21:43:37.408020+0000","last_clean":"2024-07-24T21:43:37.408020+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:43:37.408020+0000","last_undegraded":"2024-07-24T21:43:37.408020+0000","last_fullsized":"2024-07-24T21:43:37.408020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58368,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":163,"num_read_kb":142,"num_write":160,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547222,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1109745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":463742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":463640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1544192,"data_stored":1055592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528365,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1556480,"data_stored":1054885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":463959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117051,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40428,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704724,"statfs":{"total":95995035648,"available":95953637376,"internally_reserved":0,"allocated":954368,"data_stored":463442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":518641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":991232,"data_stored":513034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":958464,"data_stored":464402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":55717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:43:56.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:43:56.649 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:56.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:56 smithi123 ceph-mon[37143]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:43:56.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:56 smithi060 ceph-mon[33000]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:43:56.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:56 smithi086 ceph-mon[37128]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:43:57.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:57 smithi060 ceph-mon[33000]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-07-24T21:43:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:57 smithi123 ceph-mon[37143]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-07-24T21:43:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:57 smithi086 ceph-mon[37128]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-07-24T21:43:58.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 -- 172.21.15.60:0/1545413563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd5010cb00 msgr2=0x7fcd5010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:58.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1545413563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd5010cb00 0x7fcd5010ef60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009ca0 tx=0x7fcd4802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 -- 172.21.15.60:0/1545413563 shutdown_connections 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1545413563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd5010cb00 0x7fcd5010ef60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1545413563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50103fc0 0x7fcd5010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1545413563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd501036a0 0x7fcd50103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 -- 172.21.15.60:0/1545413563 >> 172.21.15.60:0/1545413563 conn(0x7fcd500fd100 msgr2=0x7fcd500ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 -- 172.21.15.60:0/1545413563 shutdown_connections 2024-07-24T21:43:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.319+0000 7fcd5843d640 1 -- 172.21.15.60:0/1545413563 wait complete. 2024-07-24T21:43:58.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 Processor -- start 2024-07-24T21:43:58.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 -- start start 2024-07-24T21:43:58.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd501036a0 0x7fcd50101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:58.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 0x7fcd50101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501127f0 con 0x7fcd50103fc0 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50112670 con 0x7fcd501036a0 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd5843d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50112970 con 0x7fcd5010cb00 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd559d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd559d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53036/0 (socket says 172.21.15.60:53036) 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd559d8640 1 -- 172.21.15.60:0/1372534244 learned_addr learned my addr 172.21.15.60:0/1372534244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.320+0000 7fcd561d9640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd501036a0 0x7fcd50101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd569da640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 0x7fcd50101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 -- 172.21.15.60:0/1372534244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 msgr2=0x7fcd50101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:58.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 0x7fcd50101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 -- 172.21.15.60:0/1372534244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd501036a0 msgr2=0x7fcd50101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd501036a0 0x7fcd50101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 -- 172.21.15.60:0/1372534244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd501a70c0 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd569da640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 0x7fcd50101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd559d8640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b3c0 tx=0x7fcd4000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011830 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd40016070 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd501a73b0 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40010de0 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.321+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501a7970 con 0x7fcd50103fc0 2024-07-24T21:43:58.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.322+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd50103a80 con 0x7fcd50103fc0 2024-07-24T21:43:58.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.324+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fcd400119d0 con 0x7fcd50103fc0 2024-07-24T21:43:58.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.324+0000 7fcd477fe640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 0x7fcd3c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:43:58.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.325+0000 7fcd561d9640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 0x7fcd3c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:43:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.325+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7fcd400e3400 con 0x7fcd50103fc0 2024-07-24T21:43:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.325+0000 7fcd561d9640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 0x7fcd3c0b6170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005fd0 tx=0x7fcd4c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:43:58.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.329+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcd400aa920 con 0x7fcd50103fc0 2024-07-24T21:43:58.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.520+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd500fed10 con 0x7fcd3c0b3cb0 2024-07-24T21:43:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.524+0000 7fcd477fe640 1 -- 172.21.15.60:0/1372534244 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258701 (secure 0 0 0) 0x7fcd500fed10 con 0x7fcd3c0b3cb0 2024-07-24T21:43:58.528 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:43:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 msgr2=0x7fcd3c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 0x7fcd3c0b6170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005fd0 tx=0x7fcd4c00b390 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 msgr2=0x7fcd50101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b3c0 tx=0x7fcd4000b880 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 shutdown_connections 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd3c0b3cb0 0x7fcd3c0b6170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5010cb00 0x7fcd50101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd50103fc0 0x7fcd50101890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 --2- 172.21.15.60:0/1372534244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd501036a0 0x7fcd50101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.527+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 >> 172.21.15.60:0/1372534244 conn(0x7fcd500fd100 msgr2=0x7fcd5010ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:43:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.528+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 shutdown_connections 2024-07-24T21:43:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:43:58.528+0000 7fcd5843d640 1 -- 172.21.15.60:0/1372534244 wait complete. 2024-07-24T21:43:59.324 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-07-24T21:43:58.047011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654365,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":270,"num_read_kb":232,"num_write":288,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403844,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537980,"statfs":{"total":1151940427776,"available":1151526891520,"internally_reserved":0,"allocated":13377536,"data_stored":7512788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32934,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":79,"num_write":23,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003255"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404780+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:43:37.404780+0000","last_peered":"2024-07-24T21:43:37.404780+0000","last_clean":"2024-07-24T21:43:37.404780+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:43:37.404780+0000","last_undegraded":"2024-07-24T21:43:37.404780+0000","last_fullsized":"2024-07-24T21:43:37.404780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402109+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:43:37.402109+0000","last_peered":"2024-07-24T21:43:37.402109+0000","last_clean":"2024-07-24T21:43:37.402109+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:43:37.402109+0000","last_undegraded":"2024-07-24T21:43:37.402109+0000","last_fullsized":"2024-07-24T21:43:37.402109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402092+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:43:37.402092+0000","last_peered":"2024-07-24T21:43:37.402092+0000","last_clean":"2024-07-24T21:43:37.402092+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:43:37.402092+0000","last_undegraded":"2024-07-24T21:43:37.402092+0000","last_fullsized":"2024-07-24T21:43:37.402092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874608+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:43:37.874608+0000","last_peered":"2024-07-24T21:43:37.874608+0000","last_clean":"2024-07-24T21:43:37.874608+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:43:37.874608+0000","last_undegraded":"2024-07-24T21:43:37.874608+0000","last_fullsized":"2024-07-24T21:43:37.874608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401096+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:43:37.401096+0000","last_peered":"2024-07-24T21:43:37.401096+0000","last_clean":"2024-07-24T21:43:37.401096+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:43:37.401096+0000","last_undegraded":"2024-07-24T21:43:37.401096+0000","last_fullsized":"2024-07-24T21:43:37.401096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409471+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:43:37.409471+0000","last_peered":"2024-07-24T21:43:37.409471+0000","last_clean":"2024-07-24T21:43:37.409471+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:43:37.409471+0000","last_undegraded":"2024-07-24T21:43:37.409471+0000","last_fullsized":"2024-07-24T21:43:37.409471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400290+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:43:37.400290+0000","last_peered":"2024-07-24T21:43:37.400290+0000","last_clean":"2024-07-24T21:43:37.400290+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:43:37.400290+0000","last_undegraded":"2024-07-24T21:43:37.400290+0000","last_fullsized":"2024-07-24T21:43:37.400290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873092+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:43:37.873092+0000","last_peered":"2024-07-24T21:43:37.873092+0000","last_clean":"2024-07-24T21:43:37.873092+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:43:37.873092+0000","last_undegraded":"2024-07-24T21:43:37.873092+0000","last_fullsized":"2024-07-24T21:43:37.873092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874364+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:43:37.874364+0000","last_peered":"2024-07-24T21:43:37.874364+0000","last_clean":"2024-07-24T21:43:37.874364+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:43:37.874364+0000","last_undegraded":"2024-07-24T21:43:37.874364+0000","last_fullsized":"2024-07-24T21:43:37.874364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409431+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:43:37.409431+0000","last_peered":"2024-07-24T21:43:37.409431+0000","last_clean":"2024-07-24T21:43:37.409431+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:43:37.409431+0000","last_undegraded":"2024-07-24T21:43:37.409431+0000","last_fullsized":"2024-07-24T21:43:37.409431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400770+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:43:37.400770+0000","last_peered":"2024-07-24T21:43:37.400770+0000","last_clean":"2024-07-24T21:43:37.400770+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:43:37.400770+0000","last_undegraded":"2024-07-24T21:43:37.400770+0000","last_fullsized":"2024-07-24T21:43:37.400770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874612+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:43:37.874612+0000","last_peered":"2024-07-24T21:43:37.874612+0000","last_clean":"2024-07-24T21:43:37.874612+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:43:37.874612+0000","last_undegraded":"2024-07-24T21:43:37.874612+0000","last_fullsized":"2024-07-24T21:43:37.874612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402759+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:43:37.402759+0000","last_peered":"2024-07-24T21:43:37.402759+0000","last_clean":"2024-07-24T21:43:37.402759+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:43:37.402759+0000","last_undegraded":"2024-07-24T21:43:37.402759+0000","last_fullsized":"2024-07-24T21:43:37.402759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402021+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:43:37.402021+0000","last_peered":"2024-07-24T21:43:37.402021+0000","last_clean":"2024-07-24T21:43:37.402021+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:43:37.402021+0000","last_undegraded":"2024-07-24T21:43:37.402021+0000","last_fullsized":"2024-07-24T21:43:37.402021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401947+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:43:37.401947+0000","last_peered":"2024-07-24T21:43:37.401947+0000","last_clean":"2024-07-24T21:43:37.401947+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:43:37.401947+0000","last_undegraded":"2024-07-24T21:43:37.401947+0000","last_fullsized":"2024-07-24T21:43:37.401947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873505+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:43:37.873505+0000","last_peered":"2024-07-24T21:43:37.873505+0000","last_clean":"2024-07-24T21:43:37.873505+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:43:37.873505+0000","last_undegraded":"2024-07-24T21:43:37.873505+0000","last_fullsized":"2024-07-24T21:43:37.873505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404863+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:43:37.404863+0000","last_peered":"2024-07-24T21:43:37.404863+0000","last_clean":"2024-07-24T21:43:37.404863+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:43:37.404863+0000","last_undegraded":"2024-07-24T21:43:37.404863+0000","last_fullsized":"2024-07-24T21:43:37.404863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.407884+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:43:37.407884+0000","last_peered":"2024-07-24T21:43:37.407884+0000","last_clean":"2024-07-24T21:43:37.407884+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:43:37.407884+0000","last_undegraded":"2024-07-24T21:43:37.407884+0000","last_fullsized":"2024-07-24T21:43:37.407884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404348+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:43:37.404348+0000","last_peered":"2024-07-24T21:43:37.404348+0000","last_clean":"2024-07-24T21:43:37.404348+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:43:37.404348+0000","last_undegraded":"2024-07-24T21:43:37.404348+0000","last_fullsized":"2024-07-24T21:43:37.404348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404775+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:43:37.404775+0000","last_peered":"2024-07-24T21:43:37.404775+0000","last_clean":"2024-07-24T21:43:37.404775+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:43:37.404775+0000","last_undegraded":"2024-07-24T21:43:37.404775+0000","last_fullsized":"2024-07-24T21:43:37.404775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402794+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:43:37.402794+0000","last_peered":"2024-07-24T21:43:37.402794+0000","last_clean":"2024-07-24T21:43:37.402794+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:43:37.402794+0000","last_undegraded":"2024-07-24T21:43:37.402794+0000","last_fullsized":"2024-07-24T21:43:37.402794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873680+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:43:37.873680+0000","last_peered":"2024-07-24T21:43:37.873680+0000","last_clean":"2024-07-24T21:43:37.873680+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:43:37.873680+0000","last_undegraded":"2024-07-24T21:43:37.873680+0000","last_fullsized":"2024-07-24T21:43:37.873680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872834+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:43:37.872834+0000","last_peered":"2024-07-24T21:43:37.872834+0000","last_clean":"2024-07-24T21:43:37.872834+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:43:37.872834+0000","last_undegraded":"2024-07-24T21:43:37.872834+0000","last_fullsized":"2024-07-24T21:43:37.872834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409464+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:43:37.409464+0000","last_peered":"2024-07-24T21:43:37.409464+0000","last_clean":"2024-07-24T21:43:37.409464+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:43:37.409464+0000","last_undegraded":"2024-07-24T21:43:37.409464+0000","last_fullsized":"2024-07-24T21:43:37.409464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873302+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:43:37.873302+0000","last_peered":"2024-07-24T21:43:37.873302+0000","last_clean":"2024-07-24T21:43:37.873302+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:43:37.873302+0000","last_undegraded":"2024-07-24T21:43:37.873302+0000","last_fullsized":"2024-07-24T21:43:37.873302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404573+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:43:37.404573+0000","last_peered":"2024-07-24T21:43:37.404573+0000","last_clean":"2024-07-24T21:43:37.404573+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:43:37.404573+0000","last_undegraded":"2024-07-24T21:43:37.404573+0000","last_fullsized":"2024-07-24T21:43:37.404573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.405266+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:43:37.405266+0000","last_peered":"2024-07-24T21:43:37.405266+0000","last_clean":"2024-07-24T21:43:37.405266+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:43:37.405266+0000","last_undegraded":"2024-07-24T21:43:37.405266+0000","last_fullsized":"2024-07-24T21:43:37.405266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402909+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:43:37.402909+0000","last_peered":"2024-07-24T21:43:37.402909+0000","last_clean":"2024-07-24T21:43:37.402909+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:43:37.402909+0000","last_undegraded":"2024-07-24T21:43:37.402909+0000","last_fullsized":"2024-07-24T21:43:37.402909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402344+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:43:37.402344+0000","last_peered":"2024-07-24T21:43:37.402344+0000","last_clean":"2024-07-24T21:43:37.402344+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:43:37.402344+0000","last_undegraded":"2024-07-24T21:43:37.402344+0000","last_fullsized":"2024-07-24T21:43:37.402344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403560+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:43:37.403560+0000","last_peered":"2024-07-24T21:43:37.403560+0000","last_clean":"2024-07-24T21:43:37.403560+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:43:37.403560+0000","last_undegraded":"2024-07-24T21:43:37.403560+0000","last_fullsized":"2024-07-24T21:43:37.403560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.875378+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:43:37.875378+0000","last_peered":"2024-07-24T21:43:37.875378+0000","last_clean":"2024-07-24T21:43:37.875378+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:43:37.875378+0000","last_undegraded":"2024-07-24T21:43:37.875378+0000","last_fullsized":"2024-07-24T21:43:37.875378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404358+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:43:37.404358+0000","last_peered":"2024-07-24T21:43:37.404358+0000","last_clean":"2024-07-24T21:43:37.404358+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:43:37.404358+0000","last_undegraded":"2024-07-24T21:43:37.404358+0000","last_fullsized":"2024-07-24T21:43:37.404358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401138+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:43:37.401138+0000","last_peered":"2024-07-24T21:43:37.401138+0000","last_clean":"2024-07-24T21:43:37.401138+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:43:37.401138+0000","last_undegraded":"2024-07-24T21:43:37.401138+0000","last_fullsized":"2024-07-24T21:43:37.401138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:52.413864+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:43:52.413864+0000","last_peered":"2024-07-24T21:43:52.413864+0000","last_clean":"2024-07-24T21:43:52.413864+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:43:52.413864+0000","last_undegraded":"2024-07-24T21:43:52.413864+0000","last_fullsized":"2024-07-24T21:43:52.413864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399997+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:43:37.399997+0000","last_peered":"2024-07-24T21:43:37.399997+0000","last_clean":"2024-07-24T21:43:37.399997+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:43:37.399997+0000","last_undegraded":"2024-07-24T21:43:37.399997+0000","last_fullsized":"2024-07-24T21:43:37.399997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403801+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:43:37.403801+0000","last_peered":"2024-07-24T21:43:37.403801+0000","last_clean":"2024-07-24T21:43:37.403801+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:43:37.403801+0000","last_undegraded":"2024-07-24T21:43:37.403801+0000","last_fullsized":"2024-07-24T21:43:37.403801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402472+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:43:37.402472+0000","last_peered":"2024-07-24T21:43:37.402472+0000","last_clean":"2024-07-24T21:43:37.402472+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:43:37.402472+0000","last_undegraded":"2024-07-24T21:43:37.402472+0000","last_fullsized":"2024-07-24T21:43:37.402472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872913+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:43:37.872913+0000","last_peered":"2024-07-24T21:43:37.872913+0000","last_clean":"2024-07-24T21:43:37.872913+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:43:37.872913+0000","last_undegraded":"2024-07-24T21:43:37.872913+0000","last_fullsized":"2024-07-24T21:43:37.872913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400758+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:43:37.400758+0000","last_peered":"2024-07-24T21:43:37.400758+0000","last_clean":"2024-07-24T21:43:37.400758+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:43:37.400758+0000","last_undegraded":"2024-07-24T21:43:37.400758+0000","last_fullsized":"2024-07-24T21:43:37.400758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.407915+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:43:37.407915+0000","last_peered":"2024-07-24T21:43:37.407915+0000","last_clean":"2024-07-24T21:43:37.407915+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:43:37.407915+0000","last_undegraded":"2024-07-24T21:43:37.407915+0000","last_fullsized":"2024-07-24T21:43:37.407915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:54.416428+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:43:54.416428+0000","last_peered":"2024-07-24T21:43:54.416428+0000","last_clean":"2024-07-24T21:43:54.416428+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:43:54.416428+0000","last_undegraded":"2024-07-24T21:43:54.416428+0000","last_fullsized":"2024-07-24T21:43:54.416428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873995+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:43:37.873995+0000","last_peered":"2024-07-24T21:43:37.873995+0000","last_clean":"2024-07-24T21:43:37.873995+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:43:37.873995+0000","last_undegraded":"2024-07-24T21:43:37.873995+0000","last_fullsized":"2024-07-24T21:43:37.873995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400808+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:43:37.400808+0000","last_peered":"2024-07-24T21:43:37.400808+0000","last_clean":"2024-07-24T21:43:37.400808+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:43:37.400808+0000","last_undegraded":"2024-07-24T21:43:37.400808+0000","last_fullsized":"2024-07-24T21:43:37.400808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404982+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:43:37.404982+0000","last_peered":"2024-07-24T21:43:37.404982+0000","last_clean":"2024-07-24T21:43:37.404982+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:43:37.404982+0000","last_undegraded":"2024-07-24T21:43:37.404982+0000","last_fullsized":"2024-07-24T21:43:37.404982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400259+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:43:37.400259+0000","last_peered":"2024-07-24T21:43:37.400259+0000","last_clean":"2024-07-24T21:43:37.400259+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:43:37.400259+0000","last_undegraded":"2024-07-24T21:43:37.400259+0000","last_fullsized":"2024-07-24T21:43:37.400259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874272+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:43:37.874272+0000","last_peered":"2024-07-24T21:43:37.874272+0000","last_clean":"2024-07-24T21:43:37.874272+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:43:37.874272+0000","last_undegraded":"2024-07-24T21:43:37.874272+0000","last_fullsized":"2024-07-24T21:43:37.874272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":117,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:55.422538+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:43:55.422538+0000","last_peered":"2024-07-24T21:43:55.422538+0000","last_clean":"2024-07-24T21:43:55.422538+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:43:55.422538+0000","last_undegraded":"2024-07-24T21:43:55.422538+0000","last_fullsized":"2024-07-24T21:43:55.422538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":89,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401614+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:43:37.401614+0000","last_peered":"2024-07-24T21:43:37.401614+0000","last_clean":"2024-07-24T21:43:37.401614+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:43:37.401614+0000","last_undegraded":"2024-07-24T21:43:37.401614+0000","last_fullsized":"2024-07-24T21:43:37.401614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403946+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:43:37.403946+0000","last_peered":"2024-07-24T21:43:37.403946+0000","last_clean":"2024-07-24T21:43:37.403946+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:43:37.403946+0000","last_undegraded":"2024-07-24T21:43:37.403946+0000","last_fullsized":"2024-07-24T21:43:37.403946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.409460+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:43:37.409460+0000","last_peered":"2024-07-24T21:43:37.409460+0000","last_clean":"2024-07-24T21:43:37.409460+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:43:37.409460+0000","last_undegraded":"2024-07-24T21:43:37.409460+0000","last_fullsized":"2024-07-24T21:43:37.409460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399962+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:43:37.399962+0000","last_peered":"2024-07-24T21:43:37.399962+0000","last_clean":"2024-07-24T21:43:37.399962+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:43:37.399962+0000","last_undegraded":"2024-07-24T21:43:37.399962+0000","last_fullsized":"2024-07-24T21:43:37.399962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400532+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:43:37.400532+0000","last_peered":"2024-07-24T21:43:37.400532+0000","last_clean":"2024-07-24T21:43:37.400532+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:43:37.400532+0000","last_undegraded":"2024-07-24T21:43:37.400532+0000","last_fullsized":"2024-07-24T21:43:37.400532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:55.431078+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:43:55.431078+0000","last_peered":"2024-07-24T21:43:55.431078+0000","last_clean":"2024-07-24T21:43:55.431078+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:43:55.431078+0000","last_undegraded":"2024-07-24T21:43:55.431078+0000","last_fullsized":"2024-07-24T21:43:55.431078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400223+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:43:37.400223+0000","last_peered":"2024-07-24T21:43:37.400223+0000","last_clean":"2024-07-24T21:43:37.400223+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:43:37.400223+0000","last_undegraded":"2024-07-24T21:43:37.400223+0000","last_fullsized":"2024-07-24T21:43:37.400223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401671+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:43:37.401671+0000","last_peered":"2024-07-24T21:43:37.401671+0000","last_clean":"2024-07-24T21:43:37.401671+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:43:37.401671+0000","last_undegraded":"2024-07-24T21:43:37.401671+0000","last_fullsized":"2024-07-24T21:43:37.401671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.399907+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:43:37.399907+0000","last_peered":"2024-07-24T21:43:37.399907+0000","last_clean":"2024-07-24T21:43:37.399907+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:43:37.399907+0000","last_undegraded":"2024-07-24T21:43:37.399907+0000","last_fullsized":"2024-07-24T21:43:37.399907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401927+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:43:37.401927+0000","last_peered":"2024-07-24T21:43:37.401927+0000","last_clean":"2024-07-24T21:43:37.401927+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:43:37.401927+0000","last_undegraded":"2024-07-24T21:43:37.401927+0000","last_fullsized":"2024-07-24T21:43:37.401927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404643+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:43:37.404643+0000","last_peered":"2024-07-24T21:43:37.404643+0000","last_clean":"2024-07-24T21:43:37.404643+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:43:37.404643+0000","last_undegraded":"2024-07-24T21:43:37.404643+0000","last_fullsized":"2024-07-24T21:43:37.404643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403044+0000","last_change":"2024-07-24T21:43:09.255779+0000","last_active":"2024-07-24T21:43:37.403044+0000","last_peered":"2024-07-24T21:43:37.403044+0000","last_clean":"2024-07-24T21:43:37.403044+0000","last_became_active":"2024-07-24T21:43:09.255448+0000","last_became_peered":"2024-07-24T21:43:09.255448+0000","last_unstale":"2024-07-24T21:43:37.403044+0000","last_undegraded":"2024-07-24T21:43:37.403044+0000","last_fullsized":"2024-07-24T21:43:37.403044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:13:05.781739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400808+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:43:37.400808+0000","last_peered":"2024-07-24T21:43:37.400808+0000","last_clean":"2024-07-24T21:43:37.400808+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:43:37.400808+0000","last_undegraded":"2024-07-24T21:43:37.400808+0000","last_fullsized":"2024-07-24T21:43:37.400808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400390+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:43:37.400390+0000","last_peered":"2024-07-24T21:43:37.400390+0000","last_clean":"2024-07-24T21:43:37.400390+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:43:37.400390+0000","last_undegraded":"2024-07-24T21:43:37.400390+0000","last_fullsized":"2024-07-24T21:43:37.400390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400668+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:43:37.400668+0000","last_peered":"2024-07-24T21:43:37.400668+0000","last_clean":"2024-07-24T21:43:37.400668+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:43:37.400668+0000","last_undegraded":"2024-07-24T21:43:37.400668+0000","last_fullsized":"2024-07-24T21:43:37.400668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404031+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:43:37.404031+0000","last_peered":"2024-07-24T21:43:37.404031+0000","last_clean":"2024-07-24T21:43:37.404031+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:43:37.404031+0000","last_undegraded":"2024-07-24T21:43:37.404031+0000","last_fullsized":"2024-07-24T21:43:37.404031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400776+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:43:37.400776+0000","last_peered":"2024-07-24T21:43:37.400776+0000","last_clean":"2024-07-24T21:43:37.400776+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:43:37.400776+0000","last_undegraded":"2024-07-24T21:43:37.400776+0000","last_fullsized":"2024-07-24T21:43:37.400776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400392+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:43:37.400392+0000","last_peered":"2024-07-24T21:43:37.400392+0000","last_clean":"2024-07-24T21:43:37.400392+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:43:37.400392+0000","last_undegraded":"2024-07-24T21:43:37.400392+0000","last_fullsized":"2024-07-24T21:43:37.400392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874347+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:43:37.874347+0000","last_peered":"2024-07-24T21:43:37.874347+0000","last_clean":"2024-07-24T21:43:37.874347+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:43:37.874347+0000","last_undegraded":"2024-07-24T21:43:37.874347+0000","last_fullsized":"2024-07-24T21:43:37.874347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873011+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:43:37.873011+0000","last_peered":"2024-07-24T21:43:37.873011+0000","last_clean":"2024-07-24T21:43:37.873011+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:43:37.873011+0000","last_undegraded":"2024-07-24T21:43:37.873011+0000","last_fullsized":"2024-07-24T21:43:37.873011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400387+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:43:37.400387+0000","last_peered":"2024-07-24T21:43:37.400387+0000","last_clean":"2024-07-24T21:43:37.400387+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:43:37.400387+0000","last_undegraded":"2024-07-24T21:43:37.400387+0000","last_fullsized":"2024-07-24T21:43:37.400387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402349+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:43:37.402349+0000","last_peered":"2024-07-24T21:43:37.402349+0000","last_clean":"2024-07-24T21:43:37.402349+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:43:37.402349+0000","last_undegraded":"2024-07-24T21:43:37.402349+0000","last_fullsized":"2024-07-24T21:43:37.402349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:56.431944+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:43:56.431944+0000","last_peered":"2024-07-24T21:43:56.431944+0000","last_clean":"2024-07-24T21:43:56.431944+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:43:56.431944+0000","last_undegraded":"2024-07-24T21:43:56.431944+0000","last_fullsized":"2024-07-24T21:43:56.431944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.405723+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:43:37.405723+0000","last_peered":"2024-07-24T21:43:37.405723+0000","last_clean":"2024-07-24T21:43:37.405723+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:43:37.405723+0000","last_undegraded":"2024-07-24T21:43:37.405723+0000","last_fullsized":"2024-07-24T21:43:37.405723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400582+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:43:37.400582+0000","last_peered":"2024-07-24T21:43:37.400582+0000","last_clean":"2024-07-24T21:43:37.400582+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:43:37.400582+0000","last_undegraded":"2024-07-24T21:43:37.400582+0000","last_fullsized":"2024-07-24T21:43:37.400582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403924+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:43:37.403924+0000","last_peered":"2024-07-24T21:43:37.403924+0000","last_clean":"2024-07-24T21:43:37.403924+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:43:37.403924+0000","last_undegraded":"2024-07-24T21:43:37.403924+0000","last_fullsized":"2024-07-24T21:43:37.403924+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.408073+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:43:37.408073+0000","last_peered":"2024-07-24T21:43:37.408073+0000","last_clean":"2024-07-24T21:43:37.408073+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:43:37.408073+0000","last_undegraded":"2024-07-24T21:43:37.408073+0000","last_fullsized":"2024-07-24T21:43:37.408073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404089+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:43:37.404089+0000","last_peered":"2024-07-24T21:43:37.404089+0000","last_clean":"2024-07-24T21:43:37.404089+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:43:37.404089+0000","last_undegraded":"2024-07-24T21:43:37.404089+0000","last_fullsized":"2024-07-24T21:43:37.404089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.412278+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:43:37.412278+0000","last_peered":"2024-07-24T21:43:37.412278+0000","last_clean":"2024-07-24T21:43:37.412278+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:43:37.412278+0000","last_undegraded":"2024-07-24T21:43:37.412278+0000","last_fullsized":"2024-07-24T21:43:37.412278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402915+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:43:37.402915+0000","last_peered":"2024-07-24T21:43:37.402915+0000","last_clean":"2024-07-24T21:43:37.402915+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:43:37.402915+0000","last_undegraded":"2024-07-24T21:43:37.402915+0000","last_fullsized":"2024-07-24T21:43:37.402915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:49.301112+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:43:49.301112+0000","last_peered":"2024-07-24T21:43:49.301112+0000","last_clean":"2024-07-24T21:43:49.301112+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:43:49.301112+0000","last_undegraded":"2024-07-24T21:43:49.301112+0000","last_fullsized":"2024-07-24T21:43:49.301112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873389+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:43:37.873389+0000","last_peered":"2024-07-24T21:43:37.873389+0000","last_clean":"2024-07-24T21:43:37.873389+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:43:37.873389+0000","last_undegraded":"2024-07-24T21:43:37.873389+0000","last_fullsized":"2024-07-24T21:43:37.873389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874876+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:43:37.874876+0000","last_peered":"2024-07-24T21:43:37.874876+0000","last_clean":"2024-07-24T21:43:37.874876+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:43:37.874876+0000","last_undegraded":"2024-07-24T21:43:37.874876+0000","last_fullsized":"2024-07-24T21:43:37.874876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402421+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:43:37.402421+0000","last_peered":"2024-07-24T21:43:37.402421+0000","last_clean":"2024-07-24T21:43:37.402421+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:43:37.402421+0000","last_undegraded":"2024-07-24T21:43:37.402421+0000","last_fullsized":"2024-07-24T21:43:37.402421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873657+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:43:37.873657+0000","last_peered":"2024-07-24T21:43:37.873657+0000","last_clean":"2024-07-24T21:43:37.873657+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:43:37.873657+0000","last_undegraded":"2024-07-24T21:43:37.873657+0000","last_fullsized":"2024-07-24T21:43:37.873657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400895+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:43:37.400895+0000","last_peered":"2024-07-24T21:43:37.400895+0000","last_clean":"2024-07-24T21:43:37.400895+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:43:37.400895+0000","last_undegraded":"2024-07-24T21:43:37.400895+0000","last_fullsized":"2024-07-24T21:43:37.400895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872966+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:43:37.872966+0000","last_peered":"2024-07-24T21:43:37.872966+0000","last_clean":"2024-07-24T21:43:37.872966+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:43:37.872966+0000","last_undegraded":"2024-07-24T21:43:37.872966+0000","last_fullsized":"2024-07-24T21:43:37.872966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.404498+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:43:37.404498+0000","last_peered":"2024-07-24T21:43:37.404498+0000","last_clean":"2024-07-24T21:43:37.404498+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:43:37.404498+0000","last_undegraded":"2024-07-24T21:43:37.404498+0000","last_fullsized":"2024-07-24T21:43:37.404498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.403660+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:43:37.403660+0000","last_peered":"2024-07-24T21:43:37.403660+0000","last_clean":"2024-07-24T21:43:37.403660+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:43:37.403660+0000","last_undegraded":"2024-07-24T21:43:37.403660+0000","last_fullsized":"2024-07-24T21:43:37.403660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.872939+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:43:37.872939+0000","last_peered":"2024-07-24T21:43:37.872939+0000","last_clean":"2024-07-24T21:43:37.872939+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:43:37.872939+0000","last_undegraded":"2024-07-24T21:43:37.872939+0000","last_fullsized":"2024-07-24T21:43:37.872939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401194+0000","last_change":"2024-07-24T21:43:09.256542+0000","last_active":"2024-07-24T21:43:37.401194+0000","last_peered":"2024-07-24T21:43:37.401194+0000","last_clean":"2024-07-24T21:43:37.401194+0000","last_became_active":"2024-07-24T21:43:09.256187+0000","last_became_peered":"2024-07-24T21:43:09.256187+0000","last_unstale":"2024-07-24T21:43:37.401194+0000","last_undegraded":"2024-07-24T21:43:37.401194+0000","last_fullsized":"2024-07-24T21:43:37.401194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:20:32.874780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400438+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:43:37.400438+0000","last_peered":"2024-07-24T21:43:37.400438+0000","last_clean":"2024-07-24T21:43:37.400438+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:43:37.400438+0000","last_undegraded":"2024-07-24T21:43:37.400438+0000","last_fullsized":"2024-07-24T21:43:37.400438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.400118+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:43:37.400118+0000","last_peered":"2024-07-24T21:43:37.400118+0000","last_clean":"2024-07-24T21:43:37.400118+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:43:37.400118+0000","last_undegraded":"2024-07-24T21:43:37.400118+0000","last_fullsized":"2024-07-24T21:43:37.400118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.401641+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:43:37.401641+0000","last_peered":"2024-07-24T21:43:37.401641+0000","last_clean":"2024-07-24T21:43:37.401641+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:43:37.401641+0000","last_undegraded":"2024-07-24T21:43:37.401641+0000","last_fullsized":"2024-07-24T21:43:37.401641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.874339+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:43:37.874339+0000","last_peered":"2024-07-24T21:43:37.874339+0000","last_clean":"2024-07-24T21:43:37.874339+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:43:37.874339+0000","last_undegraded":"2024-07-24T21:43:37.874339+0000","last_fullsized":"2024-07-24T21:43:37.874339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.880007+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:43:37.880007+0000","last_peered":"2024-07-24T21:43:37.880007+0000","last_clean":"2024-07-24T21:43:37.880007+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:43:37.880007+0000","last_undegraded":"2024-07-24T21:43:37.880007+0000","last_fullsized":"2024-07-24T21:43:37.880007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.402353+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:43:37.402353+0000","last_peered":"2024-07-24T21:43:37.402353+0000","last_clean":"2024-07-24T21:43:37.402353+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:43:37.402353+0000","last_undegraded":"2024-07-24T21:43:37.402353+0000","last_fullsized":"2024-07-24T21:43:37.402353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:43:37.873499+0000","last_peered":"2024-07-24T21:43:37.873499+0000","last_clean":"2024-07-24T21:43:37.873499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:43:37.873499+0000","last_undegraded":"2024-07-24T21:43:37.873499+0000","last_fullsized":"2024-07-24T21:43:37.873499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.873272+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:43:37.873272+0000","last_peered":"2024-07-24T21:43:37.873272+0000","last_clean":"2024-07-24T21:43:37.873272+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:43:37.873272+0000","last_undegraded":"2024-07-24T21:43:37.873272+0000","last_fullsized":"2024-07-24T21:43:37.873272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-07-24T21:43:37.408020+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:43:37.408020+0000","last_peered":"2024-07-24T21:43:37.408020+0000","last_clean":"2024-07-24T21:43:37.408020+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:43:37.408020+0000","last_undegraded":"2024-07-24T21:43:37.408020+0000","last_fullsized":"2024-07-24T21:43:37.408020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63763,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":189,"num_read_kb":163,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547223,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743452,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1115140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":463742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":463640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1544192,"data_stored":1055592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1556480,"data_stored":1054885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":463959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40428,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704724,"statfs":{"total":95995035648,"available":95953637376,"internally_reserved":0,"allocated":954368,"data_stored":463442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":524036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":517291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40436,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704716,"statfs":{"total":95995035648,"available":95953629184,"internally_reserved":0,"allocated":958464,"data_stored":464402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:43:59.328 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-24T21:43:59.329 INFO:tasks.cephadm:Setting up client nodes... 2024-07-24T21:43:59.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-24T21:43:59.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:59 smithi060 ceph-mon[33000]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-07-24T21:43:59.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:43:59 smithi060 ceph-mon[33000]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:43:59.599 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:43:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:59 smithi123 ceph-mon[37143]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-07-24T21:43:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:43:59 smithi123 ceph-mon[37143]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:43:59.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:59 smithi086 ceph-mon[37128]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-07-24T21:43:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:43:59 smithi086 ceph-mon[37128]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 -- 172.21.15.60:0/61791142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410cc20 msgr2=0x7f3a0410f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 --2- 172.21.15.60:0/61791142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410cc20 0x7f3a0410f010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009ca0 tx=0x7f39fc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 -- 172.21.15.60:0/61791142 shutdown_connections 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 --2- 172.21.15.60:0/61791142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410cc20 0x7f3a0410f010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 --2- 172.21.15.60:0/61791142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410a2f0 0x7f3a0410c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 --2- 172.21.15.60:0/61791142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 0x7f3a041015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 -- 172.21.15.60:0/61791142 >> 172.21.15.60:0/61791142 conn(0x7f3a04076940 msgr2=0x7f3a04076d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 -- 172.21.15.60:0/61791142 shutdown_connections 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.395+0000 7f3a0a713640 1 -- 172.21.15.60:0/61791142 wait complete. 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 Processor -- start 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 -- start start 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 0x7f3a041021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410cc20 0x7f3a04102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112710 con 0x7f3a0410a2f0 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112590 con 0x7f3a041011e0 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a0a713640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112890 con 0x7f3a0410cc20 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53056/0 (socket says 172.21.15.60:53056) 2024-07-24T21:44:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 -- 172.21.15.60:0/2630620070 learned_addr learned my addr 172.21.15.60:0/2630620070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a03fff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 0x7f3a041021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f3a08cb0640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410cc20 0x7f3a04102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 -- 172.21.15.60:0/2630620070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410cc20 msgr2=0x7f3a04102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410cc20 0x7f3a04102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 -- 172.21.15.60:0/2630620070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 msgr2=0x7f3a041021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 0x7f3a041021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.396+0000 7f39fbfff640 1 -- 172.21.15.60:0/2630620070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041a7060 con 0x7f3a0410a2f0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f39fbfff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d950 tx=0x7f39ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec017030 con 0x7f3a0410a2f0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39ec004020 con 0x7f3a0410a2f0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a041a7350 con 0x7f3a0410a2f0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0047a0 con 0x7f3a0410a2f0 2024-07-24T21:44:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.397+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a7890 con 0x7f3a0410a2f0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.398+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a041015c0 con 0x7f3a0410a2f0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.400+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f39ec009a30 con 0x7f3a0410a2f0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.401+0000 7f3a01ffb640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 0x7f39f00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.401+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f39ec0e3f20 con 0x7f3a0410a2f0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.401+0000 7f3a03fff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 0x7f39f00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.402+0000 7f3a03fff640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 0x7f39f00b6170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f40058c0 tx=0x7f39f400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.404+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f39ec0ab2d0 con 0x7f3a0410a2f0 2024-07-24T21:44:01.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.677+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3a041034b0 con 0x7f3a0410a2f0 2024-07-24T21:44:01.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:01 smithi060 ceph-mon[33000]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-07-24T21:44:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.679+0000 7f3a01ffb640 1 -- 172.21.15.60:0/2630620070 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f39ec0aabd0 con 0x7f3a0410a2f0 2024-07-24T21:44:01.680 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-07-24T21:44:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQChdaFmhvFwKBAAC672/KgWOaqGjTLW5E5vKw== 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.681+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 msgr2=0x7f39f00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.681+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 0x7f39f00b6170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f40058c0 tx=0x7f39f400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.681+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 msgr2=0x7f3a041026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.681+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d950 tx=0x7f39ec00de10 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 shutdown_connections 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f39f00b3cb0 0x7f39f00b6170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a0410cc20 0x7f3a04102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a0410a2f0 0x7f3a041026f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 --2- 172.21.15.60:0/2630620070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a041011e0 0x7f3a041021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 >> 172.21.15.60:0/2630620070 conn(0x7f3a04076940 msgr2=0x7f3a040fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 shutdown_connections 2024-07-24T21:44:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:01.682+0000 7f3a0a713640 1 -- 172.21.15.60:0/2630620070 wait complete. 2024-07-24T21:44:01.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:01 smithi123 ceph-mon[37143]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-07-24T21:44:01.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:01 smithi086 ceph-mon[37128]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-07-24T21:44:02.481 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:44:02.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-24T21:44:02.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-24T21:44:02.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph config log 1 --format=json 2024-07-24T21:44:02.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:02 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-24T21:44:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:02 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-24T21:44:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:02 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-24T21:44:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:02 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-24T21:44:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:02 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-24T21:44:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:02 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2630620070' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-24T21:44:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:03.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:03 smithi060 ceph-mon[33000]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-07-24T21:44:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:03 smithi123 ceph-mon[37143]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-07-24T21:44:04.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:03 smithi086 ceph-mon[37128]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-07-24T21:44:04.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 -- 172.21.15.60:0/1152867795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103f60 msgr2=0x7f35b010c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 --2- 172.21.15.60:0/1152867795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103f60 0x7f35b010c540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802a410 comp rx=0 tx=0).stop 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 -- 172.21.15.60:0/1152867795 shutdown_connections 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 --2- 172.21.15.60:0/1152867795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 --2- 172.21.15.60:0/1152867795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103f60 0x7f35b010c540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 --2- 172.21.15.60:0/1152867795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103560 0x7f35b0103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 -- 172.21.15.60:0/1152867795 >> 172.21.15.60:0/1152867795 conn(0x7f35b00fd0f0 msgr2=0x7f35b00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 -- 172.21.15.60:0/1152867795 shutdown_connections 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.562+0000 7f35b6776640 1 -- 172.21.15.60:0/1152867795 wait complete. 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 Processor -- start 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 -- start start 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:04.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 0x7f35b0101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b0101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0112750 con 0x7f35b0103560 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01125d0 con 0x7f35b0103f60 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35b6776640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01128d0 con 0x7f35b010cb10 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53088/0 (socket says 172.21.15.60:53088) 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.563+0000 7f35affff640 1 -- 172.21.15.60:0/762727543 learned_addr learned my addr 172.21.15.60:0/762727543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35a7fff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 0x7f35b0101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 -- 172.21.15.60:0/762727543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 msgr2=0x7f35b0101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35b4d13640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b0101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b0101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 -- 172.21.15.60:0/762727543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 msgr2=0x7f35b0101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 0x7f35b0101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 -- 172.21.15.60:0/762727543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01a7080 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35a7fff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 0x7f35b0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35b4d13640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35affff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c890 tx=0x7f35a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0004220 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a00043c0 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01a7370 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0004a00 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.564+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a7900 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.565+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b0103940 con 0x7f35b0103560 2024-07-24T21:44:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.566+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f35a00079b0 con 0x7f35b0103560 2024-07-24T21:44:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.567+0000 7f35adffb640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 0x7f359c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.568+0000 7f35a7fff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 0x7f359c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.568+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f35a00e4040 con 0x7f35b0103560 2024-07-24T21:44:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.568+0000 7f35a7fff640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 0x7f359c0b61f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f3598031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.570+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f35a0014030 con 0x7f35b0103560 2024-07-24T21:44:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.766+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f35b002e000 con 0x7f35b0103560 2024-07-24T21:44:04.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.767+0000 7f35adffb640 1 -- 172.21.15.60:0/762727543 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f35a00ab4d0 con 0x7f35b0103560 2024-07-24T21:44:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:44:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.768+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 msgr2=0x7f359c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.768+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 0x7f359c0b61f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f3598031400 comp rx=0 tx=0).stop 2024-07-24T21:44:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.768+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 msgr2=0x7f35b01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.768+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c890 tx=0x7f35a000cd50 comp rx=0 tx=0).stop 2024-07-24T21:44:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 shutdown_connections 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f359c0b3d30 0x7f359c0b61f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35b010cb10 0x7f35b0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35b0103f60 0x7f35b0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 --2- 172.21.15.60:0/762727543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b0103560 0x7f35b01013e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 >> 172.21.15.60:0/762727543 conn(0x7f35b00fd0f0 msgr2=0x7f35b00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 shutdown_connections 2024-07-24T21:44:04.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:04.769+0000 7f35b6776640 1 -- 172.21.15.60:0/762727543 wait complete. 2024-07-24T21:44:05.559 INFO:teuthology.orchestra.run.smithi060.stdout:[{"version":55,"timestamp":"2024-07-24T21:42:59.343510+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-07-24T21:44:05.559 INFO:tasks.ceph_manager:config epoch is 55 2024-07-24T21:44:05.559 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-24T21:44:05.559 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-24T21:44:05.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph mgr dump --format=json 2024-07-24T21:44:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:05 smithi060 ceph-mon[33000]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:44:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:05 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/762727543' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-24T21:44:05.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:05 smithi123 ceph-mon[37143]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:44:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:05 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/762727543' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-24T21:44:06.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:05 smithi086 ceph-mon[37128]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:44:06.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:05 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/762727543' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-24T21:44:06.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:06 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:44:06.970 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:06 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:44:06.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:06 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:44:07.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 -- 172.21.15.60:0/1129461241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6000108b20 msgr2=0x7f600010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 --2- 172.21.15.60:0/1129461241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6000108b20 0x7f600010f320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009900 tx=0x7f5ff802a410 comp rx=0 tx=0).stop 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 -- 172.21.15.60:0/1129461241 shutdown_connections 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 --2- 172.21.15.60:0/1129461241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6000108b20 0x7f600010f320 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 --2- 172.21.15.60:0/1129461241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60001081a0 0x7f60001085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 --2- 172.21.15.60:0/1129461241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 0x7f6000102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 -- 172.21.15.60:0/1129461241 >> 172.21.15.60:0/1129461241 conn(0x7f60000fe200 msgr2=0x7f6000100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 -- 172.21.15.60:0/1129461241 shutdown_connections 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.509+0000 7f60076f1640 1 -- 172.21.15.60:0/1129461241 wait complete. 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 Processor -- start 2024-07-24T21:44:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 -- start start 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 0x7f6000074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000108b20 0x7f6000075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6000112790 con 0x7f60001081a0 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6000112610 con 0x7f6000102790 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f60076f1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6000112910 con 0x7f6000108b20 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f6004c8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f6004c8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60354/0 (socket says 172.21.15.60:60354) 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f6004c8c640 1 -- 172.21.15.60:0/717544218 learned_addr learned my addr 172.21.15.60:0/717544218 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f600548d640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 0x7f6000074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.510+0000 7f6005c8e640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000108b20 0x7f6000075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 -- 172.21.15.60:0/717544218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000108b20 msgr2=0x7f6000075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000108b20 0x7f6000075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 -- 172.21.15.60:0/717544218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 msgr2=0x7f6000074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 0x7f6000074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 -- 172.21.15.60:0/717544218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60001a7000 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f6004c8c640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400d950 tx=0x7f5ff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4017070 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff4009c30 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60001a72f0 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400f620 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.511+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60001a7880 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.512+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6000102b70 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.513+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f5ff4010460 con 0x7f60001081a0 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.515+0000 7f5ffe7fc640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 0x7f5fe80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.515+0000 7f600548d640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 0x7f5fe80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:07.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.515+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f5ff40e3a20 con 0x7f60001081a0 2024-07-24T21:44:07.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.515+0000 7f600548d640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 0x7f5fe80b61f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005fd0 tx=0x7f5ff0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:07.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.518+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5ff4009de0 con 0x7f60001081a0 2024-07-24T21:44:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.779+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f600002e000 con 0x7f60001081a0 2024-07-24T21:44:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.784+0000 7f5ffe7fc640 1 -- 172.21.15.60:0/717544218 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+291034 (secure 0 0 0) 0x7f5ff40b3120 con 0x7f60001081a0 2024-07-24T21:44:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:44:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 msgr2=0x7f5fe80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 0x7f5fe80b61f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005fd0 tx=0x7f5ff0009500 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 msgr2=0x7f60000753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400d950 tx=0x7f5ff400de10 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 shutdown_connections 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5fe80b3d30 0x7f5fe80b61f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000108b20 0x7f6000075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60001081a0 0x7f60000753d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 --2- 172.21.15.60:0/717544218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6000102790 0x7f6000074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 >> 172.21.15.60:0/717544218 conn(0x7f60000fe200 msgr2=0x7f60001005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 shutdown_connections 2024-07-24T21:44:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:07.787+0000 7f60076f1640 1 -- 172.21.15.60:0/717544218 wait complete. 2024-07-24T21:44:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:07 smithi123 ceph-mon[37143]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-07-24T21:44:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:07 smithi086 ceph-mon[37128]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-07-24T21:44:08.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:07 smithi060 ceph-mon[33000]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-07-24T21:44:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"flags":0,"active_gid":14214,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1194501493},{"type":"v1","addr":"172.21.15.60:6801","nonce":1194501493}]},"active_addr":"172.21.15.60:6801/1194501493","active_change":"2024-07-24T21:36:35.954770+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24139,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24152,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":4138987889}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1577212541}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":708808603}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2330701917}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":392716182}]}]} 2024-07-24T21:44:08.627 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-24T21:44:08.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-24T21:44:08.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:44:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:08 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/717544218' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-24T21:44:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:08 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/717544218' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-24T21:44:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:08 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/717544218' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-24T21:44:09.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:09.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:09 smithi060 ceph-mon[33000]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:44:09.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:09 smithi123 ceph-mon[37143]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:44:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:09 smithi086 ceph-mon[37128]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:44:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 -- 172.21.15.60:0/3651578375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a4103fc0 msgr2=0x7f70a410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/3651578375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a4103fc0 0x7f70a410c4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7094009900 tx=0x7f709402a410 comp rx=0 tx=0).stop 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 -- 172.21.15.60:0/3651578375 shutdown_connections 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/3651578375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a410cb00 0x7f70a410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/3651578375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a4103fc0 0x7f70a410c4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/3651578375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a41036a0 0x7f70a4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 -- 172.21.15.60:0/3651578375 >> 172.21.15.60:0/3651578375 conn(0x7f70a40fd100 msgr2=0x7f70a40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 -- 172.21.15.60:0/3651578375 shutdown_connections 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.678+0000 7f70ac2e5640 1 -- 172.21.15.60:0/3651578375 wait complete. 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 Processor -- start 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 -- start start 2024-07-24T21:44:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 0x7f70a4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4103fc0 0x7f70a41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4112760 con 0x7f70a410cb00 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41125e0 con 0x7f70a4103fc0 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.679+0000 7f70ac2e5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41128e0 con 0x7f70a41036a0 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60368/0 (socket says 172.21.15.60:60368) 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 -- 172.21.15.60:0/634527514 learned_addr learned my addr 172.21.15.60:0/634527514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70a9880640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4103fc0 0x7f70a41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa081640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 0x7f70a4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 -- 172.21.15.60:0/634527514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 msgr2=0x7f70a4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 0x7f70a4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 -- 172.21.15.60:0/634527514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4103fc0 msgr2=0x7f70a41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4103fc0 0x7f70a41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 -- 172.21.15.60:0/634527514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a419ed40 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa081640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 0x7f70a4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70aa882640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f709c00cad0 tx=0x7f709c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c00ae20 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a419f030 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c005020 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c00f6b0 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.680+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a56f0 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.681+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a4103a80 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.683+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f709c0051d0 con 0x7f70a410cb00 2024-07-24T21:44:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.683+0000 7f709b7fe640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 0x7f70900b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.683+0000 7f70aa081640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 0x7f70900b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.684+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f709c0e3f60 con 0x7f70a410cb00 2024-07-24T21:44:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.684+0000 7f70aa081640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 0x7f70900b6170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f70a0005e50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.687+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f709c014030 con 0x7f70a410cb00 2024-07-24T21:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.899+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70a4107550 con 0x7f70a410cb00 2024-07-24T21:44:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.901+0000 7f709b7fe640 1 -- 172.21.15.60:0/634527514 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22978 (secure 0 0 0) 0x7f709c0ab370 con 0x7f70a410cb00 2024-07-24T21:44:10.902 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:44:10.902 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:43:37.396360+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 msgr2=0x7f70900b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 0x7f70900b6170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f70a0005e50 comp rx=0 tx=0).stop 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 msgr2=0x7f70a419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f709c00cad0 tx=0x7f709c00cf90 comp rx=0 tx=0).stop 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 shutdown_connections 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f70900b3cb0 0x7f70900b6170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a410cb00 0x7f70a419e570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4103fc0 0x7f70a41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 --2- 172.21.15.60:0/634527514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a41036a0 0x7f70a4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 >> 172.21.15.60:0/634527514 conn(0x7f70a40fd100 msgr2=0x7f70a40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 shutdown_connections 2024-07-24T21:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:10.903+0000 7f70ac2e5640 1 -- 172.21.15.60:0/634527514 wait complete. 2024-07-24T21:44:11.578 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-24T21:44:11.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:44:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/634527514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:44:11.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:44:11.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/634527514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:44:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:44:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/634527514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:44:13.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- 172.21.15.60:0/3393716213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 msgr2=0x7f09f810ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:13.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/3393716213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f810ef20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009900 tx=0x7f09f002a410 comp rx=0 tx=0).stop 2024-07-24T21:44:13.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- 172.21.15.60:0/3393716213 shutdown_connections 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/3393716213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f810ef20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/3393716213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 0x7f09f810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/3393716213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 0x7f09f8103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- 172.21.15.60:0/3393716213 >> 172.21.15.60:0/3393716213 conn(0x7f09f80fd0c0 msgr2=0x7f09f80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- 172.21.15.60:0/3393716213 shutdown_connections 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- 172.21.15.60:0/3393716213 wait complete. 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 Processor -- start 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.446+0000 7f09fd4c7640 1 -- start start 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 0x7f09f8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 0x7f09f81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112790 con 0x7f09f810cac0 2024-07-24T21:44:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112610 con 0x7f09f8103f80 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09fd4c7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8112910 con 0x7f09f8103660 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60382/0 (socket says 172.21.15.60:60382) 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 -- 172.21.15.60:0/2637739529 learned_addr learned my addr 172.21.15.60:0/2637739529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f6ffd640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 0x7f09f8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f67fc640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 0x7f09f81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 -- 172.21.15.60:0/2637739529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 msgr2=0x7f09f8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 0x7f09f8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 -- 172.21.15.60:0/2637739529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 msgr2=0x7f09f81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 0x7f09f81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 -- 172.21.15.60:0/2637739529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f819ed10 con 0x7f09f810cac0 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09f77fe640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009a20 tx=0x7f09f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.447+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f003f030 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.448+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f0003aa0 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.448+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f819efa0 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.448+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0035070 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.448+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f81a56c0 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.449+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f8103a40 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.450+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f09f0042030 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.451+0000 7f09effff640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 0x7f09e40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.451+0000 7f09f6ffd640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 0x7f09e40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.451+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f09f0104850 con 0x7f09f810cac0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.451+0000 7f09f6ffd640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 0x7f09e40b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005c40 tx=0x7f09e800a590 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.454+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f09f00cb610 con 0x7f09f810cac0 2024-07-24T21:44:13.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.648+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f81a5980 con 0x7f09f810cac0 2024-07-24T21:44:13.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.650+0000 7f09effff640 1 -- 172.21.15.60:0/2637739529 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22978 (secure 0 0 0) 0x7f09f00cfb60 con 0x7f09f810cac0 2024-07-24T21:44:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:44:13.651 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:43:37.396360+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:44:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 msgr2=0x7f09e40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 0x7f09e40b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005c40 tx=0x7f09e800a590 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 msgr2=0x7f09f819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009a20 tx=0x7f09f0004270 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 shutdown_connections 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f09e40b3cb0 0x7f09e40b6170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09f810cac0 0x7f09f819e540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f8103f80 0x7f09f81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 --2- 172.21.15.60:0/2637739529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09f8103660 0x7f09f8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 >> 172.21.15.60:0/2637739529 conn(0x7f09f80fd0c0 msgr2=0x7f09f810ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.652+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 shutdown_connections 2024-07-24T21:44:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:13.653+0000 7f09fd4c7640 1 -- 172.21.15.60:0/2637739529 wait complete. 2024-07-24T21:44:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:13 smithi123 ceph-mon[37143]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2637739529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:14.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:13 smithi060 ceph-mon[33000]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:14.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2637739529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:14.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:13 smithi086 ceph-mon[37128]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2637739529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:44:14.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-24T21:44:14.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-24T21:44:14.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-24T21:44:14.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-24T21:44:14.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-24T21:44:14.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-24T21:44:14.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-24T21:44:14.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-24T21:44:14.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-24T21:44:14.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-24T21:44:14.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-24T21:44:14.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-24T21:44:15.063 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.154 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.169 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.170 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.175 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.175 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.176 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:15.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:15 smithi060 ceph-mon[33000]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:15.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:15 smithi123 ceph-mon[37143]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:16.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:15 smithi086 ceph-mon[37128]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:17.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:17 smithi060 ceph-mon[33000]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:17 smithi123 ceph-mon[37143]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:17 smithi086 ceph-mon[37128]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:19.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:19 smithi123 ceph-mon[37143]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:20.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:19 smithi086 ceph-mon[37128]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:20.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:19 smithi060 ceph-mon[33000]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:21.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:21 smithi060 ceph-mon[33000]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:21 smithi123 ceph-mon[37143]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:21 smithi086 ceph-mon[37128]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:23.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:23 smithi060 ceph-mon[33000]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:23 smithi123 ceph-mon[37143]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:23 smithi086 ceph-mon[37128]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:44:25.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- 172.21.15.60:0/3403479959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3301081a0 msgr2=0x7fd3301085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 --2- 172.21.15.60:0/3403479959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3301081a0 0x7fd3301085e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd320009900 tx=0x7fd32002a410 comp rx=0 tx=0).stop 2024-07-24T21:44:25.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- 172.21.15.60:0/3403479959 shutdown_connections 2024-07-24T21:44:25.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 --2- 172.21.15.60:0/3403479959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd330108b20 0x7fd33010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 --2- 172.21.15.60:0/3403479959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3301081a0 0x7fd3301085e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 --2- 172.21.15.60:0/3403479959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd330102790 0x7fd330102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- 172.21.15.60:0/3403479959 >> 172.21.15.60:0/3403479959 conn(0x7fd3300fe200 msgr2=0x7fd330100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:25.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- 172.21.15.60:0/3403479959 shutdown_connections 2024-07-24T21:44:25.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- 172.21.15.60:0/3403479959 wait complete. 2024-07-24T21:44:25.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 Processor -- start 2024-07-24T21:44:25.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.204+0000 7fd33760d640 1 -- start start 2024-07-24T21:44:25.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd330102790 0x7fd330198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3301081a0 0x7fd3301991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 0x7fd33019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd3301127a0 con 0x7fd330108b20 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd330112620 con 0x7fd330102790 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd33760d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd330112920 con 0x7fd3301081a0 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 0x7fd33019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 0x7fd33019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45488/0 (socket says 172.21.15.60:45488) 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd334ba8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3301081a0 0x7fd3301991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 -- 172.21.15.60:0/2042055349 learned_addr learned my addr 172.21.15.60:0/2042055349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd3353a9640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd330102790 0x7fd330198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 -- 172.21.15.60:0/2042055349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3301081a0 msgr2=0x7fd3301991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3301081a0 0x7fd3301991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 -- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd330102790 msgr2=0x7fd330198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd330102790 0x7fd330198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33019ed40 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.205+0000 7fd335baa640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 0x7fd33019e570 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd32800b970 tx=0x7fd32800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.206+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328019070 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.206+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd328014440 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.206+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33019f030 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.206+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3280075d0 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.206+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3301a56f0 con 0x7fd330108b20 2024-07-24T21:44:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.207+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3300632d0 con 0x7fd330108b20 2024-07-24T21:44:25.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.208+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd32801c070 con 0x7fd330108b20 2024-07-24T21:44:25.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3267fc640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd31c0b3cb0 0x7fd31c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:25.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3353a9640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd31c0b3cb0 0x7fd31c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:25.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7fd3280e3b60 con 0x7fd330108b20 2024-07-24T21:44:25.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3267fc640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] conn(0x7fd31c0bdb60 0x7fd31c0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd328020e50 con 0x7fd31c0bdb60 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd3280e3fb0 con 0x7fd330108b20 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd3353a9640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd31c0b3cb0 0x7fd31c0b6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd32c005e10 tx=0x7fd32c005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.209+0000 7fd334ba8640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] conn(0x7fd31c0bdb60 0x7fd31c0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.210+0000 7fd334ba8640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] conn(0x7fd31c0bdb60 0x7fd31c0c0000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:25.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.211+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== osd.4 v2:172.21.15.86:6808/1082160365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd328020e50 con 0x7fd31c0bdb60 2024-07-24T21:44:25.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.225+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 --> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd330063750 con 0x7fd31c0bdb60 2024-07-24T21:44:25.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd3267fc640 1 -- 172.21.15.60:0/2042055349 <== osd.4 v2:172.21.15.86:6808/1082160365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd330063750 con 0x7fd31c0bdb60 2024-07-24T21:44:25.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] conn(0x7fd31c0bdb60 msgr2=0x7fd31c0c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] conn(0x7fd31c0bdb60 0x7fd31c0c0000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd31c0b3cb0 msgr2=0x7fd31c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd31c0b3cb0 0x7fd31c0b6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd32c005e10 tx=0x7fd32c005da0 comp rx=0 tx=0).stop 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 msgr2=0x7fd33019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 --2- 172.21.15.60:0/2042055349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd330108b20 0x7fd33019e570 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd32800b970 tx=0x7fd32800be30 comp rx=0 tx=0).stop 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd335baa640 1 -- 172.21.15.60:0/2042055349 reap_dead start 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 shutdown_connections 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 >> 172.21.15.60:0/2042055349 conn(0x7fd3300fe200 msgr2=0x7fd3301005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 shutdown_connections 2024-07-24T21:44:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:25.226+0000 7fd33760d640 1 -- 172.21.15.60:0/2042055349 wait complete. 2024-07-24T21:44:25.966 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:25 smithi060 ceph-mon[33000]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:25 smithi123 ceph-mon[37143]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:26.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:25 smithi086 ceph-mon[37128]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:44:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 -- 172.21.15.60:0/3338170474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c01036a0 msgr2=0x7f01c0103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/3338170474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c01036a0 0x7f01c0103a80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009900 tx=0x7f01bc02a410 comp rx=0 tx=0).stop 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 -- 172.21.15.60:0/3338170474 shutdown_connections 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/3338170474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c010cb00 0x7f01c010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/3338170474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0103fc0 0x7f01c010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/3338170474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c01036a0 0x7f01c0103a80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.455+0000 7f01c7f3d640 1 -- 172.21.15.60:0/3338170474 >> 172.21.15.60:0/3338170474 conn(0x7f01c00fd100 msgr2=0x7f01c00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- 172.21.15.60:0/3338170474 shutdown_connections 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- 172.21.15.60:0/3338170474 wait complete. 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 Processor -- start 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- start start 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c01036a0 0x7f01c0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 0x7f01c0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c010cb00 0x7f01c019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0112760 con 0x7f01c0103fc0 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01125e0 con 0x7f01c01036a0 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.456+0000 7f01c7f3d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01128e0 con 0x7f01c010cb00 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 0x7f01c0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c5cd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c01036a0 0x7f01c0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 0x7f01c0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60598/0 (socket says 172.21.15.60:60598) 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 -- 172.21.15.60:0/4279212439 learned_addr learned my addr 172.21.15.60:0/4279212439 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c64da640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c010cb00 0x7f01c019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 -- 172.21.15.60:0/4279212439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c010cb00 msgr2=0x7f01c019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c010cb00 0x7f01c019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 -- 172.21.15.60:0/4279212439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c01036a0 msgr2=0x7f01c0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c01036a0 0x7f01c0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c019edc0 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c5cd9640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c01036a0 0x7f01c0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c54d8640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 0x7f01c0199250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d740 tx=0x7f01b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017020 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b000de50 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.457+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c019f0b0 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.458+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000f620 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.458+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a5940 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.459+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01c00632d0 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.460+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f01b001a020 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01b6ffd640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01ac0b3d30 0x7f01ac0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01c5cd9640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01ac0b3d30 0x7f01ac0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f01b00e3fb0 con 0x7f01c0103fc0 2024-07-24T21:44:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01b6ffd640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] conn(0x7f01ac0bdbe0 0x7f01ac0c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01b001ee70 con 0x7f01ac0bdbe0 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f01b00e4360 con 0x7f01c0103fc0 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01c64da640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] conn(0x7f01ac0bdbe0 0x7f01ac0c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.461+0000 7f01c5cd9640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01ac0b3d30 0x7f01ac0b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f01bc031440 tx=0x7f01bc005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.462+0000 7f01c64da640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] conn(0x7f01ac0bdbe0 0x7f01ac0c0080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.462+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== osd.3 v2:172.21.15.60:6810/1240167260 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01b001ee70 con 0x7f01ac0bdbe0 2024-07-24T21:44:26.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.477+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 --> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01c00636d0 con 0x7f01ac0bdbe0 2024-07-24T21:44:26.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01b6ffd640 1 -- 172.21.15.60:0/4279212439 <== osd.3 v2:172.21.15.60:6810/1240167260 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f01c00636d0 con 0x7f01ac0bdbe0 2024-07-24T21:44:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] conn(0x7f01ac0bdbe0 msgr2=0x7f01ac0c0080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] conn(0x7f01ac0bdbe0 0x7f01ac0c0080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01ac0b3d30 msgr2=0x7f01ac0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01ac0b3d30 0x7f01ac0b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f01bc031440 tx=0x7f01bc005b50 comp rx=0 tx=0).stop 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 msgr2=0x7f01c0199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 --2- 172.21.15.60:0/4279212439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c0103fc0 0x7f01c0199250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d740 tx=0x7f01b000dc00 comp rx=0 tx=0).stop 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c64da640 1 -- 172.21.15.60:0/4279212439 reap_dead start 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 shutdown_connections 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 >> 172.21.15.60:0/4279212439 conn(0x7f01c00fd100 msgr2=0x7f01c00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 shutdown_connections 2024-07-24T21:44:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:26.478+0000 7f01c7f3d640 1 -- 172.21.15.60:0/4279212439 wait complete. 2024-07-24T21:44:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:27 smithi123 ceph-mon[37143]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:28.010 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:27 smithi060 ceph-mon[33000]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:28.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:27 smithi086 ceph-mon[37128]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:28.130 INFO:teuthology.orchestra.run.smithi060.stdout:141733920830 2024-07-24T21:44:28.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 -- 172.21.15.60:0/2217595159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c10cac0 msgr2=0x7f544c10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 --2- 172.21.15.60:0/2217595159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c10cac0 0x7f544c10ef10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5444009ca0 tx=0x7f544402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 -- 172.21.15.60:0/2217595159 shutdown_connections 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 --2- 172.21.15.60:0/2217595159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c10cac0 0x7f544c10ef10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 --2- 172.21.15.60:0/2217595159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 0x7f544c10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 --2- 172.21.15.60:0/2217595159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c103510 0x7f544c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 -- 172.21.15.60:0/2217595159 >> 172.21.15.60:0/2217595159 conn(0x7f544c0fd0c0 msgr2=0x7f544c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 -- 172.21.15.60:0/2217595159 shutdown_connections 2024-07-24T21:44:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.282+0000 7f5453c69640 1 -- 172.21.15.60:0/2217595159 wait complete. 2024-07-24T21:44:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 Processor -- start 2024-07-24T21:44:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 -- start start 2024-07-24T21:44:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 0x7f544c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 0x7f544c1018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c10cac0 0x7f544c101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f544c112700 con 0x7f544c103510 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f544c112580 con 0x7f544c103f10 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5453c69640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f544c112880 con 0x7f544c10cac0 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 0x7f544c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 0x7f544c101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60614/0 (socket says 172.21.15.60:60614) 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 -- 172.21.15.60:0/2749339803 learned_addr learned my addr 172.21.15.60:0/2749339803 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451204640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 0x7f544c1018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5452206640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c10cac0 0x7f544c101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 -- 172.21.15.60:0/2749339803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c10cac0 msgr2=0x7f544c101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c10cac0 0x7f544c101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 -- 172.21.15.60:0/2749339803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 msgr2=0x7f544c1018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 0x7f544c1018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451a05640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c1a7010 con 0x7f544c103510 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.283+0000 7f5451204640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f544c103f10 0x7f544c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5451a05640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 0x7f544c101390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f544800d950 tx=0x7f544800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448017030 con 0x7f544c103510 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448004020 con 0x7f544c103510 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c1a7300 con 0x7f544c103510 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480047a0 con 0x7f544c103510 2024-07-24T21:44:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.284+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c1a7840 con 0x7f544c103510 2024-07-24T21:44:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.285+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f544c063290 con 0x7f544c103510 2024-07-24T21:44:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.286+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f5448009a30 con 0x7f544c103510 2024-07-24T21:44:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.287+0000 7f5442ffd640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54380b3cb0 0x7f54380b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.287+0000 7f5451204640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54380b3cb0 0x7f54380b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.287+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f54480e3f40 con 0x7f544c103510 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.287+0000 7f5442ffd640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] conn(0x7f54380bdb60 0x7f54380c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.287+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54380c06d0 con 0x7f54380bdb60 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.288+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f54480e4390 con 0x7f544c103510 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.288+0000 7f5452206640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] conn(0x7f54380bdb60 0x7f54380c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.288+0000 7f5451204640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54380b3cb0 0x7f54380b6170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f543c005fd0 tx=0x7f543c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.288+0000 7f5452206640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] conn(0x7f54380bdb60 0x7f54380c0000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.289+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== osd.9 v2:172.21.15.60:6826/451392111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54380c06d0 con 0x7f54380bdb60 2024-07-24T21:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.320+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 --> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f544c063710 con 0x7f54380bdb60 2024-07-24T21:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5442ffd640 1 -- 172.21.15.60:0/2749339803 <== osd.9 v2:172.21.15.60:6826/451392111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f544c063710 con 0x7f54380bdb60 2024-07-24T21:44:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] conn(0x7f54380bdb60 msgr2=0x7f54380c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] conn(0x7f54380bdb60 0x7f54380c0000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54380b3cb0 msgr2=0x7f54380b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54380b3cb0 0x7f54380b6170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f543c005fd0 tx=0x7f543c005850 comp rx=0 tx=0).stop 2024-07-24T21:44:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 msgr2=0x7f544c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 --2- 172.21.15.60:0/2749339803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f544c103510 0x7f544c101390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f544800d950 tx=0x7f544800de10 comp rx=0 tx=0).stop 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5452206640 1 -- 172.21.15.60:0/2749339803 reap_dead start 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 shutdown_connections 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 >> 172.21.15.60:0/2749339803 conn(0x7f544c0fd0c0 msgr2=0x7f544c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 shutdown_connections 2024-07-24T21:44:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:28.321+0000 7f5453c69640 1 -- 172.21.15.60:0/2749339803 wait complete. 2024-07-24T21:44:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- 172.21.15.60:0/1630726203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10108180 msgr2=0x7f6c101085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 --2- 172.21.15.60:0/1630726203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10108180 0x7f6c101085c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf802a410 comp rx=0 tx=0).stop 2024-07-24T21:44:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- 172.21.15.60:0/1630726203 shutdown_connections 2024-07-24T21:44:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 --2- 172.21.15.60:0/1630726203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c10108b00 0x7f6c1010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 --2- 172.21.15.60:0/1630726203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10108180 0x7f6c101085c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 --2- 172.21.15.60:0/1630726203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10102770 0x7f6c10102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- 172.21.15.60:0/1630726203 >> 172.21.15.60:0/1630726203 conn(0x7f6c100fe1c0 msgr2=0x7f6c101005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- 172.21.15.60:0/1630726203 shutdown_connections 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- 172.21.15.60:0/1630726203 wait complete. 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 Processor -- start 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.880+0000 7f6c16997640 1 -- start start 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 0x7f6c10198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c10108180 0x7f6c101991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10108b00 0x7f6c1019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6c10112730 con 0x7f6c10102770 2024-07-24T21:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c101125b0 con 0x7f6c10108b00 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c16997640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c101128b0 con 0x7f6c10108180 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 0x7f6c10198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 0x7f6c10198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60632/0 (socket says 172.21.15.60:60632) 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 -- 172.21.15.60:0/2042993187 learned_addr learned my addr 172.21.15.60:0/2042993187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c07fff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c10108180 0x7f6c101991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c14f34640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10108b00 0x7f6c1019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 -- 172.21.15.60:0/2042993187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c10108180 msgr2=0x7f6c101991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c10108180 0x7f6c101991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 -- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10108b00 msgr2=0x7f6c1019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10108b00 0x7f6c1019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1019ed50 con 0x7f6c10102770 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c14f34640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c10108b00 0x7f6c1019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0ffff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 0x7f6c10198ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000e9f0 tx=0x7f6c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.881+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000cd80 con 0x7f6c10102770 2024-07-24T21:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.882+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c000108e0 con 0x7f6c10102770 2024-07-24T21:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.882+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1019f040 con 0x7f6c10102770 2024-07-24T21:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.882+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00018980 con 0x7f6c10102770 2024-07-24T21:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.882+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c101a5700 con 0x7f6c10102770 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.882+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c10063290 con 0x7f6c10102770 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.884+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f6c00010430 con 0x7f6c10102770 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c0dffb640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6bfc0b3cb0 0x7f6bfc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c07fff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6bfc0b3cb0 0x7f6bfc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f6c000e4a20 con 0x7f6c10102770 2024-07-24T21:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c0dffb640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] conn(0x7f6bfc0bdb60 0x7f6bfc0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bfc0c06d0 con 0x7f6bfc0bdb60 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f6c000e4dd0 con 0x7f6c10102770 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c07fff640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6bfc0b3cb0 0x7f6bfc0b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8031440 tx=0x7f6bf8005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.885+0000 7f6c14f34640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] conn(0x7f6bfc0bdb60 0x7f6bfc0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.886+0000 7f6c14f34640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] conn(0x7f6bfc0bdb60 0x7f6bfc0c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.887+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== osd.1 v2:172.21.15.86:6800/2205288191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bfc0c06d0 con 0x7f6bfc0bdb60 2024-07-24T21:44:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.901+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 --> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c10063740 con 0x7f6bfc0bdb60 2024-07-24T21:44:29.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c0dffb640 1 -- 172.21.15.60:0/2042993187 <== osd.1 v2:172.21.15.86:6800/2205288191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c10063740 con 0x7f6bfc0bdb60 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] conn(0x7f6bfc0bdb60 msgr2=0x7f6bfc0c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] conn(0x7f6bfc0bdb60 0x7f6bfc0c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6bfc0b3cb0 msgr2=0x7f6bfc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6bfc0b3cb0 0x7f6bfc0b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8031440 tx=0x7f6bf8005b50 comp rx=0 tx=0).stop 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 msgr2=0x7f6c10198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 --2- 172.21.15.60:0/2042993187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c10102770 0x7f6c10198ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000e9f0 tx=0x7f6c0000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c14f34640 1 -- 172.21.15.60:0/2042993187 reap_dead start 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 shutdown_connections 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 >> 172.21.15.60:0/2042993187 conn(0x7f6c100fe1c0 msgr2=0x7f6c101005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.902+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 shutdown_connections 2024-07-24T21:44:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:29.903+0000 7f6c16997640 1 -- 172.21.15.60:0/2042993187 wait complete. 2024-07-24T21:44:29.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:29 smithi060 ceph-mon[33000]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:29 smithi123 ceph-mon[37143]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:29 smithi086 ceph-mon[37128]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:30.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- 172.21.15.60:0/2329633125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d8103660 msgr2=0x7f03d8103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 --2- 172.21.15.60:0/2329633125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d8103660 0x7f03d8103a40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009900 tx=0x7f03d402a410 comp rx=0 tx=0).stop 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- 172.21.15.60:0/2329633125 shutdown_connections 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 --2- 172.21.15.60:0/2329633125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03d810cac0 0x7f03d810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 --2- 172.21.15.60:0/2329633125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103f80 0x7f03d810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 --2- 172.21.15.60:0/2329633125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d8103660 0x7f03d8103a40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- 172.21.15.60:0/2329633125 >> 172.21.15.60:0/2329633125 conn(0x7f03d80fd0c0 msgr2=0x7f03d80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- 172.21.15.60:0/2329633125 shutdown_connections 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- 172.21.15.60:0/2329633125 wait complete. 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 Processor -- start 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.439+0000 7f03df545640 1 -- start start 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103660 0x7f03d8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03d8103f80 0x7f03d81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 0x7f03d819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8112770 con 0x7f03d810cac0 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81125f0 con 0x7f03d8103660 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03df545640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81128f0 con 0x7f03d8103f80 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 0x7f03d819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03dcae0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03d8103f80 0x7f03d81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 0x7f03d819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60652/0 (socket says 172.21.15.60:60652) 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 -- 172.21.15.60:0/3222592294 learned_addr learned my addr 172.21.15.60:0/3222592294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03dd2e1640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103660 0x7f03d8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 -- 172.21.15.60:0/3222592294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03d8103f80 msgr2=0x7f03d81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03d8103f80 0x7f03d81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 -- 172.21.15.60:0/3222592294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103660 msgr2=0x7f03d8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103660 0x7f03d8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d819ed10 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03dd2e1640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d8103660 0x7f03d8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.440+0000 7f03ddae2640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 0x7f03d819e540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cad0 tx=0x7f03d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.441+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ae20 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.441+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d0005020 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.441+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d819f000 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.441+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000f6b0 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.441+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81a5890 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.442+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03d80ff730 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.443+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f03d00051d0 con 0x7f03d810cac0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03ce7fc640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03c40b3d30 0x7f03c40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03dd2e1640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03c40b3d30 0x7f03c40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f03d00e4270 con 0x7f03d810cac0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03ce7fc640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] conn(0x7f03c40bdbe0 0x7f03c40c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03d0016ec0 con 0x7f03c40bdbe0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f03d00e46c0 con 0x7f03d810cac0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.444+0000 7f03dcae0640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] conn(0x7f03c40bdbe0 0x7f03c40c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.445+0000 7f03dd2e1640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03c40b3d30 0x7f03c40b61f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000c00 tx=0x7f03d4031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.445+0000 7f03dcae0640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] conn(0x7f03c40bdbe0 0x7f03c40c0080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.445+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== osd.6 v2:172.21.15.60:6818/2857305108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03d0016ec0 con 0x7f03c40bdbe0 2024-07-24T21:44:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.460+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 --> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03d806ab70 con 0x7f03c40bdbe0 2024-07-24T21:44:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.460+0000 7f03ce7fc640 1 -- 172.21.15.60:0/3222592294 <== osd.6 v2:172.21.15.60:6818/2857305108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03d806ab70 con 0x7f03c40bdbe0 2024-07-24T21:44:30.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] conn(0x7f03c40bdbe0 msgr2=0x7f03c40c0080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] conn(0x7f03c40bdbe0 0x7f03c40c0080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:30.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03c40b3d30 msgr2=0x7f03c40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03c40b3d30 0x7f03c40b61f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000c00 tx=0x7f03d4031400 comp rx=0 tx=0).stop 2024-07-24T21:44:30.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 msgr2=0x7f03d819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 --2- 172.21.15.60:0/3222592294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03d810cac0 0x7f03d819e540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cad0 tx=0x7f03d000cf90 comp rx=0 tx=0).stop 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03ddae2640 1 -- 172.21.15.60:0/3222592294 reap_dead start 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 shutdown_connections 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 >> 172.21.15.60:0/3222592294 conn(0x7f03d80fd0c0 msgr2=0x7f03d80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 shutdown_connections 2024-07-24T21:44:30.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:30.461+0000 7f03df545640 1 -- 172.21.15.60:0/3222592294 wait complete. 2024-07-24T21:44:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.796+0000 7fea119e7640 1 -- 172.21.15.60:0/1743828580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c107a00 msgr2=0x7fea0c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.796+0000 7fea119e7640 1 --2- 172.21.15.60:0/1743828580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c107a00 0x7fea0c10e200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea04009900 tx=0x7fea0402a410 comp rx=0 tx=0).stop 2024-07-24T21:44:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 -- 172.21.15.60:0/1743828580 shutdown_connections 2024-07-24T21:44:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 --2- 172.21.15.60:0/1743828580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c107a00 0x7fea0c10e200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 --2- 172.21.15.60:0/1743828580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107080 0x7fea0c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 --2- 172.21.15.60:0/1743828580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c101670 0x7fea0c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 -- 172.21.15.60:0/1743828580 >> 172.21.15.60:0/1743828580 conn(0x7fea0c0fd0e0 msgr2=0x7fea0c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 -- 172.21.15.60:0/1743828580 shutdown_connections 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 -- 172.21.15.60:0/1743828580 wait complete. 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 Processor -- start 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.797+0000 7fea119e7640 1 -- start start 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 0x7fea0c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c107080 0x7fea0c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107a00 0x7fea0c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c111630 con 0x7fea0c101670 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1114b0 con 0x7fea0c107080 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1117b0 con 0x7fea0c107a00 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 0x7fea0c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 0x7fea0c1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60676/0 (socket says 172.21.15.60:60676) 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 -- 172.21.15.60:0/2193550292 learned_addr learned my addr 172.21.15.60:0/2193550292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0b7fe640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107a00 0x7fea0c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0a7fc640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c107080 0x7fea0c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 -- 172.21.15.60:0/2193550292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107a00 msgr2=0x7fea0c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107a00 0x7fea0c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 -- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c107080 msgr2=0x7fea0c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c107080 0x7fea0c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c1a4d80 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0b7fe640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea0c107a00 0x7fea0c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0a7fc640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea0c107080 0x7fea0c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea0affd640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 0x7fea0c1999a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00ebc0 tx=0x7fe9fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc010630 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c1a5070 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc010c80 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.798+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc018b60 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.799+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1a55b0 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.799+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea0c104990 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.801+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe9fc018d00 con 0x7fea0c101670 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea109e5640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe9f40b3cb0 0x7fe9f40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea0a7fc640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe9f40b3cb0 0x7fe9f40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7fe9fc0e4e80 con 0x7fea0c101670 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea109e5640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] conn(0x7fe9f40bdb60 0x7fe9f40c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9f40c06d0 con 0x7fe9f40bdb60 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0e5230 con 0x7fea0c101670 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea0a7fc640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe9f40b3cb0 0x7fe9f40b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea0c06ba50 tx=0x7fea0000a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.802+0000 7fea0b7fe640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] conn(0x7fe9f40bdb60 0x7fe9f40c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.803+0000 7fea0b7fe640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] conn(0x7fe9f40bdb60 0x7fe9f40c0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.804+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== osd.10 v2:172.21.15.86:6824/2411147999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9f40c06d0 con 0x7fe9f40bdb60 2024-07-24T21:44:31.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.819+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 --> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea0c102920 con 0x7fe9f40bdb60 2024-07-24T21:44:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.819+0000 7fea109e5640 1 -- 172.21.15.60:0/2193550292 <== osd.10 v2:172.21.15.86:6824/2411147999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea0c102920 con 0x7fe9f40bdb60 2024-07-24T21:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] conn(0x7fe9f40bdb60 msgr2=0x7fe9f40c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] conn(0x7fe9f40bdb60 0x7fe9f40c0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe9f40b3cb0 msgr2=0x7fe9f40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe9f40b3cb0 0x7fe9f40b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea0c06ba50 tx=0x7fea0000a5c0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 msgr2=0x7fea0c1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 --2- 172.21.15.60:0/2193550292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c101670 0x7fea0c1999a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00ebc0 tx=0x7fe9fc00c6a0 comp rx=0 tx=0).stop 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea0b7fe640 1 -- 172.21.15.60:0/2193550292 reap_dead start 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 shutdown_connections 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 >> 172.21.15.60:0/2193550292 conn(0x7fea0c0fd0e0 msgr2=0x7fea0c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 shutdown_connections 2024-07-24T21:44:31.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:31.820+0000 7fea119e7640 1 -- 172.21.15.60:0/2193550292 wait complete. 2024-07-24T21:44:31.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:31 smithi123 ceph-mon[37143]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:32.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:31 smithi086 ceph-mon[37128]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:32.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:31 smithi060 ceph-mon[33000]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:44:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 -- 172.21.15.60:0/2834534308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 msgr2=0x7f0fc410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 --2- 172.21.15.60:0/2834534308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc410c4b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009900 tx=0x7f0fac02a410 comp rx=0 tx=0).stop 2024-07-24T21:44:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 -- 172.21.15.60:0/2834534308 shutdown_connections 2024-07-24T21:44:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 --2- 172.21.15.60:0/2834534308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc410cb00 0x7f0fc410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 --2- 172.21.15.60:0/2834534308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc410c4b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 --2- 172.21.15.60:0/2834534308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc41036a0 0x7f0fc4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 -- 172.21.15.60:0/2834534308 >> 172.21.15.60:0/2834534308 conn(0x7f0fc40fd100 msgr2=0x7f0fc40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 -- 172.21.15.60:0/2834534308 shutdown_connections 2024-07-24T21:44:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.336+0000 7f0fc9170640 1 -- 172.21.15.60:0/2834534308 wait complete. 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 Processor -- start 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 -- start start 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc41036a0 0x7f0fc4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc4101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc410cb00 0x7f0fc4101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41127b0 con 0x7f0fc4103fc0 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4112630 con 0x7f0fc41036a0 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc9170640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4112930 con 0x7f0fc410cb00 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc4101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc4101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60704/0 (socket says 172.21.15.60:60704) 2024-07-24T21:44:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc259c640 1 -- 172.21.15.60:0/1356492916 learned_addr learned my addr 172.21.15.60:0/1356492916 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc2d9d640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc41036a0 0x7f0fc4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.337+0000 7f0fc359e640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc410cb00 0x7f0fc4101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 -- 172.21.15.60:0/1356492916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc410cb00 msgr2=0x7f0fc4101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc410cb00 0x7f0fc4101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 -- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc41036a0 msgr2=0x7f0fc4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc41036a0 0x7f0fc4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc41a71a0 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc259c640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc4101890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006010 tx=0x7f0fac004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac03f030 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc41a7430 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac004490 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac035070 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.338+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc41a7960 con 0x7f0fc4103fc0 2024-07-24T21:44:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.339+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0fc40632d0 con 0x7f0fc4103fc0 2024-07-24T21:44:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.341+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f0fac042030 con 0x7f0fc4103fc0 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.341+0000 7f0fbbfff640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0fb00b3cb0 0x7f0fb00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fc2d9d640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0fb00b3cb0 0x7f0fb00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f0fac1042c0 con 0x7f0fc4103fc0 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fbbfff640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] conn(0x7f0fb00bdb60 0x7f0fb00c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fac032970 con 0x7f0fb00bdb60 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f0fac104710 con 0x7f0fc4103fc0 2024-07-24T21:44:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fc2d9d640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0fb00b3cb0 0x7f0fb00b6170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009990 tx=0x7f0fbc009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.342+0000 7f0fc359e640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] conn(0x7f0fb00bdb60 0x7f0fb00c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.343+0000 7f0fc359e640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] conn(0x7f0fb00bdb60 0x7f0fb00c0000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.343+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== osd.7 v2:172.21.15.86:6816/3700904835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fac032970 con 0x7f0fb00bdb60 2024-07-24T21:44:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.358+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 --> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0fc40636d0 con 0x7f0fb00bdb60 2024-07-24T21:44:32.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fbbfff640 1 -- 172.21.15.60:0/1356492916 <== osd.7 v2:172.21.15.86:6816/3700904835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0fc40636d0 con 0x7f0fb00bdb60 2024-07-24T21:44:32.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] conn(0x7f0fb00bdb60 msgr2=0x7f0fb00c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] conn(0x7f0fb00bdb60 0x7f0fb00c0000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0fb00b3cb0 msgr2=0x7f0fb00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0fb00b3cb0 0x7f0fb00b6170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009990 tx=0x7f0fbc009110 comp rx=0 tx=0).stop 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 msgr2=0x7f0fc4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 --2- 172.21.15.60:0/1356492916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0fc4103fc0 0x7f0fc4101890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0fac006010 tx=0x7f0fac004060 comp rx=0 tx=0).stop 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc359e640 1 -- 172.21.15.60:0/1356492916 reap_dead start 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.359+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 shutdown_connections 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.360+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 >> 172.21.15.60:0/1356492916 conn(0x7f0fc40fd100 msgr2=0x7f0fc40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.360+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 shutdown_connections 2024-07-24T21:44:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.360+0000 7f0fc9170640 1 -- 172.21.15.60:0/1356492916 wait complete. 2024-07-24T21:44:32.412 INFO:teuthology.orchestra.run.smithi060.stdout:115964117059 2024-07-24T21:44:32.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-24T21:44:33.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.971+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/2224967428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 msgr2=0x7f3b18102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.971+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/2224967428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18102b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009900 tx=0x7f3b0802a410 comp rx=0 tx=0).stop 2024-07-24T21:44:33.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/2224967428 shutdown_connections 2024-07-24T21:44:33.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/2224967428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b18108b20 0x7f3b1810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/2224967428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b181081a0 0x7f3b181085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/2224967428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18102b70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/2224967428 >> 172.21.15.60:0/2224967428 conn(0x7f3b180fe200 msgr2=0x7f3b18100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/2224967428 shutdown_connections 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/2224967428 wait complete. 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 Processor -- start 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.972+0000 7f3b1c8c1640 1 -- start start 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b181081a0 0x7f3b180753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18108b20 0x7f3b18075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b181127a0 con 0x7f3b18102790 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18112620 con 0x7f3b18108b20 2024-07-24T21:44:33.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1c8c1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18112920 con 0x7f3b181081a0 2024-07-24T21:44:33.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60710/0 (socket says 172.21.15.60:60710) 2024-07-24T21:44:33.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 -- 172.21.15.60:0/1661642241 learned_addr learned my addr 172.21.15.60:0/1661642241 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:33.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b15d9b640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b181081a0 0x7f3b180753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b16d9d640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18108b20 0x7f3b18075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 -- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b181081a0 msgr2=0x7f3b180753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b181081a0 0x7f3b180753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 -- 172.21.15.60:0/1661642241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18108b20 msgr2=0x7f3b18075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18108b20 0x7f3b18075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181a7030 con 0x7f3b18102790 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b16d9d640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18108b20 0x7f3b18075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b1659c640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18074e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006010 tx=0x7f3b080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.973+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0803f030 con 0x7f3b18102790 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.974+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b181a72c0 con 0x7f3b18102790 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.974+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b08003800 con 0x7f3b18102790 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.974+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08035070 con 0x7f3b18102790 2024-07-24T21:44:33.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.974+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181a7820 con 0x7f3b18102790 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.974+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b180632d0 con 0x7f3b18102790 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.976+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f3b08003cb0 con 0x7f3b18102790 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b077fe640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b000b3cb0 0x7f3b000b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b15d9b640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b000b3cb0 0x7f3b000b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f3b08104f60 con 0x7f3b18102790 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b077fe640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] conn(0x7f3b000bdb60 0x7f3b000c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b08046e50 con 0x7f3b000bdb60 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3b08105310 con 0x7f3b18102790 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b15d9b640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b000b3cb0 0x7f3b000b6170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b18076980 tx=0x7f3b0c0075b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.977+0000 7f3b16d9d640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] conn(0x7f3b000bdb60 0x7f3b000c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.978+0000 7f3b16d9d640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] conn(0x7f3b000bdb60 0x7f3b000c0000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.978+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== osd.8 v2:172.21.15.123:6816/2616635066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b08046e50 con 0x7f3b000bdb60 2024-07-24T21:44:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.993+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 --> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b18063750 con 0x7f3b000bdb60 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.993+0000 7f3b077fe640 1 -- 172.21.15.60:0/1661642241 <== osd.8 v2:172.21.15.123:6816/2616635066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b18063750 con 0x7f3b000bdb60 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.993+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] conn(0x7f3b000bdb60 msgr2=0x7f3b000c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.993+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] conn(0x7f3b000bdb60 0x7f3b000c0000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b000b3cb0 msgr2=0x7f3b000b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b000b3cb0 0x7f3b000b6170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b18076980 tx=0x7f3b0c0075b0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 msgr2=0x7f3b18074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 --2- 172.21.15.60:0/1661642241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b18102790 0x7f3b18074e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006010 tx=0x7f3b080042f0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b16d9d640 1 -- 172.21.15.60:0/1661642241 reap_dead start 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 shutdown_connections 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 >> 172.21.15.60:0/1661642241 conn(0x7f3b180fe200 msgr2=0x7f3b181005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 shutdown_connections 2024-07-24T21:44:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:32.994+0000 7f3b1c8c1640 1 -- 172.21.15.60:0/1661642241 wait complete. 2024-07-24T21:44:33.631 INFO:teuthology.orchestra.run.smithi060.stdout:219043332146 2024-07-24T21:44:33.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 -- 172.21.15.60:0/3273170207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f678010cac0 msgr2=0x7f678010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3273170207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f678010cac0 0x7f678010ef20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6778009ca0 tx=0x7f677802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 -- 172.21.15.60:0/3273170207 shutdown_connections 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3273170207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f678010cac0 0x7f678010ef20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3273170207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6780103f80 0x7f678010c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3273170207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6780103660 0x7f6780103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 -- 172.21.15.60:0/3273170207 >> 172.21.15.60:0/3273170207 conn(0x7f67800fd0a0 msgr2=0x7f67800ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 -- 172.21.15.60:0/3273170207 shutdown_connections 2024-07-24T21:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.830+0000 7f6787d3e640 1 -- 172.21.15.60:0/3273170207 wait complete. 2024-07-24T21:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 Processor -- start 2024-07-24T21:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 -- start start 2024-07-24T21:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 0x7f6780198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6780103f80 0x7f67801990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678010cac0 0x7f678019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6780112710 con 0x7f6780103660 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6780112590 con 0x7f6780103f80 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6787d3e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6780112890 con 0x7f678010cac0 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6785ada640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 0x7f6780198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6785ada640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 0x7f6780198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60732/0 (socket says 172.21.15.60:60732) 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6785ada640 1 -- 172.21.15.60:0/3804695738 learned_addr learned my addr 172.21.15.60:0/3804695738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f67862db640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678010cac0 0x7f678019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f67852d9640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6780103f80 0x7f67801990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6785ada640 1 -- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678010cac0 msgr2=0x7f678019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.831+0000 7f6785ada640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f678010cac0 0x7f678019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6785ada640 1 -- 172.21.15.60:0/3804695738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6780103f80 msgr2=0x7f67801990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6785ada640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6780103f80 0x7f67801990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6785ada640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678019ec10 con 0x7f6780103660 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6785ada640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 0x7f6780198b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f677c00d950 tx=0x7f677c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c017070 con 0x7f6780103660 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678019ef00 con 0x7f6780103660 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c009c30 con 0x7f6780103660 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00f620 con 0x7f6780103660 2024-07-24T21:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.832+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67801a55c0 con 0x7f6780103660 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.833+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67800ff000 con 0x7f6780103660 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.834+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f677c010460 con 0x7f6780103660 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f6776ffd640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f676c0b3cb0 0x7f676c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8463+0+0 (secure 0 0 0) 0x7f677c0e3f40 con 0x7f6780103660 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f67852d9640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f676c0b3cb0 0x7f676c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f6776ffd640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] conn(0x7f676c0bdb60 0x7f676c0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f677c00d130 con 0x7f676c0bdb60 2024-07-24T21:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.835+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f677c0e4390 con 0x7f6780103660 2024-07-24T21:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.836+0000 7f67852d9640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f676c0b3cb0 0x7f676c0b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6770005e30 tx=0x7f6770005dc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.836+0000 7f67862db640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] conn(0x7f676c0bdb60 0x7f676c0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.836+0000 7f67862db640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] conn(0x7f676c0bdb60 0x7f676c0c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.837+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== osd.5 v2:172.21.15.123:6808/44992122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f677c00d130 con 0x7f676c0bdb60 2024-07-24T21:44:33.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.851+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 --> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f678006b550 con 0x7f676c0bdb60 2024-07-24T21:44:33.852 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:33 smithi060 ceph-mon[33000]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:33.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6776ffd640 1 -- 172.21.15.60:0/3804695738 <== osd.5 v2:172.21.15.123:6808/44992122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f678006b550 con 0x7f676c0bdb60 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] conn(0x7f676c0bdb60 msgr2=0x7f676c0c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] conn(0x7f676c0bdb60 0x7f676c0c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f676c0b3cb0 msgr2=0x7f676c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f676c0b3cb0 0x7f676c0b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6770005e30 tx=0x7f6770005dc0 comp rx=0 tx=0).stop 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 msgr2=0x7f6780198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 --2- 172.21.15.60:0/3804695738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6780103660 0x7f6780198b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f677c00d950 tx=0x7f677c00de10 comp rx=0 tx=0).stop 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f67862db640 1 -- 172.21.15.60:0/3804695738 reap_dead start 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 shutdown_connections 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.852+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 >> 172.21.15.60:0/3804695738 conn(0x7f67800fd0a0 msgr2=0x7f678010eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.853+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 shutdown_connections 2024-07-24T21:44:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:33.853+0000 7f6787d3e640 1 -- 172.21.15.60:0/3804695738 wait complete. 2024-07-24T21:44:33.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:33 smithi123 ceph-mon[37143]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:34.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:33 smithi086 ceph-mon[37128]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:34.244 INFO:teuthology.orchestra.run.smithi060.stdout:60129542221 2024-07-24T21:44:34.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-24T21:44:34.653 INFO:teuthology.orchestra.run.smithi060.stdout:193273528373 2024-07-24T21:44:34.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-24T21:44:35.550 INFO:teuthology.orchestra.run.smithi060.stdout:296352743460 2024-07-24T21:44:35.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-24T21:44:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:35 smithi060 ceph-mon[33000]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-24T21:44:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:35 smithi123 ceph-mon[37143]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-24T21:44:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:35 smithi086 ceph-mon[37128]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-24T21:44:36.534 INFO:teuthology.orchestra.run.smithi060.stdout:274877906984 2024-07-24T21:44:36.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-24T21:44:36.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-24T21:44:36.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-07-24T21:44:36.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-24T21:44:36.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-07-24T21:44:37.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-24T21:44:37.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-07-24T21:44:37.155 INFO:teuthology.orchestra.run.smithi060.stdout:249108103213 2024-07-24T21:44:37.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 -- 172.21.15.60:0/4197133488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190107060 msgr2=0x7fc1901074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 --2- 172.21.15.60:0/4197133488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190107060 0x7fc1901074a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002a410 comp rx=0 tx=0).stop 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 -- 172.21.15.60:0/4197133488 shutdown_connections 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 --2- 172.21.15.60:0/4197133488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1901079e0 0x7fc19010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 --2- 172.21.15.60:0/4197133488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190107060 0x7fc1901074a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 --2- 172.21.15.60:0/4197133488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc190101650 0x7fc190101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 -- 172.21.15.60:0/4197133488 >> 172.21.15.60:0/4197133488 conn(0x7fc1900fd0c0 msgr2=0x7fc1900ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 -- 172.21.15.60:0/4197133488 shutdown_connections 2024-07-24T21:44:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.373+0000 7fc197b7d640 1 -- 172.21.15.60:0/4197133488 wait complete. 2024-07-24T21:44:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 Processor -- start 2024-07-24T21:44:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 -- start start 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 0x7fc1900fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc190107060 0x7fc1900fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1901079e0 0x7fc1900fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111610 con 0x7fc190101650 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111490 con 0x7fc1901079e0 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc197b7d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111790 con 0x7fc190107060 2024-07-24T21:44:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc195919640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 0x7fc1900fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc195919640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 0x7fc1900fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36384/0 (socket says 172.21.15.60:36384) 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.374+0000 7fc195919640 1 -- 172.21.15.60:0/3164064476 learned_addr learned my addr 172.21.15.60:0/3164064476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195118640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc190107060 0x7fc1900fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc19611a640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1901079e0 0x7fc1900fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 -- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc190107060 msgr2=0x7fc1900fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc190107060 0x7fc1900fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 -- 172.21.15.60:0/3164064476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1901079e0 msgr2=0x7fc1900fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1901079e0 0x7fc1900fe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a70a0 con 0x7fc190101650 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc195919640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 0x7fc1900fd5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00d950 tx=0x7fc18c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c017030 con 0x7fc190101650 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a7390 con 0x7fc190101650 2024-07-24T21:44:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c004020 con 0x7fc190101650 2024-07-24T21:44:37.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0047a0 con 0x7fc190101650 2024-07-24T21:44:37.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.375+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a78f0 con 0x7fc190101650 2024-07-24T21:44:37.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.376+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc19006a780 con 0x7fc190101650 2024-07-24T21:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.379+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fc18c009a30 con 0x7fc190101650 2024-07-24T21:44:37.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc186ffd640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc17c0b3cb0 0x7fc17c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc195118640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc17c0b3cb0 0x7fc17c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8521+0+0 (secure 0 0 0) 0x7fc18c0e4100 con 0x7fc190101650 2024-07-24T21:44:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc186ffd640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] conn(0x7fc17c0bdc60 0x7fc17c0c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc18c01ee70 con 0x7fc17c0bdc60 2024-07-24T21:44:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc18c0e4550 con 0x7fc190101650 2024-07-24T21:44:37.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc195118640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc17c0b3cb0 0x7fc17c0b6170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc180031440 tx=0x7fc1800095f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:37.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.381+0000 7fc19611a640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] conn(0x7fc17c0bdc60 0x7fc17c0c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:37.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.382+0000 7fc19611a640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] conn(0x7fc17c0bdc60 0x7fc17c0c0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:37.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.383+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== osd.2 v2:172.21.15.123:6800/3586156659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc18c01ee70 con 0x7fc17c0bdc60 2024-07-24T21:44:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.399+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 --> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc190102900 con 0x7fc17c0bdc60 2024-07-24T21:44:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.399+0000 7fc186ffd640 1 -- 172.21.15.60:0/3164064476 <== osd.2 v2:172.21.15.123:6800/3586156659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc190102900 con 0x7fc17c0bdc60 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] conn(0x7fc17c0bdc60 msgr2=0x7fc17c0c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] conn(0x7fc17c0bdc60 0x7fc17c0c0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc17c0b3cb0 msgr2=0x7fc17c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc17c0b3cb0 0x7fc17c0b6170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc180031440 tx=0x7fc1800095f0 comp rx=0 tx=0).stop 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 msgr2=0x7fc1900fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 --2- 172.21.15.60:0/3164064476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190101650 0x7fc1900fd5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00d950 tx=0x7fc18c00de10 comp rx=0 tx=0).stop 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc19611a640 1 -- 172.21.15.60:0/3164064476 reap_dead start 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 shutdown_connections 2024-07-24T21:44:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 >> 172.21.15.60:0/3164064476 conn(0x7fc1900fd0c0 msgr2=0x7fc190104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 shutdown_connections 2024-07-24T21:44:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:37.400+0000 7fc197b7d640 1 -- 172.21.15.60:0/3164064476 wait complete. 2024-07-24T21:44:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:37 smithi123 ceph-mon[37143]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T21:44:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-24T21:44:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:37 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-07-24T21:44:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:37 smithi123 ceph-mon[37143]: osdmap e86: 12 total, 12 up, 12 in 2024-07-24T21:44:38.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:37 smithi060 ceph-mon[33000]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T21:44:38.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-24T21:44:38.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:37 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-07-24T21:44:38.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:37 smithi060 ceph-mon[33000]: osdmap e86: 12 total, 12 up, 12 in 2024-07-24T21:44:38.038 INFO:teuthology.orchestra.run.smithi060.stdout:167503724603 2024-07-24T21:44:38.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-24T21:44:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:37 smithi086 ceph-mon[37128]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T21:44:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-24T21:44:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:37 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-07-24T21:44:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:37 smithi086 ceph-mon[37128]: osdmap e86: 12 total, 12 up, 12 in 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 -- 172.21.15.60:0/3481713628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103f80 msgr2=0x7f20d010c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 --2- 172.21.15.60:0/3481713628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103f80 0x7f20d010c470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009900 tx=0x7f20c002a410 comp rx=0 tx=0).stop 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 -- 172.21.15.60:0/3481713628 shutdown_connections 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 --2- 172.21.15.60:0/3481713628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d010cac0 0x7f20d010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 --2- 172.21.15.60:0/3481713628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103f80 0x7f20d010c470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 --2- 172.21.15.60:0/3481713628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20d0103660 0x7f20d0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 -- 172.21.15.60:0/3481713628 >> 172.21.15.60:0/3481713628 conn(0x7f20d00fd0c0 msgr2=0x7f20d00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 -- 172.21.15.60:0/3481713628 shutdown_connections 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.187+0000 7f20d72a7640 1 -- 172.21.15.60:0/3481713628 wait complete. 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 Processor -- start 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 -- start start 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 0x7f20d0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20d0103f80 0x7f20d0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d010cac0 0x7f20d019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0112770 con 0x7f20d0103660 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01125f0 con 0x7f20d010cac0 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d72a7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01128f0 con 0x7f20d0103f80 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d5043640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 0x7f20d0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d5043640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 0x7f20d0198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36406/0 (socket says 172.21.15.60:36406) 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d5043640 1 -- 172.21.15.60:0/2977573722 learned_addr learned my addr 172.21.15.60:0/2977573722 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.188+0000 7f20d5844640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d010cac0 0x7f20d019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d4842640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20d0103f80 0x7f20d0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 -- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20d0103f80 msgr2=0x7f20d0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20d0103f80 0x7f20d0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 -- 172.21.15.60:0/2977573722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d010cac0 msgr2=0x7f20d019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d010cac0 0x7f20d019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d019edf0 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d5043640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 0x7f20d0198d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b4a0 tx=0x7f20cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc014070 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d019f0e0 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20cc010ac0 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc017c30 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.189+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d01a5970 con 0x7f20d0103660 2024-07-24T21:44:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.190+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20d0063290 con 0x7f20d0103660 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.191+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f20cc017070 con 0x7f20d0103660 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20c67fc640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20bc0b3cb0 0x7f20bc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20d4842640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20bc0b3cb0 0x7f20bc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20cc0e3700 con 0x7f20d0103660 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20c67fc640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] conn(0x7f20bc0bdc60 0x7f20bc0c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20cc022e50 con 0x7f20bc0bdc60 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.192+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f20cc0e3ab0 con 0x7f20d0103660 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.193+0000 7f20d4842640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20bc0b3cb0 0x7f20bc0b6170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c0000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.193+0000 7f20d5844640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] conn(0x7f20bc0bdc60 0x7f20bc0c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.193+0000 7f20d5844640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] conn(0x7f20bc0bdc60 0x7f20bc0c0100 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.194+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== osd.11 v2:172.21.15.123:6824/1138852346 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20cc022e50 con 0x7f20bc0bdc60 2024-07-24T21:44:38.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.209+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 --> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20d0063690 con 0x7f20bc0bdc60 2024-07-24T21:44:38.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.209+0000 7f20c67fc640 1 -- 172.21.15.60:0/2977573722 <== osd.11 v2:172.21.15.123:6824/1138852346 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20d0063690 con 0x7f20bc0bdc60 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.209+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] conn(0x7f20bc0bdc60 msgr2=0x7f20bc0c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] conn(0x7f20bc0bdc60 0x7f20bc0c0100 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20bc0b3cb0 msgr2=0x7f20bc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20bc0b3cb0 0x7f20bc0b6170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20c0006010 tx=0x7f20c0000bc0 comp rx=0 tx=0).stop 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 msgr2=0x7f20d0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 --2- 172.21.15.60:0/2977573722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d0103660 0x7f20d0198d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b4a0 tx=0x7f20cc00b960 comp rx=0 tx=0).stop 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d5844640 1 -- 172.21.15.60:0/2977573722 reap_dead start 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 shutdown_connections 2024-07-24T21:44:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 >> 172.21.15.60:0/2977573722 conn(0x7f20d00fd0c0 msgr2=0x7f20d00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:38.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 shutdown_connections 2024-07-24T21:44:38.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:38.210+0000 7f20d72a7640 1 -- 172.21.15.60:0/2977573722 wait complete. 2024-07-24T21:44:38.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:38 smithi123 ceph-mon[37143]: osdmap e87: 12 total, 12 up, 12 in 2024-07-24T21:44:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:38 smithi123 ceph-mon[37143]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:44:39.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:38 smithi086 ceph-mon[37128]: osdmap e87: 12 total, 12 up, 12 in 2024-07-24T21:44:39.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:38 smithi086 ceph-mon[37128]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:44:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:38 smithi060 ceph-mon[33000]: osdmap e87: 12 total, 12 up, 12 in 2024-07-24T21:44:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:38 smithi060 ceph-mon[33000]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:44:39.238 INFO:teuthology.orchestra.run.smithi060.stdout:81604378697 2024-07-24T21:44:39.238 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-24T21:44:39.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 -- 172.21.15.60:0/3868057807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 msgr2=0x7f8818103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 --2- 172.21.15.60:0/3868057807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818103a80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a410 comp rx=0 tx=0).stop 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 -- 172.21.15.60:0/3868057807 shutdown_connections 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 --2- 172.21.15.60:0/3868057807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881810cb00 0x7f881810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 --2- 172.21.15.60:0/3868057807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 0x7f881810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 --2- 172.21.15.60:0/3868057807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818103a80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 -- 172.21.15.60:0/3868057807 >> 172.21.15.60:0/3868057807 conn(0x7f88180fd100 msgr2=0x7f88180ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 -- 172.21.15.60:0/3868057807 shutdown_connections 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.485+0000 7f881d177640 1 -- 172.21.15.60:0/3868057807 wait complete. 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 Processor -- start 2024-07-24T21:44:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 -- start start 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 0x7f88181991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881810cb00 0x7f881819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8818112760 con 0x7f88181036a0 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88181125e0 con 0x7f881810cb00 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f881d177640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88181128e0 con 0x7f8818103fc0 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36428/0 (socket says 172.21.15.60:36428) 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.486+0000 7f8816d9d640 1 -- 172.21.15.60:0/3231592166 learned_addr learned my addr 172.21.15.60:0/3231592166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:39.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f881759e640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881810cb00 0x7f881819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f881659c640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 0x7f88181991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 -- 172.21.15.60:0/3231592166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 msgr2=0x7f88181991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 0x7f88181991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 -- 172.21.15.60:0/3231592166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881810cb00 msgr2=0x7f881819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881810cb00 0x7f881819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819ed60 con 0x7f88181036a0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f881659c640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8818103fc0 0x7f88181991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f8816d9d640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818198cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8808005940 tx=0x7f88080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803f030 con 0x7f88181036a0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881819eff0 con 0x7f88181036a0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8808003cb0 con 0x7f88181036a0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808035070 con 0x7f88181036a0 2024-07-24T21:44:39.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.487+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a5710 con 0x7f88181036a0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.488+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8818063290 con 0x7f88181036a0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.490+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8808042030 con 0x7f88181036a0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.490+0000 7f87fffff640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88000b3cb0 0x7f88000b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.490+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8808104a70 con 0x7f88181036a0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.490+0000 7f881659c640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88000b3cb0 0x7f88000b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f87fffff640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] conn(0x7f88000bdc60 0x7f88000c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8808046e50 con 0x7f88000bdc60 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f8808104ec0 con 0x7f88181036a0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f881759e640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] conn(0x7f88000bdc60 0x7f88000c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f881659c640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88000b3cb0 0x7f88000b6170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f880c0058c0 tx=0x7f880c005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.491+0000 7f881759e640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] conn(0x7f88000bdc60 0x7f88000c0100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:39.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.492+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== osd.0 v2:172.21.15.60:6802/2629637807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8808046e50 con 0x7f88000bdc60 2024-07-24T21:44:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.509+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 --> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8818063690 con 0x7f88000bdc60 2024-07-24T21:44:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.509+0000 7f87fffff640 1 -- 172.21.15.60:0/3231592166 <== osd.0 v2:172.21.15.60:6802/2629637807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8818063690 con 0x7f88000bdc60 2024-07-24T21:44:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] conn(0x7f88000bdc60 msgr2=0x7f88000c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] conn(0x7f88000bdc60 0x7f88000c0100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88000b3cb0 msgr2=0x7f88000b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88000b3cb0 0x7f88000b6170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f880c0058c0 tx=0x7f880c005850 comp rx=0 tx=0).stop 2024-07-24T21:44:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 msgr2=0x7f8818198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 --2- 172.21.15.60:0/3231592166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88181036a0 0x7f8818198cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8808005940 tx=0x7f88080042f0 comp rx=0 tx=0).stop 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881759e640 1 -- 172.21.15.60:0/3231592166 reap_dead start 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 shutdown_connections 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 >> 172.21.15.60:0/3231592166 conn(0x7f88180fd100 msgr2=0x7f88180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 shutdown_connections 2024-07-24T21:44:39.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:39.510+0000 7f881d177640 1 -- 172.21.15.60:0/3231592166 wait complete. 2024-07-24T21:44:39.529 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:39.530 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:39.530 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:39.531 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:39.951 INFO:teuthology.orchestra.run.smithi060.stdout:322122547232 2024-07-24T21:44:39.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-24T21:44:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:39 smithi086 ceph-mon[37128]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-24T21:44:40.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:39 smithi060 ceph-mon[33000]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-24T21:44:40.159 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:40.162 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:40.163 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:40.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:39 smithi123 ceph-mon[37143]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-24T21:44:40.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:40 smithi060 ceph-mon[33000]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:40 smithi086 ceph-mon[37128]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:41.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:40 smithi123 ceph-mon[37143]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:43.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:43 smithi060 ceph-mon[33000]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:43 smithi123 ceph-mon[37143]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:43 smithi086 ceph-mon[37128]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:44.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:44 smithi060 ceph-mon[33000]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-24T21:44:44.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:44 smithi060 ceph-mon[33000]: Cluster is now healthy 2024-07-24T21:44:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:44 smithi123 ceph-mon[37143]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-24T21:44:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:44 smithi123 ceph-mon[37143]: Cluster is now healthy 2024-07-24T21:44:44.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:44 smithi086 ceph-mon[37128]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-24T21:44:44.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:44 smithi086 ceph-mon[37128]: Cluster is now healthy 2024-07-24T21:44:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:45 smithi123 ceph-mon[37143]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:45 smithi086 ceph-mon[37128]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:45 smithi060 ceph-mon[33000]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/1785681455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e410ad10 msgr2=0x7fb4e4111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 --2- 172.21.15.60:0/1785681455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e410ad10 0x7fb4e4111510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009ca0 tx=0x7fb4dc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/1785681455 shutdown_connections 2024-07-24T21:44:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 --2- 172.21.15.60:0/1785681455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e410ad10 0x7fb4e4111510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 --2- 172.21.15.60:0/1785681455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 --2- 172.21.15.60:0/1785681455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e4104980 0x7fb4e4104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/1785681455 >> 172.21.15.60:0/1785681455 conn(0x7fb4e41003b0 msgr2=0x7fb4e41027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/1785681455 shutdown_connections 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.883+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/1785681455 wait complete. 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.884+0000 7fb4e8fd1640 1 Processor -- start 2024-07-24T21:44:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.884+0000 7fb4e8fd1640 1 -- start start 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e410ad10 0x7fb4e41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4114990 con 0x7fb4e4104980 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4114810 con 0x7fb4e410ad10 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e8fd1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4114b10 con 0x7fb4e410a390 2024-07-24T21:44:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56000/0 (socket says 172.21.15.60:56000) 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 -- 172.21.15.60:0/2130302297 learned_addr learned my addr 172.21.15.60:0/2130302297 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e1d9b640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 -- 172.21.15.60:0/2130302297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 msgr2=0x7fb4e419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 -- 172.21.15.60:0/2130302297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e410ad10 msgr2=0x7fb4e41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e410ad10 0x7fb4e41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e259c640 1 -- 172.21.15.60:0/2130302297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e41a0e90 con 0x7fb4e4104980 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.885+0000 7fb4e1d9b640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4e259c640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400d950 tx=0x7fb4d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4017070 con 0x7fb4e4104980 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/2130302297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e41a1180 con 0x7fb4e4104980 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4e8fd1640 1 -- 172.21.15.60:0/2130302297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e41a7a10 con 0x7fb4e4104980 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d4009c30 con 0x7fb4e4104980 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.886+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400f620 con 0x7fb4e4104980 2024-07-24T21:44:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.889+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e4104d60 con 0x7fb4e4104980 2024-07-24T21:44:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.889+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb4d400f7c0 con 0x7fb4e4104980 2024-07-24T21:44:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.890+0000 7fb4d37fe640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 0x7fb4cc0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.890+0000 7fb4e1d9b640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 0x7fb4cc0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.890+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb4d40e3b70 con 0x7fb4e4104980 2024-07-24T21:44:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.891+0000 7fb4e1d9b640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 0x7fb4cc0b61f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4106990 tx=0x7fb4d8005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:45.894+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb4d4009de0 con 0x7fb4e4104980 2024-07-24T21:44:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.092+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb4e419bf10 con 0x7fb4e4104980 2024-07-24T21:44:46.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.092+0000 7fb4d37fe640 1 -- 172.21.15.60:0/2130302297 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb4d40aaf40 con 0x7fb4e4104980 2024-07-24T21:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout:141733920834 2024-07-24T21:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 msgr2=0x7fb4cc0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 0x7fb4cc0b61f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4106990 tx=0x7fb4d8005d30 comp rx=0 tx=0).stop 2024-07-24T21:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 msgr2=0x7fb4e419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400d950 tx=0x7fb4d400de10 comp rx=0 tx=0).stop 2024-07-24T21:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 shutdown_connections 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4cc0b3d30 0x7fb4cc0b61f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e410ad10 0x7fb4e41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4e410a390 0x7fb4e419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 --2- 172.21.15.60:0/2130302297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e4104980 0x7fb4e419ade0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 >> 172.21.15.60:0/2130302297 conn(0x7fb4e41003b0 msgr2=0x7fb4e41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.094+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 shutdown_connections 2024-07-24T21:44:46.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:46.095+0000 7fb4d17fa640 1 -- 172.21.15.60:0/2130302297 wait complete. 2024-07-24T21:44:46.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:46 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2130302297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-24T21:44:46.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:46 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2130302297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-24T21:44:46.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:46 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2130302297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 -- 172.21.15.60:0/620739557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac103f30 msgr2=0x7f7bac10c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/620739557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac103f30 0x7f7bac10c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009900 tx=0x7f7b9c02a410 comp rx=0 tx=0).stop 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 -- 172.21.15.60:0/620739557 shutdown_connections 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/620739557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac10cae0 0x7f7bac10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/620739557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac103f30 0x7f7bac10c510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/620739557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 0x7f7bac103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 -- 172.21.15.60:0/620739557 >> 172.21.15.60:0/620739557 conn(0x7f7bac0fd0c0 msgr2=0x7f7bac0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 -- 172.21.15.60:0/620739557 shutdown_connections 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 -- 172.21.15.60:0/620739557 wait complete. 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.060+0000 7f7bb2daf640 1 Processor -- start 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 -- start start 2024-07-24T21:44:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 0x7f7bac198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac103f30 0x7f7bac199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112700 con 0x7f7bac10cae0 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112580 con 0x7f7bac103530 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb2daf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112880 con 0x7f7bac103f30 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb0b4b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 0x7f7bac198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56010/0 (socket says 172.21.15.60:56010) 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 -- 172.21.15.60:0/1405589581 learned_addr learned my addr 172.21.15.60:0/1405589581 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7babfff640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac103f30 0x7f7bac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 -- 172.21.15.60:0/1405589581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac103f30 msgr2=0x7f7bac199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac103f30 0x7f7bac199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 -- 172.21.15.60:0/1405589581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 msgr2=0x7f7bac198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 0x7f7bac198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.061+0000 7f7bb134c640 1 -- 172.21.15.60:0/1405589581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac19ed80 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7bb134c640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400cad0 tx=0x7f7ba400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400ae20 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba4005020 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac19f070 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400f6b0 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.062+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1a5730 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.063+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac06aad0 con 0x7f7bac10cae0 2024-07-24T21:44:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.064+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f7ba40051d0 con 0x7f7bac10cae0 2024-07-24T21:44:47.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.065+0000 7f7ba9ffb640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 0x7f7b980b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:47.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.065+0000 7f7bb0b4b640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 0x7f7b980b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:47.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.066+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7ba40e3df0 con 0x7f7bac10cae0 2024-07-24T21:44:47.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.066+0000 7f7bb0b4b640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 0x7f7b980b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005d00 tx=0x7f7b9400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:47.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.069+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7ba4014030 con 0x7f7bac10cae0 2024-07-24T21:44:47.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.266+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7bac02e000 con 0x7f7bac10cae0 2024-07-24T21:44:47.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.267+0000 7f7ba9ffb640 1 -- 172.21.15.60:0/1405589581 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ba40aab70 con 0x7f7bac10cae0 2024-07-24T21:44:47.268 INFO:teuthology.orchestra.run.smithi060.stdout:60129542224 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 msgr2=0x7f7b980b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 0x7f7b980b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005d00 tx=0x7f7b9400b410 comp rx=0 tx=0).stop 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 msgr2=0x7f7bac19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400cad0 tx=0x7f7ba400cf90 comp rx=0 tx=0).stop 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 shutdown_connections 2024-07-24T21:44:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7b980b3cb0 0x7f7b980b6170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bac10cae0 0x7f7bac19e5b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bac103f30 0x7f7bac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 --2- 172.21.15.60:0/1405589581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bac103530 0x7f7bac198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 >> 172.21.15.60:0/1405589581 conn(0x7f7bac0fd0c0 msgr2=0x7f7bac0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 shutdown_connections 2024-07-24T21:44:47.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:47.269+0000 7f7bb2daf640 1 -- 172.21.15.60:0/1405589581 wait complete. 2024-07-24T21:44:47.409 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:47 smithi060 ceph-mon[33000]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:47 smithi123 ceph-mon[37143]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:47.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920834 for osd.4 2024-07-24T21:44:47.530 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:47.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:47 smithi086 ceph-mon[37128]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:48.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:48 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1405589581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-24T21:44:48.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:48 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1405589581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-24T21:44:48.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542221 got 60129542224 for osd.1 2024-07-24T21:44:48.530 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:48.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:48 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1405589581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-24T21:44:49.211 INFO:teuthology.orchestra.run.smithi060.stdout:38654705747 2024-07-24T21:44:49.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-24T21:44:49.476 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:49 smithi060 ceph-mon[33000]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:49 smithi123 ceph-mon[37143]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:49.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:49 smithi086 ceph-mon[37128]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/1667029167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c107a20 msgr2=0x7fb49c10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/1667029167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c107a20 0x7fb49c10e220 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb494009ca0 tx=0x7fb49402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/1667029167 shutdown_connections 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/1667029167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c107a20 0x7fb49c10e220 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/1667029167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c1070a0 0x7fb49c1074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/1667029167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/1667029167 >> 172.21.15.60:0/1667029167 conn(0x7fb49c0fd100 msgr2=0x7fb49c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/1667029167 shutdown_connections 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.712+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/1667029167 wait complete. 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 Processor -- start 2024-07-24T21:44:49.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 -- start start 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 0x7fb49c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1116b0 con 0x7fb49c1070a0 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c111530 con 0x7fb49c107a20 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a3bd8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c111830 con 0x7fb49c101690 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a1173640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a1173640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56024/0 (socket says 172.21.15.60:56024) 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.713+0000 7fb4a1173640 1 -- 172.21.15.60:0/656960181 learned_addr learned my addr 172.21.15.60:0/656960181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a2175640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 0x7fb49c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1974640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 -- 172.21.15.60:0/656960181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 msgr2=0x7fb49c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:49.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 -- 172.21.15.60:0/656960181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 msgr2=0x7fb49c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 0x7fb49c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 -- 172.21.15.60:0/656960181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c1a6fd0 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a2175640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 0x7fb49c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1974640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a1173640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c920 tx=0x7fb48c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c007c70 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48c007e10 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c1a72c0 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c018820 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.714+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c1a7880 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.715+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c101a70 con 0x7fb49c1070a0 2024-07-24T21:44:49.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.716+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb48c00f450 con 0x7fb49c1070a0 2024-07-24T21:44:49.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.717+0000 7fb492ffd640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 0x7fb4880b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:49.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.717+0000 7fb4a1974640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 0x7fb4880b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:49.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.717+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb48c0e4110 con 0x7fb49c1070a0 2024-07-24T21:44:49.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.718+0000 7fb4a1974640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 0x7fb4880b6170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:49.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.721+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb48c014030 con 0x7fb49c1070a0 2024-07-24T21:44:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.917+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb49c02e000 con 0x7fb49c1070a0 2024-07-24T21:44:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.917+0000 7fb492ffd640 1 -- 172.21.15.60:0/656960181 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb48c0ab4e0 con 0x7fb49c1070a0 2024-07-24T21:44:49.919 INFO:teuthology.orchestra.run.smithi060.stdout:219043332149 2024-07-24T21:44:49.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 msgr2=0x7fb4880b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 0x7fb4880b6170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb498006fd0 tx=0x7fb498008040 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 msgr2=0x7fb49c197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c920 tx=0x7fb48c00cde0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 shutdown_connections 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb4880b3cb0 0x7fb4880b6170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c107a20 0x7fb49c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb49c1070a0 0x7fb49c197d30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 --2- 172.21.15.60:0/656960181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb49c101690 0x7fb49c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.919+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 >> 172.21.15.60:0/656960181 conn(0x7fb49c0fd100 msgr2=0x7fb49c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.920+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 shutdown_connections 2024-07-24T21:44:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:49.920+0000 7fb4a3bd8640 1 -- 172.21.15.60:0/656960181 wait complete. 2024-07-24T21:44:50.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:50 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/656960181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-24T21:44:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:50 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/656960181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-24T21:44:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:50 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/656960181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-24T21:44:50.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 -- 172.21.15.60:0/2764746606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c010cb00 msgr2=0x7f61c010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 --2- 172.21.15.60:0/2764746606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c010cb00 0x7f61c010ef60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009ca0 tx=0x7f61b802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 -- 172.21.15.60:0/2764746606 shutdown_connections 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 --2- 172.21.15.60:0/2764746606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c010cb00 0x7f61c010ef60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 --2- 172.21.15.60:0/2764746606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c0103fc0 0x7f61c010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 --2- 172.21.15.60:0/2764746606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 0x7f61c0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 -- 172.21.15.60:0/2764746606 >> 172.21.15.60:0/2764746606 conn(0x7f61c00fd100 msgr2=0x7f61c00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 -- 172.21.15.60:0/2764746606 shutdown_connections 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.616+0000 7f61c74a0640 1 -- 172.21.15.60:0/2764746606 wait complete. 2024-07-24T21:44:50.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 Processor -- start 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 -- start start 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 0x7f61c0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c010cb00 0x7f61c0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01127a0 con 0x7f61c0103fc0 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0112620 con 0x7f61c01036a0 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c74a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0112920 con 0x7f61c010cb00 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c4a3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:50.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c4a3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56040/0 (socket says 172.21.15.60:56040) 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c4a3b640 1 -- 172.21.15.60:0/1460664159 learned_addr learned my addr 172.21.15.60:0/1460664159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c5a3d640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c010cb00 0x7f61c0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.617+0000 7f61c523c640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 0x7f61c0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 -- 172.21.15.60:0/1460664159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c010cb00 msgr2=0x7f61c0101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c010cb00 0x7f61c0101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 -- 172.21.15.60:0/1460664159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 msgr2=0x7f61c0101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 0x7f61c0101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 -- 172.21.15.60:0/1460664159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01a7090 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c4a3b640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d750 tx=0x7f61b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0004020 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003c00 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01a7380 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.618+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a7910 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.619+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c0103a80 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.621+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f61b0009a30 con 0x7f61c0103fc0 2024-07-24T21:44:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.622+0000 7f61b67fc640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 0x7f61ac0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.622+0000 7f61c523c640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 0x7f61ac0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.622+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f61b00e3b50 con 0x7f61c0103fc0 2024-07-24T21:44:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.623+0000 7f61c523c640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 0x7f61ac0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00bbe0 tx=0x7f61bc00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.625+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f61b00aa950 con 0x7f61c0103fc0 2024-07-24T21:44:50.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.825+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f61c01a7c90 con 0x7f61c0103fc0 2024-07-24T21:44:50.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.826+0000 7f61b67fc640 1 -- 172.21.15.60:0/1460664159 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61b00aeea0 con 0x7f61c0103fc0 2024-07-24T21:44:50.827 INFO:teuthology.orchestra.run.smithi060.stdout:193273528377 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 msgr2=0x7f61ac0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 0x7f61ac0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00bbe0 tx=0x7f61bc00b390 comp rx=0 tx=0).stop 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 msgr2=0x7f61c0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d750 tx=0x7f61b000dc10 comp rx=0 tx=0).stop 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 shutdown_connections 2024-07-24T21:44:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61ac0b3d30 0x7f61ac0b61f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61c010cb00 0x7f61c0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c0103fc0 0x7f61c0101890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 --2- 172.21.15.60:0/1460664159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c01036a0 0x7f61c0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 >> 172.21.15.60:0/1460664159 conn(0x7f61c00fd100 msgr2=0x7f61c010ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 shutdown_connections 2024-07-24T21:44:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:50.828+0000 7f61c74a0640 1 -- 172.21.15.60:0/1460664159 wait complete. 2024-07-24T21:44:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.212+0000 7f51efcf7640 1 -- 172.21.15.60:0/2492240266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 msgr2=0x7f51e810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.212+0000 7f51efcf7640 1 --2- 172.21.15.60:0/2492240266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e810e1e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009900 tx=0x7f51e002a410 comp rx=0 tx=0).stop 2024-07-24T21:44:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 -- 172.21.15.60:0/2492240266 shutdown_connections 2024-07-24T21:44:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 --2- 172.21.15.60:0/2492240266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e810e1e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 --2- 172.21.15.60:0/2492240266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8107060 0x7f51e81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 --2- 172.21.15.60:0/2492240266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8101650 0x7f51e8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 -- 172.21.15.60:0/2492240266 >> 172.21.15.60:0/2492240266 conn(0x7f51e80fd0c0 msgr2=0x7f51e80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 -- 172.21.15.60:0/2492240266 shutdown_connections 2024-07-24T21:44:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 -- 172.21.15.60:0/2492240266 wait complete. 2024-07-24T21:44:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 Processor -- start 2024-07-24T21:44:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 -- start start 2024-07-24T21:44:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.213+0000 7f51efcf7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8101650 0x7f51e8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8107060 0x7f51e8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8111670 con 0x7f51e81079e0 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81114f0 con 0x7f51e8101650 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81117f0 con 0x7f51e8107060 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56062/0 (socket says 172.21.15.60:56062) 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 -- 172.21.15.60:0/1336405052 learned_addr learned my addr 172.21.15.60:0/1336405052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51eda93640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8101650 0x7f51e8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ed292640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8107060 0x7f51e8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 -- 172.21.15.60:0/1336405052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8107060 msgr2=0x7f51e8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8107060 0x7f51e8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 -- 172.21.15.60:0/1336405052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8101650 msgr2=0x7f51e8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8101650 0x7f51e8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 -- 172.21.15.60:0/1336405052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e819ec90 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51ee294640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009a20 tx=0x7f51e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e003f030 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e0003aa0 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e819ef20 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0035070 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.214+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e81a5640 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.215+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e8101a30 con 0x7f51e81079e0 2024-07-24T21:44:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.217+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f51e0042030 con 0x7f51e81079e0 2024-07-24T21:44:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.217+0000 7f51e6ffd640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 0x7f51d00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.218+0000 7f51eda93640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 0x7f51d00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.218+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f51e01049f0 con 0x7f51e81079e0 2024-07-24T21:44:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.218+0000 7f51eda93640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 0x7f51d00b6170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009f20 tx=0x7f51d80094f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.220+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f51e00cbdc0 con 0x7f51e81079e0 2024-07-24T21:44:51.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.421+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f51e81a5900 con 0x7f51e81079e0 2024-07-24T21:44:51.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:51 smithi060 ceph-mon[33000]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:51.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:51 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1460664159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-24T21:44:51.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.422+0000 7f51e6ffd640 1 -- 172.21.15.60:0/1336405052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f51e00cb6c0 con 0x7f51e81079e0 2024-07-24T21:44:51.423 INFO:teuthology.orchestra.run.smithi060.stdout:249108103217 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 msgr2=0x7f51d00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 0x7f51d00b6170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009f20 tx=0x7f51d80094f0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 msgr2=0x7f51e819e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009a20 tx=0x7f51e0004270 comp rx=0 tx=0).stop 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 shutdown_connections 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51d00b3cb0 0x7f51d00b6170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51e81079e0 0x7f51e819e4c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e8107060 0x7f51e8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 --2- 172.21.15.60:0/1336405052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51e8101650 0x7f51e8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 >> 172.21.15.60:0/1336405052 conn(0x7f51e80fd0c0 msgr2=0x7f51e806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 shutdown_connections 2024-07-24T21:44:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:51.424+0000 7f51efcf7640 1 -- 172.21.15.60:0/1336405052 wait complete. 2024-07-24T21:44:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:51 smithi123 ceph-mon[37143]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:51 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1460664159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-24T21:44:51.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:51 smithi086 ceph-mon[37128]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:51 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1460664159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-24T21:44:52.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:52 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1336405052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-24T21:44:52.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:52 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1336405052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-24T21:44:52.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:52 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1336405052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-24T21:44:52.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103213 got 249108103217 for osd.8 2024-07-24T21:44:52.597 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:52.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- 172.21.15.60:0/661387331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 msgr2=0x7fd90c10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:52.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- 172.21.15.60:0/661387331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c10ef10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd904009ca0 tx=0x7fd90402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- 172.21.15.60:0/661387331 shutdown_connections 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- 172.21.15.60:0/661387331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c10ef10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- 172.21.15.60:0/661387331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 0x7fd90c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- 172.21.15.60:0/661387331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- 172.21.15.60:0/661387331 >> 172.21.15.60:0/661387331 conn(0x7fd90c0fd0c0 msgr2=0x7fd90c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- 172.21.15.60:0/661387331 shutdown_connections 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- 172.21.15.60:0/661387331 wait complete. 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 Processor -- start 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 -- start start 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.728+0000 7fd91314c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 0x7fd90c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:52.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd91314c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd91314c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c112740 con 0x7fd90c10caa0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd91314c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1125c0 con 0x7fd90c1034f0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd91314c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1128c0 con 0x7fd90c103ef0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56086/0 (socket says 172.21.15.60:56086) 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 -- 172.21.15.60:0/1771153129 learned_addr learned my addr 172.21.15.60:0/1771153129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd910ee8640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd90bfff640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 0x7fd90c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 -- 172.21.15.60:0/1771153129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 msgr2=0x7fd90c199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 0x7fd90c199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 -- 172.21.15.60:0/1771153129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 msgr2=0x7fd90c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 -- 172.21.15.60:0/1771153129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c19ebe0 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd910ee8640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd9116e9640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd904031a20 tx=0x7fd904031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003d30 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9040044f0 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c19ee70 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.729+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90402fd90 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.730+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1a5590 con 0x7fd90c10caa0 2024-07-24T21:44:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.731+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c1038d0 con 0x7fd90c10caa0 2024-07-24T21:44:52.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.732+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd904004040 con 0x7fd90c10caa0 2024-07-24T21:44:52.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.733+0000 7fd909ffb640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 0x7fd8f80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:52.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.733+0000 7fd910ee8640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 0x7fd8f80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:52.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.733+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd9041054a0 con 0x7fd90c10caa0 2024-07-24T21:44:52.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.734+0000 7fd910ee8640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 0x7fd8f80b61f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40058c0 tx=0x7fd8f4005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:52.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.737+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd9040cca10 con 0x7fd90c10caa0 2024-07-24T21:44:52.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.934+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd90c1a5900 con 0x7fd90c10caa0 2024-07-24T21:44:52.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.934+0000 7fd909ffb640 1 -- 172.21.15.60:0/1771153129 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd9040cc310 con 0x7fd90c10caa0 2024-07-24T21:44:52.936 INFO:teuthology.orchestra.run.smithi060.stdout:115964117064 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 msgr2=0x7fd8f80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 0x7fd8f80b61f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40058c0 tx=0x7fd8f4005ec0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 msgr2=0x7fd90c19e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd904031a20 tx=0x7fd904031a50 comp rx=0 tx=0).stop 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 shutdown_connections 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8f80b3d30 0x7fd8f80b61f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd90c10caa0 0x7fd90c19e410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c103ef0 0x7fd90c199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 --2- 172.21.15.60:0/1771153129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd90c1034f0 0x7fd90c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 >> 172.21.15.60:0/1771153129 conn(0x7fd90c0fd0c0 msgr2=0x7fd90c10ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 shutdown_connections 2024-07-24T21:44:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:52.937+0000 7fd91314c640 1 -- 172.21.15.60:0/1771153129 wait complete. 2024-07-24T21:44:53.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:53 smithi060 ceph-mon[33000]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:53.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:53 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1771153129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-24T21:44:53.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:53 smithi123 ceph-mon[37143]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:53.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:53 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1771153129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-24T21:44:53.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:53 smithi086 ceph-mon[37128]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:53 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1771153129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-24T21:44:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:44:54.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117059 got 115964117064 for osd.3 2024-07-24T21:44:54.181 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.940+0000 7f58e2b5d640 1 -- 172.21.15.60:0/1867614032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc10eda0 msgr2=0x7f58dc111190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.940+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/1867614032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc10eda0 0x7f58dc111190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ca0 tx=0x7f58d402a950 comp rx=0 tx=0).stop 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 -- 172.21.15.60:0/1867614032 shutdown_connections 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/1867614032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc10eda0 0x7f58dc111190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/1867614032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc101b10 0x7f58dc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/1867614032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc1011f0 0x7f58dc1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 -- 172.21.15.60:0/1867614032 >> 172.21.15.60:0/1867614032 conn(0x7f58dc0fd0c0 msgr2=0x7f58dc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 -- 172.21.15.60:0/1867614032 shutdown_connections 2024-07-24T21:44:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 -- 172.21.15.60:0/1867614032 wait complete. 2024-07-24T21:44:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 Processor -- start 2024-07-24T21:44:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 -- start start 2024-07-24T21:44:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc1011f0 0x7f58dc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.941+0000 7f58e2b5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e2b5d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc10eda0 0x7f58dc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e2b5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc114920 con 0x7f58dc101b10 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e2b5d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1147a0 con 0x7f58dc1011f0 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e2b5d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc114aa0 con 0x7f58dc10eda0 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56102/0 (socket says 172.21.15.60:56102) 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 -- 172.21.15.60:0/206347067 learned_addr learned my addr 172.21.15.60:0/206347067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e10fa640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc10eda0 0x7f58dc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e08f9640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc1011f0 0x7f58dc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 -- 172.21.15.60:0/206347067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc10eda0 msgr2=0x7f58dc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc10eda0 0x7f58dc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 -- 172.21.15.60:0/206347067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc1011f0 msgr2=0x7f58dc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc1011f0 0x7f58dc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 -- 172.21.15.60:0/206347067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc1a0ec0 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58dbfff640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d950 tx=0x7f58c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8017030 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c8004020 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc1a11b0 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.942+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80047a0 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.943+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1a7870 con 0x7f58dc101b10 2024-07-24T21:44:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.944+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc1015d0 con 0x7f58dc101b10 2024-07-24T21:44:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.945+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f58c8009a30 con 0x7f58dc101b10 2024-07-24T21:44:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.946+0000 7f58d9ffb640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 0x7f58cc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:44:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.946+0000 7f58e08f9640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 0x7f58cc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:44:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.946+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f58c80e3ab0 con 0x7f58dc101b10 2024-07-24T21:44:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.946+0000 7f58e08f9640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 0x7f58cc0b6170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005fd0 tx=0x7f58d0005e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:44:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:54.950+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f58c80aae20 con 0x7f58dc101b10 2024-07-24T21:44:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.148+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f58dc1a7c40 con 0x7f58dc101b10 2024-07-24T21:44:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.149+0000 7f58d9ffb640 1 -- 172.21.15.60:0/206347067 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58c80aa720 con 0x7f58dc101b10 2024-07-24T21:44:55.150 INFO:teuthology.orchestra.run.smithi060.stdout:274877906989 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 msgr2=0x7f58cc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 0x7f58cc0b6170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58d0005fd0 tx=0x7f58d0005e20 comp rx=0 tx=0).stop 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 msgr2=0x7f58dc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d950 tx=0x7f58c800de10 comp rx=0 tx=0).stop 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 shutdown_connections 2024-07-24T21:44:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58cc0b3cb0 0x7f58cc0b6170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58dc10eda0 0x7f58dc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc101b10 0x7f58dc19b350 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 --2- 172.21.15.60:0/206347067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58dc1011f0 0x7f58dc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 >> 172.21.15.60:0/206347067 conn(0x7f58dc0fd0c0 msgr2=0x7f58dc110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 shutdown_connections 2024-07-24T21:44:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:44:55.151+0000 7f58e2b5d640 1 -- 172.21.15.60:0/206347067 wait complete. 2024-07-24T21:44:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:55 smithi123 ceph-mon[37143]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:55 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/206347067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-24T21:44:55.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:55 smithi060 ceph-mon[33000]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:55.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:55 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/206347067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-24T21:44:55.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:55 smithi086 ceph-mon[37128]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-07-24T21:44:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:55 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/206347067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-24T21:44:56.633 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528373 got 193273528377 for osd.6 2024-07-24T21:44:56.633 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:57.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:57 smithi123 ceph-mon[37143]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:57.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:57 smithi060 ceph-mon[33000]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:57.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:57 smithi086 ceph-mon[37128]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:44:58.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906989 for osd.9 2024-07-24T21:44:58.230 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:58.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332146 got 219043332149 for osd.7 2024-07-24T21:44:58.623 DEBUG:teuthology.parallel:result is None 2024-07-24T21:44:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:44:59 smithi123 ceph-mon[37143]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:59.540 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:44:59 smithi060 ceph-mon[33000]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:44:59.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:44:59 smithi086 ceph-mon[37128]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- 172.21.15.60:0/379772019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c107070 msgr2=0x7f927c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 --2- 172.21.15.60:0/379772019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c107070 0x7f927c1074b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9264009900 tx=0x7f926402a410 comp rx=0 tx=0).stop 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- 172.21.15.60:0/379772019 shutdown_connections 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 --2- 172.21.15.60:0/379772019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c1079f0 0x7f927c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 --2- 172.21.15.60:0/379772019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c107070 0x7f927c1074b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 --2- 172.21.15.60:0/379772019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c101660 0x7f927c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- 172.21.15.60:0/379772019 >> 172.21.15.60:0/379772019 conn(0x7f927c0fd0f0 msgr2=0x7f927c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:00.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- 172.21.15.60:0/379772019 shutdown_connections 2024-07-24T21:45:00.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- 172.21.15.60:0/379772019 wait complete. 2024-07-24T21:45:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 Processor -- start 2024-07-24T21:45:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.109+0000 7f9282855640 1 -- start start 2024-07-24T21:45:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c107070 0x7f927c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 0x7f927c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f927c111670 con 0x7f927c101660 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1114f0 con 0x7f927c1079f0 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9282855640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1117f0 con 0x7f927c107070 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38602/0 (socket says 172.21.15.60:38602) 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 -- 172.21.15.60:0/2097458835 learned_addr learned my addr 172.21.15.60:0/2097458835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9273fff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c107070 0x7f927c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9280df2640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 0x7f927c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 -- 172.21.15.60:0/2097458835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c107070 msgr2=0x7f927c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c107070 0x7f927c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 -- 172.21.15.60:0/2097458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 msgr2=0x7f927c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 0x7f927c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 -- 172.21.15.60:0/2097458835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1a0ea0 con 0x7f927c101660 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f9280df2640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 0x7f927c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.110+0000 7f927bfff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c920 tx=0x7f926c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.111+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c007c70 con 0x7f927c101660 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.111+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c007e10 con 0x7f927c101660 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.111+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1a1190 con 0x7f927c101660 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.111+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c018820 con 0x7f927c101660 2024-07-24T21:45:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.111+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a7850 con 0x7f927c101660 2024-07-24T21:45:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.112+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c102910 con 0x7f927c101660 2024-07-24T21:45:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.113+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f926c00f450 con 0x7f927c101660 2024-07-24T21:45:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.114+0000 7f9279ffb640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 0x7f92680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.115+0000 7f9273fff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 0x7f92680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.115+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f926c0e4350 con 0x7f927c101660 2024-07-24T21:45:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.115+0000 7f9273fff640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 0x7f92680b6170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9264005ab0 tx=0x7f9264005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:00.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.118+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f926c014030 con 0x7f927c101660 2024-07-24T21:45:00.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.315+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f927c1a7be0 con 0x7f927c101660 2024-07-24T21:45:00.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.315+0000 7f9279ffb640 1 -- 172.21.15.60:0/2097458835 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f926c0ab750 con 0x7f927c101660 2024-07-24T21:45:00.316 INFO:teuthology.orchestra.run.smithi060.stdout:167503724609 2024-07-24T21:45:00.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 msgr2=0x7f92680b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 0x7f92680b6170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9264005ab0 tx=0x7f9264005a20 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 msgr2=0x7f927c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c920 tx=0x7f926c00cde0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 shutdown_connections 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92680b3cb0 0x7f92680b6170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1079f0 0x7f927c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f927c107070 0x7f927c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 --2- 172.21.15.60:0/2097458835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c101660 0x7f927c19adf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 >> 172.21.15.60:0/2097458835 conn(0x7f927c0fd0f0 msgr2=0x7f927c06b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 shutdown_connections 2024-07-24T21:45:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.317+0000 7f9282855640 1 -- 172.21.15.60:0/2097458835 wait complete. 2024-07-24T21:45:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 -- 172.21.15.60:0/1679062930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 msgr2=0x7feffc103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 --2- 172.21.15.60:0/1679062930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc103910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fefec009900 tx=0x7fefec02a410 comp rx=0 tx=0).stop 2024-07-24T21:45:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 -- 172.21.15.60:0/1679062930 shutdown_connections 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 --2- 172.21.15.60:0/1679062930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 --2- 172.21.15.60:0/1679062930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 0x7feffc10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 --2- 172.21.15.60:0/1679062930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc103910 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 -- 172.21.15.60:0/1679062930 >> 172.21.15.60:0/1679062930 conn(0x7feffc0fd100 msgr2=0x7feffc0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 -- 172.21.15.60:0/1679062930 shutdown_connections 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 -- 172.21.15.60:0/1679062930 wait complete. 2024-07-24T21:45:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.728+0000 7ff0010e1640 1 Processor -- start 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 -- start start 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 0x7feffc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feffc112730 con 0x7feffc103530 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1125b0 con 0x7feffc10cae0 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7ff0010e1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1128b0 con 0x7feffc103f30 2024-07-24T21:45:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38622/0 (socket says 172.21.15.60:38622) 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 -- 172.21.15.60:0/2022007385 learned_addr learned my addr 172.21.15.60:0/2022007385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feff259c640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 0x7feffc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffb59e640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 -- 172.21.15.60:0/2022007385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 msgr2=0x7feffc1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 0x7feffc1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.729+0000 7feffad9d640 1 -- 172.21.15.60:0/2022007385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 msgr2=0x7feffc19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feffad9d640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feffad9d640 1 -- 172.21.15.60:0/2022007385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc19ed50 con 0x7feffc103530 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feffb59e640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feffad9d640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefec006010 tx=0x7fefec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec03f030 con 0x7feffc103530 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec003800 con 0x7feffc103530 2024-07-24T21:45:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc19efe0 con 0x7feffc103530 2024-07-24T21:45:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec035070 con 0x7feffc103530 2024-07-24T21:45:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.730+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc1a5700 con 0x7feffc103530 2024-07-24T21:45:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.731+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc103910 con 0x7feffc103530 2024-07-24T21:45:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.732+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fefec003cb0 con 0x7feffc103530 2024-07-24T21:45:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.733+0000 7feff8d99640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 0x7fefe80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.733+0000 7feff259c640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 0x7fefe80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.733+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fefec104aa0 con 0x7feffc103530 2024-07-24T21:45:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.734+0000 7feff259c640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 0x7fefe80b6170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feffc10f0e0 tx=0x7fefe400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.736+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fefec0cbe70 con 0x7feffc103530 2024-07-24T21:45:00.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.933+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7feffc199dd0 con 0x7feffc103530 2024-07-24T21:45:00.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.933+0000 7feff8d99640 1 -- 172.21.15.60:0/2022007385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7feffc199dd0 con 0x7feffc103530 2024-07-24T21:45:00.934 INFO:teuthology.orchestra.run.smithi060.stdout:296352743465 2024-07-24T21:45:00.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 msgr2=0x7fefe80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 0x7fefe80b6170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feffc10f0e0 tx=0x7fefe400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 msgr2=0x7feffc198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:00.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefec006010 tx=0x7fefec0042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 shutdown_connections 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe80b3cb0 0x7fefe80b6170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc10cae0 0x7feffc19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc103f30 0x7feffc1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 --2- 172.21.15.60:0/2022007385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc103530 0x7feffc198ca0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 >> 172.21.15.60:0/2022007385 conn(0x7feffc0fd100 msgr2=0x7feffc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 shutdown_connections 2024-07-24T21:45:00.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:00.935+0000 7ff0010e1640 1 -- 172.21.15.60:0/2022007385 wait complete. 2024-07-24T21:45:01.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:01 smithi060 ceph-mon[33000]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:01.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:01 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2097458835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-24T21:45:01.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:01 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2022007385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-24T21:45:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:01 smithi123 ceph-mon[37143]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:01 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2097458835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-24T21:45:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:01 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2022007385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-24T21:45:01.498 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724603 got 167503724609 for osd.5 2024-07-24T21:45:01.498 DEBUG:teuthology.parallel:result is None 2024-07-24T21:45:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:01 smithi086 ceph-mon[37128]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:01.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:01 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2097458835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-24T21:45:01.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:01 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2022007385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-24T21:45:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 -- 172.21.15.60:0/2942500540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 msgr2=0x7faa801085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 --2- 172.21.15.60:0/2942500540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa801085e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa7002a410 comp rx=0 tx=0).stop 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 -- 172.21.15.60:0/2942500540 shutdown_connections 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 --2- 172.21.15.60:0/2942500540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 0x7faa8010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 --2- 172.21.15.60:0/2942500540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa801085e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 --2- 172.21.15.60:0/2942500540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 0x7faa80102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 -- 172.21.15.60:0/2942500540 >> 172.21.15.60:0/2942500540 conn(0x7faa800fe200 msgr2=0x7faa80100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 -- 172.21.15.60:0/2942500540 shutdown_connections 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.847+0000 7faa870c4640 1 -- 172.21.15.60:0/2942500540 wait complete. 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 Processor -- start 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 -- start start 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 0x7faa8019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 0x7faa801a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faa80112730 con 0x7faa801081a0 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa801125b0 con 0x7faa80102790 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa870c4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa801128b0 con 0x7faa80108b20 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38652/0 (socket says 172.21.15.60:38652) 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa77fff640 1 -- 172.21.15.60:0/613978178 learned_addr learned my addr 172.21.15.60:0/613978178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.848+0000 7faa84e60640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 0x7faa8019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa85661640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 0x7faa801a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 -- 172.21.15.60:0/613978178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 msgr2=0x7faa801a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 0x7faa801a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 -- 172.21.15.60:0/613978178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 msgr2=0x7faa8019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 0x7faa8019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 -- 172.21.15.60:0/613978178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa801a0f30 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa77fff640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faa70005940 tx=0x7faa700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7003f030 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa70003cb0 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801a11c0 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70035070 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.849+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a78e0 con 0x7faa801081a0 2024-07-24T21:45:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.850+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa80102b70 con 0x7faa801081a0 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.853+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7faa70042030 con 0x7faa801081a0 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.854+0000 7faa75ffb640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 0x7faa680b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.854+0000 7faa84e60640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 0x7faa680b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.854+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7faa70104910 con 0x7faa801081a0 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.854+0000 7faa84e60640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 0x7faa680b61f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa7c005d00 tx=0x7faa7c00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:01.856+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7faa700cb690 con 0x7faa801081a0 2024-07-24T21:45:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.059+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7faa8010ed10 con 0x7faa801081a0 2024-07-24T21:45:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.060+0000 7faa75ffb640 1 -- 172.21.15.60:0/613978178 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7faa700cfbe0 con 0x7faa801081a0 2024-07-24T21:45:02.061 INFO:teuthology.orchestra.run.smithi060.stdout:322122547237 2024-07-24T21:45:02.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 msgr2=0x7faa680b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 0x7faa680b61f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa7c005d00 tx=0x7faa7c00b410 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 msgr2=0x7faa8019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faa70005940 tx=0x7faa700042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 shutdown_connections 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faa680b3d30 0x7faa680b61f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa80108b20 0x7faa801a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa801081a0 0x7faa8019b3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 --2- 172.21.15.60:0/613978178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa80102790 0x7faa8019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 >> 172.21.15.60:0/613978178 conn(0x7faa800fe200 msgr2=0x7faa80105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.062+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 shutdown_connections 2024-07-24T21:45:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:02.063+0000 7faa870c4640 1 -- 172.21.15.60:0/613978178 wait complete. 2024-07-24T21:45:02.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:02 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/613978178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-24T21:45:02.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743460 got 296352743465 for osd.10 2024-07-24T21:45:02.371 DEBUG:teuthology.parallel:result is None 2024-07-24T21:45:02.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:02 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/613978178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-24T21:45:02.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:02 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/613978178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-24T21:45:02.629 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:03.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:03 smithi060 ceph-mon[33000]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:03.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:03 smithi123 ceph-mon[37143]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:03.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:03 smithi086 ceph-mon[37128]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:03.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547232 got 322122547237 for osd.11 2024-07-24T21:45:03.592 DEBUG:teuthology.parallel:result is None 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 -- 172.21.15.60:0/3152953889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 msgr2=0x7f4114101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 --2- 172.21.15.60:0/3152953889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114101a30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4104009900 tx=0x7f410402a410 comp rx=0 tx=0).stop 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 -- 172.21.15.60:0/3152953889 shutdown_connections 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 --2- 172.21.15.60:0/3152953889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 --2- 172.21.15.60:0/3152953889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 --2- 172.21.15.60:0/3152953889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114101a30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 -- 172.21.15.60:0/3152953889 >> 172.21.15.60:0/3152953889 conn(0x7f41140fd0c0 msgr2=0x7f41140ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:05.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 -- 172.21.15.60:0/3152953889 shutdown_connections 2024-07-24T21:45:05.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 -- 172.21.15.60:0/3152953889 wait complete. 2024-07-24T21:45:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.079+0000 7f411baf1640 1 Processor -- start 2024-07-24T21:45:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 -- start start 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f41141127e0 con 0x7f4114101650 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4114112660 con 0x7f41141079e0 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411baf1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4114112960 con 0x7f4114107060 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38668/0 (socket says 172.21.15.60:38668) 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 -- 172.21.15.60:0/861387003 learned_addr learned my addr 172.21.15.60:0/861387003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411908c640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411a08e640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 -- 172.21.15.60:0/861387003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 msgr2=0x7f41141992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 -- 172.21.15.60:0/861387003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 msgr2=0x7f411419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411988d640 1 -- 172.21.15.60:0/861387003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411419ee40 con 0x7f4114101650 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411a08e640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.080+0000 7f411908c640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f411988d640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4104006010 tx=0x7f41040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410403f030 con 0x7f4114101650 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4104003800 con 0x7f4114101650 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411419f0d0 con 0x7f4114101650 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104035070 con 0x7f4114101650 2024-07-24T21:45:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.081+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41141a57f0 con 0x7f4114101650 2024-07-24T21:45:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.082+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4114101ab0 con 0x7f4114101650 2024-07-24T21:45:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.083+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f4104003cb0 con 0x7f4114101650 2024-07-24T21:45:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.084+0000 7f4112ffd640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 0x7f40fc0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.084+0000 7f411908c640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 0x7f40fc0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.084+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4104104940 con 0x7f4114101650 2024-07-24T21:45:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.085+0000 7f411908c640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 0x7f40fc0b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4114100a20 tx=0x7f4108008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:05.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.089+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f41040cbd10 con 0x7f4114101650 2024-07-24T21:45:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.290+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4114109230 con 0x7f4114101650 2024-07-24T21:45:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.290+0000 7f4112ffd640 1 -- 172.21.15.60:0/861387003 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41040cb610 con 0x7f4114101650 2024-07-24T21:45:05.292 INFO:teuthology.orchestra.run.smithi060.stdout:81604378702 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 msgr2=0x7f40fc0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 0x7f40fc0b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4114100a20 tx=0x7f4108008040 comp rx=0 tx=0).stop 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 msgr2=0x7f4114198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4104006010 tx=0x7f41040042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 shutdown_connections 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f40fc0b3d30 0x7f40fc0b61f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41141079e0 0x7f411419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4114107060 0x7f41141992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 --2- 172.21.15.60:0/861387003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4114101650 0x7f4114198d90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:05.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 >> 172.21.15.60:0/861387003 conn(0x7f41140fd0c0 msgr2=0x7f41140ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:05.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 shutdown_connections 2024-07-24T21:45:05.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:05.293+0000 7f411baf1640 1 -- 172.21.15.60:0/861387003 wait complete. 2024-07-24T21:45:05.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:05 smithi060 ceph-mon[33000]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:05.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:05 smithi123 ceph-mon[37143]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:05.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:05 smithi086 ceph-mon[37128]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:06.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378697 got 81604378702 for osd.2 2024-07-24T21:45:06.102 DEBUG:teuthology.parallel:result is None 2024-07-24T21:45:06.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:06 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/861387003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-24T21:45:06.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/2054022787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e0101650 msgr2=0x7fb3e0101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:06.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/2054022787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e0101650 0x7fb3e0101a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009900 tx=0x7fb3d002a410 comp rx=0 tx=0).stop 2024-07-24T21:45:06.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/2054022787 shutdown_connections 2024-07-24T21:45:06.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/2054022787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e01079e0 0x7fb3e010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/2054022787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 0x7fb3e01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/2054022787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e0101650 0x7fb3e0101a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/2054022787 >> 172.21.15.60:0/2054022787 conn(0x7fb3e00fd0c0 msgr2=0x7fb3e00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/2054022787 shutdown_connections 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.384+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/2054022787 wait complete. 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 Processor -- start 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 -- start start 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e0101650 0x7fb3e0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 0x7fb3e01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e01116e0 con 0x7fb3e01079e0 2024-07-24T21:45:06.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0111560 con 0x7fb3e0107060 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3e5f6c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0111860 con 0x7fb3e0101650 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60320/0 (socket says 172.21.15.60:60320) 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3dffff640 1 -- 172.21.15.60:0/3077333932 learned_addr learned my addr 172.21.15.60:0/3077333932 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.385+0000 7fb3df7fe640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e0101650 0x7fb3e0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3d6ffd640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 0x7fb3e01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 -- 172.21.15.60:0/3077333932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e0101650 msgr2=0x7fb3e0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e0101650 0x7fb3e0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 -- 172.21.15.60:0/3077333932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 msgr2=0x7fb3e01992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 0x7fb3e01992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 -- 172.21.15.60:0/3077333932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e019ee40 con 0x7fb3e01079e0 2024-07-24T21:45:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dffff640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800b970 tx=0x7fb3d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8019070 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d8014440 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e019f130 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80075d0 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.386+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e01a5870 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.387+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e0101ab0 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.389+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fb3d801c070 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.389+0000 7fb3dd7fa640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 0x7fb3cc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.389+0000 7fb3df7fe640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 0x7fb3cc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.389+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb3d80e3820 con 0x7fb3e01079e0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.390+0000 7fb3df7fe640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 0x7fb3cc0b60e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0031440 tx=0x7fb3d0005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:06.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.392+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb3d80aa6a0 con 0x7fb3e01079e0 2024-07-24T21:45:06.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:06 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/861387003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-24T21:45:06.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:06 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/861387003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-24T21:45:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.589+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb3e0199ec0 con 0x7fb3e01079e0 2024-07-24T21:45:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.590+0000 7fb3dd7fa640 1 -- 172.21.15.60:0/3077333932 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3d80aebf0 con 0x7fb3e01079e0 2024-07-24T21:45:06.591 INFO:teuthology.orchestra.run.smithi060.stdout:38654705752 2024-07-24T21:45:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 msgr2=0x7fb3cc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 0x7fb3cc0b60e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0031440 tx=0x7fb3d0005e80 comp rx=0 tx=0).stop 2024-07-24T21:45:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 msgr2=0x7fb3e019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800b970 tx=0x7fb3d800be30 comp rx=0 tx=0).stop 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 shutdown_connections 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb3cc0b3c20 0x7fb3cc0b60e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e01079e0 0x7fb3e019e670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3e0107060 0x7fb3e01992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 --2- 172.21.15.60:0/3077333932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3e0101650 0x7fb3e0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 >> 172.21.15.60:0/3077333932 conn(0x7fb3e00fd0c0 msgr2=0x7fb3e006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 shutdown_connections 2024-07-24T21:45:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:06.592+0000 7fb3e5f6c640 1 -- 172.21.15.60:0/3077333932 wait complete. 2024-07-24T21:45:07.349 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705747 got 38654705752 for osd.0 2024-07-24T21:45:07.349 DEBUG:teuthology.parallel:result is None 2024-07-24T21:45:07.349 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-24T21:45:07.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:45:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:07 smithi123 ceph-mon[37143]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:07 smithi123 ceph-mon[37143]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-07-24T21:45:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:07 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3077333932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-24T21:45:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:07 smithi086 ceph-mon[37128]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:07 smithi086 ceph-mon[37128]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-07-24T21:45:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:07 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3077333932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-24T21:45:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:07 smithi060 ceph-mon[33000]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:07 smithi060 ceph-mon[33000]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-07-24T21:45:07.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:07 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3077333932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-24T21:45:07.624 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:09.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.389+0000 7fe653d4f640 1 -- 172.21.15.60:0/1312832615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101230 msgr2=0x7fe64c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.389+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1312832615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101230 0x7fe64c101610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe648009ca0 tx=0x7fe64802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.389+0000 7fe653d4f640 1 -- 172.21.15.60:0/1312832615 shutdown_connections 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1312832615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 0x7fe64c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1312832615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101b50 0x7fe64c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1312832615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101230 0x7fe64c101610 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- 172.21.15.60:0/1312832615 >> 172.21.15.60:0/1312832615 conn(0x7fe64c0fd100 msgr2=0x7fe64c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- 172.21.15.60:0/1312832615 shutdown_connections 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- 172.21.15.60:0/1312832615 wait complete. 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 Processor -- start 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- start start 2024-07-24T21:45:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101230 0x7fe64c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 0x7fe64c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c114960 con 0x7fe64c101b50 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1147e0 con 0x7fe64c10ede0 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.390+0000 7fe653d4f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c114ae0 con 0x7fe64c101230 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60326/0 (socket says 172.21.15.60:60326) 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 -- 172.21.15.60:0/1146782833 learned_addr learned my addr 172.21.15.60:0/1146782833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6522ec640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 0x7fe64c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe651aeb640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101230 0x7fe64c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:09.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 -- 172.21.15.60:0/1146782833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101230 msgr2=0x7fe64c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101230 0x7fe64c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 -- 172.21.15.60:0/1146782833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 msgr2=0x7fe64c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 0x7fe64c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 -- 172.21.15.60:0/1146782833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1a0ea0 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe6512ea640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c920 tx=0x7fe63c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c007c70 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c007e10 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1a1190 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c018820 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.391+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1a7a20 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.392+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c101610 con 0x7fe64c101b50 2024-07-24T21:45:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.394+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fe63c00f450 con 0x7fe64c101b50 2024-07-24T21:45:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.394+0000 7fe642ffd640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 0x7fe6380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.394+0000 7fe651aeb640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 0x7fe6380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.394+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe63c0e4110 con 0x7fe64c101b50 2024-07-24T21:45:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.395+0000 7fe651aeb640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 0x7fe6380b6160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.397+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe63c014030 con 0x7fe64c101b50 2024-07-24T21:45:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:09 smithi123 ceph-mon[37143]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:09.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:09 smithi086 ceph-mon[37128]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:09.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:09 smithi060 ceph-mon[33000]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.590+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe64c110150 con 0x7fe6380b3ca0 2024-07-24T21:45:09.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.594+0000 7fe642ffd640 1 -- 172.21.15.60:0/1146782833 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258845 (secure 0 0 0) 0x7fe64c110150 con 0x7fe6380b3ca0 2024-07-24T21:45:09.598 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:45:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 msgr2=0x7fe6380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 0x7fe6380b6160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648000f40 comp rx=0 tx=0).stop 2024-07-24T21:45:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 msgr2=0x7fe64c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c920 tx=0x7fe63c00cde0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 shutdown_connections 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3ca0 0x7fe6380b6160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c10ede0 0x7fe64c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe64c101b50 0x7fe64c19b330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 --2- 172.21.15.60:0/1146782833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64c101230 0x7fe64c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.597+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 >> 172.21.15.60:0/1146782833 conn(0x7fe64c0fd100 msgr2=0x7fe64c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.598+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 shutdown_connections 2024-07-24T21:45:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:09.598+0000 7fe653d4f640 1 -- 172.21.15.60:0/1146782833 wait complete. 2024-07-24T21:45:10.444 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-07-24T21:45:08.066276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654755,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":689,"num_read_kb":441,"num_write":295,"num_write_kb":2266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":469,"ondisk_log_size":469,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404248,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537576,"statfs":{"total":1151940427776,"available":1151526477824,"internally_reserved":0,"allocated":13774848,"data_stored":7729047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":46,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743306+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:44:37.743306+0000","last_peered":"2024-07-24T21:44:37.743306+0000","last_clean":"2024-07-24T21:44:37.743306+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:44:37.743306+0000","last_undegraded":"2024-07-24T21:44:37.743306+0000","last_fullsized":"2024-07-24T21:44:37.743306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067795+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.067795+0000","last_peered":"2024-07-24T21:45:06.067795+0000","last_clean":"2024-07-24T21:45:06.067795+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.067795+0000","last_undegraded":"2024-07-24T21:45:06.067795+0000","last_fullsized":"2024-07-24T21:45:06.067795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072928+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.072928+0000","last_peered":"2024-07-24T21:45:06.072928+0000","last_clean":"2024-07-24T21:45:06.072928+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.072928+0000","last_undegraded":"2024-07-24T21:45:06.072928+0000","last_fullsized":"2024-07-24T21:45:06.072928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072029+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072029+0000","last_peered":"2024-07-24T21:45:06.072029+0000","last_clean":"2024-07-24T21:45:06.072029+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072029+0000","last_undegraded":"2024-07-24T21:45:06.072029+0000","last_fullsized":"2024-07-24T21:45:06.072029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740836+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:44:37.740836+0000","last_peered":"2024-07-24T21:44:37.740836+0000","last_clean":"2024-07-24T21:44:37.740836+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:44:37.740836+0000","last_undegraded":"2024-07-24T21:44:37.740836+0000","last_fullsized":"2024-07-24T21:44:37.740836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056196+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056196+0000","last_peered":"2024-07-24T21:45:06.056196+0000","last_clean":"2024-07-24T21:45:06.056196+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056196+0000","last_undegraded":"2024-07-24T21:45:06.056196+0000","last_fullsized":"2024-07-24T21:45:06.056196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065977+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.065977+0000","last_peered":"2024-07-24T21:45:06.065977+0000","last_clean":"2024-07-24T21:45:06.065977+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.065977+0000","last_undegraded":"2024-07-24T21:45:06.065977+0000","last_fullsized":"2024-07-24T21:45:06.065977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.737811+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:44:37.737811+0000","last_peered":"2024-07-24T21:44:37.737811+0000","last_clean":"2024-07-24T21:44:37.737811+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:44:37.737811+0000","last_undegraded":"2024-07-24T21:44:37.737811+0000","last_fullsized":"2024-07-24T21:44:37.737811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891446+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:44:37.891446+0000","last_peered":"2024-07-24T21:44:37.891446+0000","last_clean":"2024-07-24T21:44:37.891446+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:44:37.891446+0000","last_undegraded":"2024-07-24T21:44:37.891446+0000","last_fullsized":"2024-07-24T21:44:37.891446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059158+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059158+0000","last_peered":"2024-07-24T21:45:06.059158+0000","last_clean":"2024-07-24T21:45:06.059158+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059158+0000","last_undegraded":"2024-07-24T21:45:06.059158+0000","last_fullsized":"2024-07-24T21:45:06.059158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074599+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074599+0000","last_peered":"2024-07-24T21:45:06.074599+0000","last_clean":"2024-07-24T21:45:06.074599+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074599+0000","last_undegraded":"2024-07-24T21:45:06.074599+0000","last_fullsized":"2024-07-24T21:45:06.074599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071669+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071669+0000","last_peered":"2024-07-24T21:45:06.071669+0000","last_clean":"2024-07-24T21:45:06.071669+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071669+0000","last_undegraded":"2024-07-24T21:45:06.071669+0000","last_fullsized":"2024-07-24T21:45:06.071669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061601+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061601+0000","last_peered":"2024-07-24T21:45:06.061601+0000","last_clean":"2024-07-24T21:45:06.061601+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061601+0000","last_undegraded":"2024-07-24T21:45:06.061601+0000","last_fullsized":"2024-07-24T21:45:06.061601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071671+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071671+0000","last_peered":"2024-07-24T21:45:06.071671+0000","last_clean":"2024-07-24T21:45:06.071671+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071671+0000","last_undegraded":"2024-07-24T21:45:06.071671+0000","last_fullsized":"2024-07-24T21:45:06.071671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067822+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067822+0000","last_peered":"2024-07-24T21:45:06.067822+0000","last_clean":"2024-07-24T21:45:06.067822+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067822+0000","last_undegraded":"2024-07-24T21:45:06.067822+0000","last_fullsized":"2024-07-24T21:45:06.067822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.890953+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:44:37.890953+0000","last_peered":"2024-07-24T21:44:37.890953+0000","last_clean":"2024-07-24T21:44:37.890953+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:44:37.890953+0000","last_undegraded":"2024-07-24T21:44:37.890953+0000","last_fullsized":"2024-07-24T21:44:37.890953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743366+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:44:37.743366+0000","last_peered":"2024-07-24T21:44:37.743366+0000","last_clean":"2024-07-24T21:44:37.743366+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:44:37.743366+0000","last_undegraded":"2024-07-24T21:44:37.743366+0000","last_fullsized":"2024-07-24T21:44:37.743366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056795+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056795+0000","last_peered":"2024-07-24T21:45:06.056795+0000","last_clean":"2024-07-24T21:45:06.056795+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056795+0000","last_undegraded":"2024-07-24T21:45:06.056795+0000","last_fullsized":"2024-07-24T21:45:06.056795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742261+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:44:37.742261+0000","last_peered":"2024-07-24T21:44:37.742261+0000","last_clean":"2024-07-24T21:44:37.742261+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:44:37.742261+0000","last_undegraded":"2024-07-24T21:44:37.742261+0000","last_fullsized":"2024-07-24T21:44:37.742261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742853+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:44:37.742853+0000","last_peered":"2024-07-24T21:44:37.742853+0000","last_clean":"2024-07-24T21:44:37.742853+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:44:37.742853+0000","last_undegraded":"2024-07-24T21:44:37.742853+0000","last_fullsized":"2024-07-24T21:44:37.742853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066365+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066365+0000","last_peered":"2024-07-24T21:45:06.066365+0000","last_clean":"2024-07-24T21:45:06.066365+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066365+0000","last_undegraded":"2024-07-24T21:45:06.066365+0000","last_fullsized":"2024-07-24T21:45:06.066365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891201+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:44:37.891201+0000","last_peered":"2024-07-24T21:44:37.891201+0000","last_clean":"2024-07-24T21:44:37.891201+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:44:37.891201+0000","last_undegraded":"2024-07-24T21:44:37.891201+0000","last_fullsized":"2024-07-24T21:44:37.891201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.737868+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:44:37.737868+0000","last_peered":"2024-07-24T21:44:37.737868+0000","last_clean":"2024-07-24T21:44:37.737868+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:44:37.737868+0000","last_undegraded":"2024-07-24T21:44:37.737868+0000","last_fullsized":"2024-07-24T21:44:37.737868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081074+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:44:38.081074+0000","last_peered":"2024-07-24T21:44:38.081074+0000","last_clean":"2024-07-24T21:44:38.081074+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:44:38.081074+0000","last_undegraded":"2024-07-24T21:44:38.081074+0000","last_fullsized":"2024-07-24T21:44:38.081074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738319+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:44:37.738319+0000","last_peered":"2024-07-24T21:44:37.738319+0000","last_clean":"2024-07-24T21:44:37.738319+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:44:37.738319+0000","last_undegraded":"2024-07-24T21:44:37.738319+0000","last_fullsized":"2024-07-24T21:44:37.738319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742671+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:44:37.742671+0000","last_peered":"2024-07-24T21:44:37.742671+0000","last_clean":"2024-07-24T21:44:37.742671+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:44:37.742671+0000","last_undegraded":"2024-07-24T21:44:37.742671+0000","last_fullsized":"2024-07-24T21:44:37.742671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743064+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:44:37.743064+0000","last_peered":"2024-07-24T21:44:37.743064+0000","last_clean":"2024-07-24T21:44:37.743064+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:44:37.743064+0000","last_undegraded":"2024-07-24T21:44:37.743064+0000","last_fullsized":"2024-07-24T21:44:37.743064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065876+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.065876+0000","last_peered":"2024-07-24T21:45:06.065876+0000","last_clean":"2024-07-24T21:45:06.065876+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.065876+0000","last_undegraded":"2024-07-24T21:45:06.065876+0000","last_fullsized":"2024-07-24T21:45:06.065876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062795+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.062795+0000","last_peered":"2024-07-24T21:45:06.062795+0000","last_clean":"2024-07-24T21:45:06.062795+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.062795+0000","last_undegraded":"2024-07-24T21:45:06.062795+0000","last_fullsized":"2024-07-24T21:45:06.062795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742114+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:44:37.742114+0000","last_peered":"2024-07-24T21:44:37.742114+0000","last_clean":"2024-07-24T21:44:37.742114+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:44:37.742114+0000","last_undegraded":"2024-07-24T21:44:37.742114+0000","last_fullsized":"2024-07-24T21:44:37.742114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742142+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:44:37.742142+0000","last_peered":"2024-07-24T21:44:37.742142+0000","last_clean":"2024-07-24T21:44:37.742142+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:44:37.742142+0000","last_undegraded":"2024-07-24T21:44:37.742142+0000","last_fullsized":"2024-07-24T21:44:37.742142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743054+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:44:37.743054+0000","last_peered":"2024-07-24T21:44:37.743054+0000","last_clean":"2024-07-24T21:44:37.743054+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:44:37.743054+0000","last_undegraded":"2024-07-24T21:44:37.743054+0000","last_fullsized":"2024-07-24T21:44:37.743054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740806+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:44:37.740806+0000","last_peered":"2024-07-24T21:44:37.740806+0000","last_clean":"2024-07-24T21:44:37.740806+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:44:37.740806+0000","last_undegraded":"2024-07-24T21:44:37.740806+0000","last_fullsized":"2024-07-24T21:44:37.740806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:02.518079+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:45:02.518079+0000","last_peered":"2024-07-24T21:45:02.518079+0000","last_clean":"2024-07-24T21:45:02.518079+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:45:02.518079+0000","last_undegraded":"2024-07-24T21:45:02.518079+0000","last_fullsized":"2024-07-24T21:45:02.518079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081112+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:44:38.081112+0000","last_peered":"2024-07-24T21:44:38.081112+0000","last_clean":"2024-07-24T21:44:38.081112+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:44:38.081112+0000","last_undegraded":"2024-07-24T21:44:38.081112+0000","last_fullsized":"2024-07-24T21:44:38.081112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974115+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974115+0000","last_peered":"2024-07-24T21:44:37.974115+0000","last_clean":"2024-07-24T21:44:37.974115+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974115+0000","last_undegraded":"2024-07-24T21:44:37.974115+0000","last_fullsized":"2024-07-24T21:44:37.974115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739489+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739489+0000","last_peered":"2024-07-24T21:44:37.739489+0000","last_clean":"2024-07-24T21:44:37.739489+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739489+0000","last_undegraded":"2024-07-24T21:44:37.739489+0000","last_fullsized":"2024-07-24T21:44:37.739489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.737828+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:44:37.737828+0000","last_peered":"2024-07-24T21:44:37.737828+0000","last_clean":"2024-07-24T21:44:37.737828+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:44:37.737828+0000","last_undegraded":"2024-07-24T21:44:37.737828+0000","last_fullsized":"2024-07-24T21:44:37.737828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081260+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081260+0000","last_peered":"2024-07-24T21:44:38.081260+0000","last_clean":"2024-07-24T21:44:38.081260+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081260+0000","last_undegraded":"2024-07-24T21:44:38.081260+0000","last_fullsized":"2024-07-24T21:44:38.081260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053785+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053785+0000","last_peered":"2024-07-24T21:45:06.053785+0000","last_clean":"2024-07-24T21:45:06.053785+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053785+0000","last_undegraded":"2024-07-24T21:45:06.053785+0000","last_fullsized":"2024-07-24T21:45:06.053785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:04.520816+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:45:04.520816+0000","last_peered":"2024-07-24T21:45:04.520816+0000","last_clean":"2024-07-24T21:45:04.520816+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:45:04.520816+0000","last_undegraded":"2024-07-24T21:45:04.520816+0000","last_fullsized":"2024-07-24T21:45:04.520816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891419+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:44:37.891419+0000","last_peered":"2024-07-24T21:44:37.891419+0000","last_clean":"2024-07-24T21:44:37.891419+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:44:37.891419+0000","last_undegraded":"2024-07-24T21:44:37.891419+0000","last_fullsized":"2024-07-24T21:44:37.891419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741528+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741528+0000","last_peered":"2024-07-24T21:44:37.741528+0000","last_clean":"2024-07-24T21:44:37.741528+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741528+0000","last_undegraded":"2024-07-24T21:44:37.741528+0000","last_fullsized":"2024-07-24T21:44:37.741528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062829+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062829+0000","last_peered":"2024-07-24T21:45:06.062829+0000","last_clean":"2024-07-24T21:45:06.062829+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062829+0000","last_undegraded":"2024-07-24T21:45:06.062829+0000","last_fullsized":"2024-07-24T21:45:06.062829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081389+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081389+0000","last_peered":"2024-07-24T21:44:38.081389+0000","last_clean":"2024-07-24T21:44:38.081389+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081389+0000","last_undegraded":"2024-07-24T21:44:38.081389+0000","last_fullsized":"2024-07-24T21:44:38.081389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891055+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:44:37.891055+0000","last_peered":"2024-07-24T21:44:37.891055+0000","last_clean":"2024-07-24T21:44:37.891055+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:44:37.891055+0000","last_undegraded":"2024-07-24T21:44:37.891055+0000","last_fullsized":"2024-07-24T21:44:37.891055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":264,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.027885+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:45:06.027885+0000","last_peered":"2024-07-24T21:45:06.027885+0000","last_clean":"2024-07-24T21:45:06.027885+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:45:06.027885+0000","last_undegraded":"2024-07-24T21:45:06.027885+0000","last_fullsized":"2024-07-24T21:45:06.027885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":159,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740890+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:44:37.740890+0000","last_peered":"2024-07-24T21:44:37.740890+0000","last_clean":"2024-07-24T21:44:37.740890+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:44:37.740890+0000","last_undegraded":"2024-07-24T21:44:37.740890+0000","last_fullsized":"2024-07-24T21:44:37.740890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742400+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742400+0000","last_peered":"2024-07-24T21:44:37.742400+0000","last_clean":"2024-07-24T21:44:37.742400+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742400+0000","last_undegraded":"2024-07-24T21:44:37.742400+0000","last_fullsized":"2024-07-24T21:44:37.742400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063618+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063618+0000","last_peered":"2024-07-24T21:45:06.063618+0000","last_clean":"2024-07-24T21:45:06.063618+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063618+0000","last_undegraded":"2024-07-24T21:45:06.063618+0000","last_fullsized":"2024-07-24T21:45:06.063618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081127+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081127+0000","last_peered":"2024-07-24T21:44:38.081127+0000","last_clean":"2024-07-24T21:44:38.081127+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081127+0000","last_undegraded":"2024-07-24T21:44:38.081127+0000","last_fullsized":"2024-07-24T21:44:38.081127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066482+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066482+0000","last_peered":"2024-07-24T21:45:06.066482+0000","last_clean":"2024-07-24T21:45:06.066482+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066482+0000","last_undegraded":"2024-07-24T21:45:06.066482+0000","last_fullsized":"2024-07-24T21:45:06.066482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.549513+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:45:06.549513+0000","last_peered":"2024-07-24T21:45:06.549513+0000","last_clean":"2024-07-24T21:45:06.549513+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:45:06.549513+0000","last_undegraded":"2024-07-24T21:45:06.549513+0000","last_fullsized":"2024-07-24T21:45:06.549513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066079+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066079+0000","last_peered":"2024-07-24T21:45:06.066079+0000","last_clean":"2024-07-24T21:45:06.066079+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066079+0000","last_undegraded":"2024-07-24T21:45:06.066079+0000","last_fullsized":"2024-07-24T21:45:06.066079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740677+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740677+0000","last_peered":"2024-07-24T21:44:37.740677+0000","last_clean":"2024-07-24T21:44:37.740677+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740677+0000","last_undegraded":"2024-07-24T21:44:37.740677+0000","last_fullsized":"2024-07-24T21:44:37.740677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081003+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:44:38.081003+0000","last_peered":"2024-07-24T21:44:38.081003+0000","last_clean":"2024-07-24T21:44:38.081003+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:44:38.081003+0000","last_undegraded":"2024-07-24T21:44:38.081003+0000","last_fullsized":"2024-07-24T21:44:38.081003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741177+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741177+0000","last_peered":"2024-07-24T21:44:37.741177+0000","last_clean":"2024-07-24T21:44:37.741177+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741177+0000","last_undegraded":"2024-07-24T21:44:37.741177+0000","last_fullsized":"2024-07-24T21:44:37.741177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743149+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:44:37.743149+0000","last_peered":"2024-07-24T21:44:37.743149+0000","last_clean":"2024-07-24T21:44:37.743149+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:44:37.743149+0000","last_undegraded":"2024-07-24T21:44:37.743149+0000","last_fullsized":"2024-07-24T21:44:37.743149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.894806+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.894806+0000","last_peered":"2024-07-24T21:44:37.894806+0000","last_clean":"2024-07-24T21:44:37.894806+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.894806+0000","last_undegraded":"2024-07-24T21:44:37.894806+0000","last_fullsized":"2024-07-24T21:44:37.894806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074172+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074172+0000","last_peered":"2024-07-24T21:45:06.074172+0000","last_clean":"2024-07-24T21:45:06.074172+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074172+0000","last_undegraded":"2024-07-24T21:45:06.074172+0000","last_fullsized":"2024-07-24T21:45:06.074172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081459+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:44:38.081459+0000","last_peered":"2024-07-24T21:44:38.081459+0000","last_clean":"2024-07-24T21:44:38.081459+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:44:38.081459+0000","last_undegraded":"2024-07-24T21:44:38.081459+0000","last_fullsized":"2024-07-24T21:44:38.081459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066868+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.066868+0000","last_peered":"2024-07-24T21:45:06.066868+0000","last_clean":"2024-07-24T21:45:06.066868+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.066868+0000","last_undegraded":"2024-07-24T21:45:06.066868+0000","last_fullsized":"2024-07-24T21:45:06.066868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742578+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742578+0000","last_peered":"2024-07-24T21:44:37.742578+0000","last_clean":"2024-07-24T21:44:37.742578+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742578+0000","last_undegraded":"2024-07-24T21:44:37.742578+0000","last_fullsized":"2024-07-24T21:44:37.742578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073352+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073352+0000","last_peered":"2024-07-24T21:45:06.073352+0000","last_clean":"2024-07-24T21:45:06.073352+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073352+0000","last_undegraded":"2024-07-24T21:45:06.073352+0000","last_fullsized":"2024-07-24T21:45:06.073352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080770+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080770+0000","last_peered":"2024-07-24T21:44:38.080770+0000","last_clean":"2024-07-24T21:44:38.080770+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080770+0000","last_undegraded":"2024-07-24T21:44:38.080770+0000","last_fullsized":"2024-07-24T21:44:38.080770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738010+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:44:37.738010+0000","last_peered":"2024-07-24T21:44:37.738010+0000","last_clean":"2024-07-24T21:44:37.738010+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:44:37.738010+0000","last_undegraded":"2024-07-24T21:44:37.738010+0000","last_fullsized":"2024-07-24T21:44:37.738010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741191+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741191+0000","last_peered":"2024-07-24T21:44:37.741191+0000","last_clean":"2024-07-24T21:44:37.741191+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741191+0000","last_undegraded":"2024-07-24T21:44:37.741191+0000","last_fullsized":"2024-07-24T21:44:37.741191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063907+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.063907+0000","last_peered":"2024-07-24T21:45:06.063907+0000","last_clean":"2024-07-24T21:45:06.063907+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.063907+0000","last_undegraded":"2024-07-24T21:45:06.063907+0000","last_fullsized":"2024-07-24T21:45:06.063907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.549148+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:45:06.549148+0000","last_peered":"2024-07-24T21:45:06.549148+0000","last_clean":"2024-07-24T21:45:06.549148+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:45:06.549148+0000","last_undegraded":"2024-07-24T21:45:06.549148+0000","last_fullsized":"2024-07-24T21:45:06.549148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081141+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:44:38.081141+0000","last_peered":"2024-07-24T21:44:38.081141+0000","last_clean":"2024-07-24T21:44:38.081141+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:44:38.081141+0000","last_undegraded":"2024-07-24T21:44:38.081141+0000","last_fullsized":"2024-07-24T21:44:38.081141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741358+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741358+0000","last_peered":"2024-07-24T21:44:37.741358+0000","last_clean":"2024-07-24T21:44:37.741358+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741358+0000","last_undegraded":"2024-07-24T21:44:37.741358+0000","last_fullsized":"2024-07-24T21:44:37.741358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069081+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069081+0000","last_peered":"2024-07-24T21:45:06.069081+0000","last_clean":"2024-07-24T21:45:06.069081+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069081+0000","last_undegraded":"2024-07-24T21:45:06.069081+0000","last_fullsized":"2024-07-24T21:45:06.069081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080699+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080699+0000","last_peered":"2024-07-24T21:44:38.080699+0000","last_clean":"2024-07-24T21:44:38.080699+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080699+0000","last_undegraded":"2024-07-24T21:44:38.080699+0000","last_fullsized":"2024-07-24T21:44:38.080699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.051846+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.051846+0000","last_peered":"2024-07-24T21:45:06.051846+0000","last_clean":"2024-07-24T21:45:06.051846+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.051846+0000","last_undegraded":"2024-07-24T21:45:06.051846+0000","last_fullsized":"2024-07-24T21:45:06.051846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742882+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.742882+0000","last_peered":"2024-07-24T21:44:37.742882+0000","last_clean":"2024-07-24T21:44:37.742882+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.742882+0000","last_undegraded":"2024-07-24T21:44:37.742882+0000","last_fullsized":"2024-07-24T21:44:37.742882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.060947+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.060947+0000","last_peered":"2024-07-24T21:45:06.060947+0000","last_clean":"2024-07-24T21:45:06.060947+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.060947+0000","last_undegraded":"2024-07-24T21:45:06.060947+0000","last_fullsized":"2024-07-24T21:45:06.060947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080905+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:44:38.080905+0000","last_peered":"2024-07-24T21:44:38.080905+0000","last_clean":"2024-07-24T21:44:38.080905+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:44:38.080905+0000","last_undegraded":"2024-07-24T21:44:38.080905+0000","last_fullsized":"2024-07-24T21:44:38.080905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738335+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:44:37.738335+0000","last_peered":"2024-07-24T21:44:37.738335+0000","last_clean":"2024-07-24T21:44:37.738335+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:44:37.738335+0000","last_undegraded":"2024-07-24T21:44:37.738335+0000","last_fullsized":"2024-07-24T21:44:37.738335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080975+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.080975+0000","last_peered":"2024-07-24T21:44:38.080975+0000","last_clean":"2024-07-24T21:44:38.080975+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.080975+0000","last_undegraded":"2024-07-24T21:44:38.080975+0000","last_fullsized":"2024-07-24T21:44:38.080975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069379+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069379+0000","last_peered":"2024-07-24T21:45:06.069379+0000","last_clean":"2024-07-24T21:45:06.069379+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069379+0000","last_undegraded":"2024-07-24T21:45:06.069379+0000","last_fullsized":"2024-07-24T21:45:06.069379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891042+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891042+0000","last_peered":"2024-07-24T21:44:37.891042+0000","last_clean":"2024-07-24T21:44:37.891042+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891042+0000","last_undegraded":"2024-07-24T21:44:37.891042+0000","last_fullsized":"2024-07-24T21:44:37.891042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740630+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:44:37.740630+0000","last_peered":"2024-07-24T21:44:37.740630+0000","last_clean":"2024-07-24T21:44:37.740630+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:44:37.740630+0000","last_undegraded":"2024-07-24T21:44:37.740630+0000","last_fullsized":"2024-07-24T21:44:37.740630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.737994+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.737994+0000","last_peered":"2024-07-24T21:44:37.737994+0000","last_clean":"2024-07-24T21:44:37.737994+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.737994+0000","last_undegraded":"2024-07-24T21:44:37.737994+0000","last_fullsized":"2024-07-24T21:44:37.737994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743244+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:44:37.743244+0000","last_peered":"2024-07-24T21:44:37.743244+0000","last_clean":"2024-07-24T21:44:37.743244+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:44:37.743244+0000","last_undegraded":"2024-07-24T21:44:37.743244+0000","last_fullsized":"2024-07-24T21:44:37.743244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742442+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742442+0000","last_peered":"2024-07-24T21:44:37.742442+0000","last_clean":"2024-07-24T21:44:37.742442+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742442+0000","last_undegraded":"2024-07-24T21:44:37.742442+0000","last_fullsized":"2024-07-24T21:44:37.742442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738070+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:44:37.738070+0000","last_peered":"2024-07-24T21:44:37.738070+0000","last_clean":"2024-07-24T21:44:37.738070+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:44:37.738070+0000","last_undegraded":"2024-07-24T21:44:37.738070+0000","last_fullsized":"2024-07-24T21:44:37.738070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073428+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073428+0000","last_peered":"2024-07-24T21:45:06.073428+0000","last_clean":"2024-07-24T21:45:06.073428+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073428+0000","last_undegraded":"2024-07-24T21:45:06.073428+0000","last_fullsized":"2024-07-24T21:45:06.073428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081306+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:44:38.081306+0000","last_peered":"2024-07-24T21:44:38.081306+0000","last_clean":"2024-07-24T21:44:38.081306+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:44:38.081306+0000","last_undegraded":"2024-07-24T21:44:38.081306+0000","last_fullsized":"2024-07-24T21:44:38.081306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741102+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741102+0000","last_peered":"2024-07-24T21:44:37.741102+0000","last_clean":"2024-07-24T21:44:37.741102+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741102+0000","last_undegraded":"2024-07-24T21:44:37.741102+0000","last_fullsized":"2024-07-24T21:44:37.741102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891039+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:44:37.891039+0000","last_peered":"2024-07-24T21:44:37.891039+0000","last_clean":"2024-07-24T21:44:37.891039+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:44:37.891039+0000","last_undegraded":"2024-07-24T21:44:37.891039+0000","last_fullsized":"2024-07-24T21:44:37.891039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740965+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.740965+0000","last_peered":"2024-07-24T21:44:37.740965+0000","last_clean":"2024-07-24T21:44:37.740965+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.740965+0000","last_undegraded":"2024-07-24T21:44:37.740965+0000","last_fullsized":"2024-07-24T21:44:37.740965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073053+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073053+0000","last_peered":"2024-07-24T21:45:06.073053+0000","last_clean":"2024-07-24T21:45:06.073053+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073053+0000","last_undegraded":"2024-07-24T21:45:06.073053+0000","last_fullsized":"2024-07-24T21:45:06.073053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738452+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738452+0000","last_peered":"2024-07-24T21:44:37.738452+0000","last_clean":"2024-07-24T21:44:37.738452+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738452+0000","last_undegraded":"2024-07-24T21:44:37.738452+0000","last_fullsized":"2024-07-24T21:44:37.738452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738276+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:44:37.738276+0000","last_peered":"2024-07-24T21:44:37.738276+0000","last_clean":"2024-07-24T21:44:37.738276+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:44:37.738276+0000","last_undegraded":"2024-07-24T21:44:37.738276+0000","last_fullsized":"2024-07-24T21:44:37.738276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080772+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:44:38.080772+0000","last_peered":"2024-07-24T21:44:38.080772+0000","last_clean":"2024-07-24T21:44:38.080772+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:44:38.080772+0000","last_undegraded":"2024-07-24T21:44:38.080772+0000","last_fullsized":"2024-07-24T21:44:38.080772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64153,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":608,"num_read_kb":372,"num_write":170,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1133046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1072401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":541552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":540202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40468,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704684,"statfs":{"total":95995035648,"available":95953596416,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:45:10.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:45:10.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:10 smithi123 ceph-mon[37143]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:10.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:10 smithi086 ceph-mon[37128]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:10.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:10 smithi060 ceph-mon[33000]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:10.717 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:11 smithi123 ceph-mon[37143]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:11.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:11 smithi060 ceph-mon[33000]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:11 smithi086 ceph-mon[37128]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 -- 172.21.15.60:0/1165684364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898107a10 msgr2=0x7fd89810e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 --2- 172.21.15.60:0/1165684364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898107a10 0x7fd89810e210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd890009ca0 tx=0x7fd89002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 -- 172.21.15.60:0/1165684364 shutdown_connections 2024-07-24T21:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 --2- 172.21.15.60:0/1165684364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898107a10 0x7fd89810e210 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 --2- 172.21.15.60:0/1165684364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 0x7fd8981074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 --2- 172.21.15.60:0/1165684364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898101680 0x7fd898101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 -- 172.21.15.60:0/1165684364 >> 172.21.15.60:0/1165684364 conn(0x7fd8980fd0d0 msgr2=0x7fd8980ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 -- 172.21.15.60:0/1165684364 shutdown_connections 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.332+0000 7fd89da70640 1 -- 172.21.15.60:0/1165684364 wait complete. 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 Processor -- start 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 -- start start 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 0x7fd89819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 0x7fd8981a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd898111640 con 0x7fd898101680 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8981114c0 con 0x7fd898107a10 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd89da70640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8981117c0 con 0x7fd898107090 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd8977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd8977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60336/0 (socket says 172.21.15.60:60336) 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd8977fe640 1 -- 172.21.15.60:0/1893821105 learned_addr learned my addr 172.21.15.60:0/1893821105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:12.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd88ffff640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 0x7fd89819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.333+0000 7fd897fff640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 0x7fd8981a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 -- 172.21.15.60:0/1893821105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 msgr2=0x7fd89819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 0x7fd89819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 -- 172.21.15.60:0/1893821105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 msgr2=0x7fd8981a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 0x7fd8981a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 -- 172.21.15.60:0/1893821105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8981a1180 con 0x7fd898101680 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd897fff640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 0x7fd8981a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8977fe640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c890 tx=0x7fd88800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888004220 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8981a1470 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8880043c0 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888004a00 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.334+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8981a7d00 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.335+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd898101a60 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.337+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fd8880079b0 con 0x7fd898101680 2024-07-24T21:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.338+0000 7fd8957fa640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 0x7fd8840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.338+0000 7fd88ffff640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 0x7fd8840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.338+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd8880e3ee0 con 0x7fd898101680 2024-07-24T21:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.339+0000 7fd88ffff640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 0x7fd8840b60e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd898106ed0 tx=0x7fd88000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.341+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd888014030 con 0x7fd898101680 2024-07-24T21:45:12.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:45:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.538+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd89806ac20 con 0x7fd8840b3c20 2024-07-24T21:45:12.540 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:45:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.543+0000 7fd8957fa640 1 -- 172.21.15.60:0/1893821105 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258919 (secure 0 0 0) 0x7fd89806ac20 con 0x7fd8840b3c20 2024-07-24T21:45:12.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:45:12.550 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:12.552 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:45:12.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 msgr2=0x7fd8840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:12.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 0x7fd8840b60e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd898106ed0 tx=0x7fd88000b390 comp rx=0 tx=0).stop 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 msgr2=0x7fd89819b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c890 tx=0x7fd88800cd50 comp rx=0 tx=0).stop 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 shutdown_connections 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8840b3c20 0x7fd8840b60e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd898107a10 0x7fd8981a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd898107090 0x7fd89819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 --2- 172.21.15.60:0/1893821105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd898101680 0x7fd89819b0d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:12.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 >> 172.21.15.60:0/1893821105 conn(0x7fd8980fd0d0 msgr2=0x7fd89806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:12.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 shutdown_connections 2024-07-24T21:45:12.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:12.547+0000 7fd89da70640 1 -- 172.21.15.60:0/1893821105 wait complete. 2024-07-24T21:45:13.599 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-07-24T21:45:12.067420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654755,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":715,"num_read_kb":452,"num_write":295,"num_write_kb":2266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":739,"ondisk_log_size":739,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404204,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537620,"statfs":{"total":1151940427776,"available":1151526522880,"internally_reserved":0,"allocated":13774848,"data_stored":7729545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003119"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068429+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.068429+0000","last_peered":"2024-07-24T21:45:06.068429+0000","last_clean":"2024-07-24T21:45:06.068429+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.068429+0000","last_undegraded":"2024-07-24T21:45:06.068429+0000","last_fullsized":"2024-07-24T21:45:06.068429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067795+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.067795+0000","last_peered":"2024-07-24T21:45:06.067795+0000","last_clean":"2024-07-24T21:45:06.067795+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.067795+0000","last_undegraded":"2024-07-24T21:45:06.067795+0000","last_fullsized":"2024-07-24T21:45:06.067795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072928+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.072928+0000","last_peered":"2024-07-24T21:45:06.072928+0000","last_clean":"2024-07-24T21:45:06.072928+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.072928+0000","last_undegraded":"2024-07-24T21:45:06.072928+0000","last_fullsized":"2024-07-24T21:45:06.072928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068097+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068097+0000","last_peered":"2024-07-24T21:45:06.068097+0000","last_clean":"2024-07-24T21:45:06.068097+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068097+0000","last_undegraded":"2024-07-24T21:45:06.068097+0000","last_fullsized":"2024-07-24T21:45:06.068097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056196+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056196+0000","last_peered":"2024-07-24T21:45:06.056196+0000","last_clean":"2024-07-24T21:45:06.056196+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056196+0000","last_undegraded":"2024-07-24T21:45:06.056196+0000","last_fullsized":"2024-07-24T21:45:06.056196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065977+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.065977+0000","last_peered":"2024-07-24T21:45:06.065977+0000","last_clean":"2024-07-24T21:45:06.065977+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.065977+0000","last_undegraded":"2024-07-24T21:45:06.065977+0000","last_fullsized":"2024-07-24T21:45:06.065977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072406+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072406+0000","last_peered":"2024-07-24T21:45:06.072406+0000","last_clean":"2024-07-24T21:45:06.072406+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072406+0000","last_undegraded":"2024-07-24T21:45:06.072406+0000","last_fullsized":"2024-07-24T21:45:06.072406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072557+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.072557+0000","last_peered":"2024-07-24T21:45:06.072557+0000","last_clean":"2024-07-24T21:45:06.072557+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.072557+0000","last_undegraded":"2024-07-24T21:45:06.072557+0000","last_fullsized":"2024-07-24T21:45:06.072557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059158+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059158+0000","last_peered":"2024-07-24T21:45:06.059158+0000","last_clean":"2024-07-24T21:45:06.059158+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059158+0000","last_undegraded":"2024-07-24T21:45:06.059158+0000","last_fullsized":"2024-07-24T21:45:06.059158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074599+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074599+0000","last_peered":"2024-07-24T21:45:06.074599+0000","last_clean":"2024-07-24T21:45:06.074599+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074599+0000","last_undegraded":"2024-07-24T21:45:06.074599+0000","last_fullsized":"2024-07-24T21:45:06.074599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071669+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071669+0000","last_peered":"2024-07-24T21:45:06.071669+0000","last_clean":"2024-07-24T21:45:06.071669+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071669+0000","last_undegraded":"2024-07-24T21:45:06.071669+0000","last_fullsized":"2024-07-24T21:45:06.071669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061601+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061601+0000","last_peered":"2024-07-24T21:45:06.061601+0000","last_clean":"2024-07-24T21:45:06.061601+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061601+0000","last_undegraded":"2024-07-24T21:45:06.061601+0000","last_fullsized":"2024-07-24T21:45:06.061601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071671+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071671+0000","last_peered":"2024-07-24T21:45:06.071671+0000","last_clean":"2024-07-24T21:45:06.071671+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071671+0000","last_undegraded":"2024-07-24T21:45:06.071671+0000","last_fullsized":"2024-07-24T21:45:06.071671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067822+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067822+0000","last_peered":"2024-07-24T21:45:06.067822+0000","last_clean":"2024-07-24T21:45:06.067822+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067822+0000","last_undegraded":"2024-07-24T21:45:06.067822+0000","last_fullsized":"2024-07-24T21:45:06.067822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071648+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.071648+0000","last_peered":"2024-07-24T21:45:06.071648+0000","last_clean":"2024-07-24T21:45:06.071648+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.071648+0000","last_undegraded":"2024-07-24T21:45:06.071648+0000","last_fullsized":"2024-07-24T21:45:06.071648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070468+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070468+0000","last_peered":"2024-07-24T21:45:06.070468+0000","last_clean":"2024-07-24T21:45:06.070468+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070468+0000","last_undegraded":"2024-07-24T21:45:06.070468+0000","last_fullsized":"2024-07-24T21:45:06.070468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056795+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056795+0000","last_peered":"2024-07-24T21:45:06.056795+0000","last_clean":"2024-07-24T21:45:06.056795+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056795+0000","last_undegraded":"2024-07-24T21:45:06.056795+0000","last_fullsized":"2024-07-24T21:45:06.056795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069114+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069114+0000","last_peered":"2024-07-24T21:45:06.069114+0000","last_clean":"2024-07-24T21:45:06.069114+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069114+0000","last_undegraded":"2024-07-24T21:45:06.069114+0000","last_fullsized":"2024-07-24T21:45:06.069114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056081+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.056081+0000","last_peered":"2024-07-24T21:45:06.056081+0000","last_clean":"2024-07-24T21:45:06.056081+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.056081+0000","last_undegraded":"2024-07-24T21:45:06.056081+0000","last_fullsized":"2024-07-24T21:45:06.056081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066365+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066365+0000","last_peered":"2024-07-24T21:45:06.066365+0000","last_clean":"2024-07-24T21:45:06.066365+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066365+0000","last_undegraded":"2024-07-24T21:45:06.066365+0000","last_fullsized":"2024-07-24T21:45:06.066365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072060+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.072060+0000","last_peered":"2024-07-24T21:45:06.072060+0000","last_clean":"2024-07-24T21:45:06.072060+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.072060+0000","last_undegraded":"2024-07-24T21:45:06.072060+0000","last_fullsized":"2024-07-24T21:45:06.072060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068884+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.068884+0000","last_peered":"2024-07-24T21:45:06.068884+0000","last_clean":"2024-07-24T21:45:06.068884+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.068884+0000","last_undegraded":"2024-07-24T21:45:06.068884+0000","last_fullsized":"2024-07-24T21:45:06.068884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067566+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067566+0000","last_peered":"2024-07-24T21:45:06.067566+0000","last_clean":"2024-07-24T21:45:06.067566+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067566+0000","last_undegraded":"2024-07-24T21:45:06.067566+0000","last_fullsized":"2024-07-24T21:45:06.067566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063376+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.063376+0000","last_peered":"2024-07-24T21:45:06.063376+0000","last_clean":"2024-07-24T21:45:06.063376+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.063376+0000","last_undegraded":"2024-07-24T21:45:06.063376+0000","last_fullsized":"2024-07-24T21:45:06.063376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073204+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073204+0000","last_peered":"2024-07-24T21:45:06.073204+0000","last_clean":"2024-07-24T21:45:06.073204+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073204+0000","last_undegraded":"2024-07-24T21:45:06.073204+0000","last_fullsized":"2024-07-24T21:45:06.073204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.054884+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.054884+0000","last_peered":"2024-07-24T21:45:06.054884+0000","last_clean":"2024-07-24T21:45:06.054884+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.054884+0000","last_undegraded":"2024-07-24T21:45:06.054884+0000","last_fullsized":"2024-07-24T21:45:06.054884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065876+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.065876+0000","last_peered":"2024-07-24T21:45:06.065876+0000","last_clean":"2024-07-24T21:45:06.065876+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.065876+0000","last_undegraded":"2024-07-24T21:45:06.065876+0000","last_fullsized":"2024-07-24T21:45:06.065876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062795+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.062795+0000","last_peered":"2024-07-24T21:45:06.062795+0000","last_clean":"2024-07-24T21:45:06.062795+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.062795+0000","last_undegraded":"2024-07-24T21:45:06.062795+0000","last_fullsized":"2024-07-24T21:45:06.062795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066524+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066524+0000","last_peered":"2024-07-24T21:45:06.066524+0000","last_clean":"2024-07-24T21:45:06.066524+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066524+0000","last_undegraded":"2024-07-24T21:45:06.066524+0000","last_fullsized":"2024-07-24T21:45:06.066524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064518+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064518+0000","last_peered":"2024-07-24T21:45:06.064518+0000","last_clean":"2024-07-24T21:45:06.064518+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064518+0000","last_undegraded":"2024-07-24T21:45:06.064518+0000","last_fullsized":"2024-07-24T21:45:06.064518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065780+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.065780+0000","last_peered":"2024-07-24T21:45:06.065780+0000","last_clean":"2024-07-24T21:45:06.065780+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.065780+0000","last_undegraded":"2024-07-24T21:45:06.065780+0000","last_fullsized":"2024-07-24T21:45:06.065780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067829+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.067829+0000","last_peered":"2024-07-24T21:45:06.067829+0000","last_clean":"2024-07-24T21:45:06.067829+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.067829+0000","last_undegraded":"2024-07-24T21:45:06.067829+0000","last_fullsized":"2024-07-24T21:45:06.067829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:07.525825+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:45:07.525825+0000","last_peered":"2024-07-24T21:45:07.525825+0000","last_clean":"2024-07-24T21:45:07.525825+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:45:07.525825+0000","last_undegraded":"2024-07-24T21:45:07.525825+0000","last_fullsized":"2024-07-24T21:45:07.525825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070262+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070262+0000","last_peered":"2024-07-24T21:45:06.070262+0000","last_clean":"2024-07-24T21:45:06.070262+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070262+0000","last_undegraded":"2024-07-24T21:45:06.070262+0000","last_fullsized":"2024-07-24T21:45:06.070262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974115+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974115+0000","last_peered":"2024-07-24T21:44:37.974115+0000","last_clean":"2024-07-24T21:44:37.974115+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974115+0000","last_undegraded":"2024-07-24T21:44:37.974115+0000","last_fullsized":"2024-07-24T21:44:37.974115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739489+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739489+0000","last_peered":"2024-07-24T21:44:37.739489+0000","last_clean":"2024-07-24T21:44:37.739489+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739489+0000","last_undegraded":"2024-07-24T21:44:37.739489+0000","last_fullsized":"2024-07-24T21:44:37.739489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068639+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.068639+0000","last_peered":"2024-07-24T21:45:06.068639+0000","last_clean":"2024-07-24T21:45:06.068639+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.068639+0000","last_undegraded":"2024-07-24T21:45:06.068639+0000","last_fullsized":"2024-07-24T21:45:06.068639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081260+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081260+0000","last_peered":"2024-07-24T21:44:38.081260+0000","last_clean":"2024-07-24T21:44:38.081260+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081260+0000","last_undegraded":"2024-07-24T21:44:38.081260+0000","last_fullsized":"2024-07-24T21:44:38.081260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053785+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053785+0000","last_peered":"2024-07-24T21:45:06.053785+0000","last_clean":"2024-07-24T21:45:06.053785+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053785+0000","last_undegraded":"2024-07-24T21:45:06.053785+0000","last_fullsized":"2024-07-24T21:45:06.053785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":135,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:09.527873+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:45:09.527873+0000","last_peered":"2024-07-24T21:45:09.527873+0000","last_clean":"2024-07-24T21:45:09.527873+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:45:09.527873+0000","last_undegraded":"2024-07-24T21:45:09.527873+0000","last_fullsized":"2024-07-24T21:45:09.527873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073895+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.073895+0000","last_peered":"2024-07-24T21:45:06.073895+0000","last_clean":"2024-07-24T21:45:06.073895+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.073895+0000","last_undegraded":"2024-07-24T21:45:06.073895+0000","last_fullsized":"2024-07-24T21:45:06.073895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741528+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741528+0000","last_peered":"2024-07-24T21:44:37.741528+0000","last_clean":"2024-07-24T21:44:37.741528+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741528+0000","last_undegraded":"2024-07-24T21:44:37.741528+0000","last_fullsized":"2024-07-24T21:44:37.741528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062829+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062829+0000","last_peered":"2024-07-24T21:45:06.062829+0000","last_clean":"2024-07-24T21:45:06.062829+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062829+0000","last_undegraded":"2024-07-24T21:45:06.062829+0000","last_fullsized":"2024-07-24T21:45:06.062829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081389+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081389+0000","last_peered":"2024-07-24T21:44:38.081389+0000","last_clean":"2024-07-24T21:44:38.081389+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081389+0000","last_undegraded":"2024-07-24T21:44:38.081389+0000","last_fullsized":"2024-07-24T21:44:38.081389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074174+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074174+0000","last_peered":"2024-07-24T21:45:06.074174+0000","last_clean":"2024-07-24T21:45:06.074174+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074174+0000","last_undegraded":"2024-07-24T21:45:06.074174+0000","last_fullsized":"2024-07-24T21:45:06.074174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":275,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:10.534086+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:45:10.534086+0000","last_peered":"2024-07-24T21:45:10.534086+0000","last_clean":"2024-07-24T21:45:10.534086+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:45:10.534086+0000","last_undegraded":"2024-07-24T21:45:10.534086+0000","last_fullsized":"2024-07-24T21:45:10.534086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":165,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071981+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.071981+0000","last_peered":"2024-07-24T21:45:06.071981+0000","last_clean":"2024-07-24T21:45:06.071981+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.071981+0000","last_undegraded":"2024-07-24T21:45:06.071981+0000","last_fullsized":"2024-07-24T21:45:06.071981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742400+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742400+0000","last_peered":"2024-07-24T21:44:37.742400+0000","last_clean":"2024-07-24T21:44:37.742400+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742400+0000","last_undegraded":"2024-07-24T21:44:37.742400+0000","last_fullsized":"2024-07-24T21:44:37.742400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063618+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063618+0000","last_peered":"2024-07-24T21:45:06.063618+0000","last_clean":"2024-07-24T21:45:06.063618+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063618+0000","last_undegraded":"2024-07-24T21:45:06.063618+0000","last_fullsized":"2024-07-24T21:45:06.063618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081127+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081127+0000","last_peered":"2024-07-24T21:44:38.081127+0000","last_clean":"2024-07-24T21:44:38.081127+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081127+0000","last_undegraded":"2024-07-24T21:44:38.081127+0000","last_fullsized":"2024-07-24T21:44:38.081127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066482+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066482+0000","last_peered":"2024-07-24T21:45:06.066482+0000","last_clean":"2024-07-24T21:45:06.066482+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066482+0000","last_undegraded":"2024-07-24T21:45:06.066482+0000","last_fullsized":"2024-07-24T21:45:06.066482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:10.556103+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:45:10.556103+0000","last_peered":"2024-07-24T21:45:10.556103+0000","last_clean":"2024-07-24T21:45:10.556103+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:45:10.556103+0000","last_undegraded":"2024-07-24T21:45:10.556103+0000","last_fullsized":"2024-07-24T21:45:10.556103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066079+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066079+0000","last_peered":"2024-07-24T21:45:06.066079+0000","last_clean":"2024-07-24T21:45:06.066079+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066079+0000","last_undegraded":"2024-07-24T21:45:06.066079+0000","last_fullsized":"2024-07-24T21:45:06.066079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740677+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740677+0000","last_peered":"2024-07-24T21:44:37.740677+0000","last_clean":"2024-07-24T21:44:37.740677+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740677+0000","last_undegraded":"2024-07-24T21:44:37.740677+0000","last_fullsized":"2024-07-24T21:44:37.740677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067258+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067258+0000","last_peered":"2024-07-24T21:45:06.067258+0000","last_clean":"2024-07-24T21:45:06.067258+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067258+0000","last_undegraded":"2024-07-24T21:45:06.067258+0000","last_fullsized":"2024-07-24T21:45:06.067258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741177+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741177+0000","last_peered":"2024-07-24T21:44:37.741177+0000","last_clean":"2024-07-24T21:44:37.741177+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741177+0000","last_undegraded":"2024-07-24T21:44:37.741177+0000","last_fullsized":"2024-07-24T21:44:37.741177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052808+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.052808+0000","last_peered":"2024-07-24T21:45:06.052808+0000","last_clean":"2024-07-24T21:45:06.052808+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.052808+0000","last_undegraded":"2024-07-24T21:45:06.052808+0000","last_fullsized":"2024-07-24T21:45:06.052808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.894806+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.894806+0000","last_peered":"2024-07-24T21:44:37.894806+0000","last_clean":"2024-07-24T21:44:37.894806+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.894806+0000","last_undegraded":"2024-07-24T21:44:37.894806+0000","last_fullsized":"2024-07-24T21:44:37.894806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074172+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074172+0000","last_peered":"2024-07-24T21:45:06.074172+0000","last_clean":"2024-07-24T21:45:06.074172+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074172+0000","last_undegraded":"2024-07-24T21:45:06.074172+0000","last_fullsized":"2024-07-24T21:45:06.074172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081459+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:44:38.081459+0000","last_peered":"2024-07-24T21:44:38.081459+0000","last_clean":"2024-07-24T21:44:38.081459+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:44:38.081459+0000","last_undegraded":"2024-07-24T21:44:38.081459+0000","last_fullsized":"2024-07-24T21:44:38.081459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066868+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.066868+0000","last_peered":"2024-07-24T21:45:06.066868+0000","last_clean":"2024-07-24T21:45:06.066868+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.066868+0000","last_undegraded":"2024-07-24T21:45:06.066868+0000","last_fullsized":"2024-07-24T21:45:06.066868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742578+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742578+0000","last_peered":"2024-07-24T21:44:37.742578+0000","last_clean":"2024-07-24T21:44:37.742578+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742578+0000","last_undegraded":"2024-07-24T21:44:37.742578+0000","last_fullsized":"2024-07-24T21:44:37.742578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073352+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073352+0000","last_peered":"2024-07-24T21:45:06.073352+0000","last_clean":"2024-07-24T21:45:06.073352+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073352+0000","last_undegraded":"2024-07-24T21:45:06.073352+0000","last_fullsized":"2024-07-24T21:45:06.073352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080770+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080770+0000","last_peered":"2024-07-24T21:44:38.080770+0000","last_clean":"2024-07-24T21:44:38.080770+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080770+0000","last_undegraded":"2024-07-24T21:44:38.080770+0000","last_fullsized":"2024-07-24T21:44:38.080770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072252+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072252+0000","last_peered":"2024-07-24T21:45:06.072252+0000","last_clean":"2024-07-24T21:45:06.072252+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072252+0000","last_undegraded":"2024-07-24T21:45:06.072252+0000","last_fullsized":"2024-07-24T21:45:06.072252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063907+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.063907+0000","last_peered":"2024-07-24T21:45:06.063907+0000","last_clean":"2024-07-24T21:45:06.063907+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.063907+0000","last_undegraded":"2024-07-24T21:45:06.063907+0000","last_fullsized":"2024-07-24T21:45:06.063907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.549148+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:45:06.549148+0000","last_peered":"2024-07-24T21:45:06.549148+0000","last_clean":"2024-07-24T21:45:06.549148+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:45:06.549148+0000","last_undegraded":"2024-07-24T21:45:06.549148+0000","last_fullsized":"2024-07-24T21:45:06.549148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068326+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.068326+0000","last_peered":"2024-07-24T21:45:06.068326+0000","last_clean":"2024-07-24T21:45:06.068326+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.068326+0000","last_undegraded":"2024-07-24T21:45:06.068326+0000","last_fullsized":"2024-07-24T21:45:06.068326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741358+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741358+0000","last_peered":"2024-07-24T21:44:37.741358+0000","last_clean":"2024-07-24T21:44:37.741358+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741358+0000","last_undegraded":"2024-07-24T21:44:37.741358+0000","last_fullsized":"2024-07-24T21:44:37.741358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069081+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069081+0000","last_peered":"2024-07-24T21:45:06.069081+0000","last_clean":"2024-07-24T21:45:06.069081+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069081+0000","last_undegraded":"2024-07-24T21:45:06.069081+0000","last_fullsized":"2024-07-24T21:45:06.069081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080699+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080699+0000","last_peered":"2024-07-24T21:44:38.080699+0000","last_clean":"2024-07-24T21:44:38.080699+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080699+0000","last_undegraded":"2024-07-24T21:44:38.080699+0000","last_fullsized":"2024-07-24T21:44:38.080699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.051846+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.051846+0000","last_peered":"2024-07-24T21:45:06.051846+0000","last_clean":"2024-07-24T21:45:06.051846+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.051846+0000","last_undegraded":"2024-07-24T21:45:06.051846+0000","last_fullsized":"2024-07-24T21:45:06.051846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742882+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.742882+0000","last_peered":"2024-07-24T21:44:37.742882+0000","last_clean":"2024-07-24T21:44:37.742882+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.742882+0000","last_undegraded":"2024-07-24T21:44:37.742882+0000","last_fullsized":"2024-07-24T21:44:37.742882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.060947+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.060947+0000","last_peered":"2024-07-24T21:45:06.060947+0000","last_clean":"2024-07-24T21:45:06.060947+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.060947+0000","last_undegraded":"2024-07-24T21:45:06.060947+0000","last_fullsized":"2024-07-24T21:45:06.060947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080905+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:44:38.080905+0000","last_peered":"2024-07-24T21:44:38.080905+0000","last_clean":"2024-07-24T21:44:38.080905+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:44:38.080905+0000","last_undegraded":"2024-07-24T21:44:38.080905+0000","last_fullsized":"2024-07-24T21:44:38.080905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071010+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071010+0000","last_peered":"2024-07-24T21:45:06.071010+0000","last_clean":"2024-07-24T21:45:06.071010+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071010+0000","last_undegraded":"2024-07-24T21:45:06.071010+0000","last_fullsized":"2024-07-24T21:45:06.071010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080975+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.080975+0000","last_peered":"2024-07-24T21:44:38.080975+0000","last_clean":"2024-07-24T21:44:38.080975+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.080975+0000","last_undegraded":"2024-07-24T21:44:38.080975+0000","last_fullsized":"2024-07-24T21:44:38.080975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069379+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069379+0000","last_peered":"2024-07-24T21:45:06.069379+0000","last_clean":"2024-07-24T21:45:06.069379+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069379+0000","last_undegraded":"2024-07-24T21:45:06.069379+0000","last_fullsized":"2024-07-24T21:45:06.069379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891042+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891042+0000","last_peered":"2024-07-24T21:44:37.891042+0000","last_clean":"2024-07-24T21:44:37.891042+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891042+0000","last_undegraded":"2024-07-24T21:44:37.891042+0000","last_fullsized":"2024-07-24T21:44:37.891042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.036891+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.036891+0000","last_peered":"2024-07-24T21:45:06.036891+0000","last_clean":"2024-07-24T21:45:06.036891+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.036891+0000","last_undegraded":"2024-07-24T21:45:06.036891+0000","last_fullsized":"2024-07-24T21:45:06.036891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.737994+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.737994+0000","last_peered":"2024-07-24T21:44:37.737994+0000","last_clean":"2024-07-24T21:44:37.737994+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.737994+0000","last_undegraded":"2024-07-24T21:44:37.737994+0000","last_fullsized":"2024-07-24T21:44:37.737994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.065296+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.065296+0000","last_peered":"2024-07-24T21:45:06.065296+0000","last_clean":"2024-07-24T21:45:06.065296+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.065296+0000","last_undegraded":"2024-07-24T21:45:06.065296+0000","last_fullsized":"2024-07-24T21:45:06.065296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742442+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742442+0000","last_peered":"2024-07-24T21:44:37.742442+0000","last_clean":"2024-07-24T21:44:37.742442+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742442+0000","last_undegraded":"2024-07-24T21:44:37.742442+0000","last_fullsized":"2024-07-24T21:44:37.742442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069384+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.069384+0000","last_peered":"2024-07-24T21:45:06.069384+0000","last_clean":"2024-07-24T21:45:06.069384+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.069384+0000","last_undegraded":"2024-07-24T21:45:06.069384+0000","last_fullsized":"2024-07-24T21:45:06.069384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073428+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073428+0000","last_peered":"2024-07-24T21:45:06.073428+0000","last_clean":"2024-07-24T21:45:06.073428+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073428+0000","last_undegraded":"2024-07-24T21:45:06.073428+0000","last_fullsized":"2024-07-24T21:45:06.073428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071729+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071729+0000","last_peered":"2024-07-24T21:45:06.071729+0000","last_clean":"2024-07-24T21:45:06.071729+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071729+0000","last_undegraded":"2024-07-24T21:45:06.071729+0000","last_fullsized":"2024-07-24T21:45:06.071729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741102+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741102+0000","last_peered":"2024-07-24T21:44:37.741102+0000","last_clean":"2024-07-24T21:44:37.741102+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741102+0000","last_undegraded":"2024-07-24T21:44:37.741102+0000","last_fullsized":"2024-07-24T21:44:37.741102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073755+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073755+0000","last_peered":"2024-07-24T21:45:06.073755+0000","last_clean":"2024-07-24T21:45:06.073755+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073755+0000","last_undegraded":"2024-07-24T21:45:06.073755+0000","last_fullsized":"2024-07-24T21:45:06.073755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740965+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.740965+0000","last_peered":"2024-07-24T21:44:37.740965+0000","last_clean":"2024-07-24T21:44:37.740965+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.740965+0000","last_undegraded":"2024-07-24T21:44:37.740965+0000","last_fullsized":"2024-07-24T21:44:37.740965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073053+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073053+0000","last_peered":"2024-07-24T21:45:06.073053+0000","last_clean":"2024-07-24T21:45:06.073053+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073053+0000","last_undegraded":"2024-07-24T21:45:06.073053+0000","last_fullsized":"2024-07-24T21:45:06.073053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071458+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.071458+0000","last_peered":"2024-07-24T21:45:06.071458+0000","last_clean":"2024-07-24T21:45:06.071458+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.071458+0000","last_undegraded":"2024-07-24T21:45:06.071458+0000","last_fullsized":"2024-07-24T21:45:06.071458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080772+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:44:38.080772+0000","last_peered":"2024-07-24T21:44:38.080772+0000","last_clean":"2024-07-24T21:44:38.080772+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:44:38.080772+0000","last_undegraded":"2024-07-24T21:44:38.080772+0000","last_fullsized":"2024-07-24T21:44:38.080772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64153,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":634,"num_read_kb":383,"num_write":170,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743468,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1133046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1072401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117068,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":541801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":540451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:45:13.603 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-24T21:45:13.603 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-24T21:45:13.603 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-24T21:45:13.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph health --format=json 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:13.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.009 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:45:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 -- 172.21.15.60:0/1119266023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 msgr2=0x7fc31c1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1119266023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c1074a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009900 tx=0x7fc30c02a410 comp rx=0 tx=0).stop 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 -- 172.21.15.60:0/1119266023 shutdown_connections 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1119266023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 0x7fc31c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1119266023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c1074a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1119266023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 0x7fc31c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 -- 172.21.15.60:0/1119266023 >> 172.21.15.60:0/1119266023 conn(0x7fc31c0fd0c0 msgr2=0x7fc31c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 -- 172.21.15.60:0/1119266023 shutdown_connections 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.776+0000 7fc3239c5640 1 -- 172.21.15.60:0/1119266023 wait complete. 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 Processor -- start 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 -- start start 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 0x7fc31c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 0x7fc31c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111680 con 0x7fc31c107060 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111500 con 0x7fc31c1079e0 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc3239c5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111800 con 0x7fc31c101650 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60352/0 (socket says 172.21.15.60:60352) 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 -- 172.21.15.60:0/1329743194 learned_addr learned my addr 172.21.15.60:0/1329743194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc321761640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 0x7fc31c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc321f62640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 0x7fc31c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 -- 172.21.15.60:0/1329743194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 msgr2=0x7fc31c198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 0x7fc31c198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 -- 172.21.15.60:0/1329743194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 msgr2=0x7fc31c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 0x7fc31c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.777+0000 7fc320f60640 1 -- 172.21.15.60:0/1329743194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c19ed70 con 0x7fc31c107060 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc320f60640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005940 tx=0x7fc30c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c03f030 con 0x7fc31c107060 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c003cb0 con 0x7fc31c107060 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c19f000 con 0x7fc31c107060 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c035070 con 0x7fc31c107060 2024-07-24T21:45:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.778+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1a5720 con 0x7fc31c107060 2024-07-24T21:45:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.779+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c101a30 con 0x7fc31c107060 2024-07-24T21:45:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.780+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc30c042030 con 0x7fc31c107060 2024-07-24T21:45:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.781+0000 7fc3127fc640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 0x7fc3080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.781+0000 7fc321761640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 0x7fc3080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.781+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc30c1048b0 con 0x7fc31c107060 2024-07-24T21:45:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.782+0000 7fc321761640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 0x7fc3080b60e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc318005fd0 tx=0x7fc31800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:15.784+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc30c0cb790 con 0x7fc31c107060 2024-07-24T21:45:15.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:15 smithi060 ceph-mon[33000]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:16.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:15 smithi086 ceph-mon[37128]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.084+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc31c1a5a70 con 0x7fc31c107060 2024-07-24T21:45:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.084+0000 7fc3127fc640 1 -- 172.21.15.60:0/1329743194 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc30c0cfce0 con 0x7fc31c107060 2024-07-24T21:45:16.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:16.086 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-24T21:45:16.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 msgr2=0x7fc3080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 0x7fc3080b60e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc318005fd0 tx=0x7fc31800b390 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 msgr2=0x7fc31c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005940 tx=0x7fc30c0042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 shutdown_connections 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3080b3c20 0x7fc3080b60e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc31c1079e0 0x7fc31c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c107060 0x7fc31c199200 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 --2- 172.21.15.60:0/1329743194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc31c101650 0x7fc31c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 >> 172.21.15.60:0/1329743194 conn(0x7fc31c0fd0c0 msgr2=0x7fc31c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 shutdown_connections 2024-07-24T21:45:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:16.086+0000 7fc3239c5640 1 -- 172.21.15.60:0/1329743194 wait complete. 2024-07-24T21:45:16.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:15 smithi123 ceph-mon[37143]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-07-24T21:45:17.009 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-24T21:45:17.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:45:17.282 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:17.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-07-24T21:45:17.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1329743194' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-24T21:45:17.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:45:17.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:45:17.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1329743194' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:45:17.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1329743194' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:45:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:45:19.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.015+0000 7f3c4675e640 1 -- 172.21.15.60:0/2294728241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c401070a0 msgr2=0x7f3c401074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.015+0000 7f3c4675e640 1 --2- 172.21.15.60:0/2294728241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c401070a0 0x7f3c401074e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009900 tx=0x7f3c3402a410 comp rx=0 tx=0).stop 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 -- 172.21.15.60:0/2294728241 shutdown_connections 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 --2- 172.21.15.60:0/2294728241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c40107a20 0x7f3c4010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 --2- 172.21.15.60:0/2294728241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c401070a0 0x7f3c401074e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 --2- 172.21.15.60:0/2294728241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40101690 0x7f3c40101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 -- 172.21.15.60:0/2294728241 >> 172.21.15.60:0/2294728241 conn(0x7f3c400fd100 msgr2=0x7f3c400ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 -- 172.21.15.60:0/2294728241 shutdown_connections 2024-07-24T21:45:19.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 -- 172.21.15.60:0/2294728241 wait complete. 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 Processor -- start 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.016+0000 7f3c4675e640 1 -- start start 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c401070a0 0x7f3c40199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 0x7f3c4019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112800 con 0x7f3c40101690 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112680 con 0x7f3c401070a0 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c4675e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112980 con 0x7f3c40107a20 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52988/0 (socket says 172.21.15.60:52988) 2024-07-24T21:45:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 -- 172.21.15.60:0/3388187840 learned_addr learned my addr 172.21.15.60:0/3388187840 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c44cfb640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 0x7f3c4019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 -- 172.21.15.60:0/3388187840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 msgr2=0x7f3c4019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3f7fe640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c401070a0 0x7f3c40199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 0x7f3c4019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 -- 172.21.15.60:0/3388187840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c401070a0 msgr2=0x7f3c40199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c401070a0 0x7f3c40199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c3ffff640 1 -- 172.21.15.60:0/3388187840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4019eca0 con 0x7f3c40101690 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.017+0000 7f3c44cfb640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 0x7f3c4019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c3ffff640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c970 tx=0x7f3c3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30007bb0 con 0x7f3c40101690 2024-07-24T21:45:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c30007d50 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4019ef90 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000f450 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.018+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a5650 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.019+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40101a70 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.020+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f3c30004f00 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.021+0000 7f3c3d7fa640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 0x7f3c280b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.021+0000 7f3c3f7fe640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 0x7f3c280b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.021+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3c300e3f10 con 0x7f3c40101690 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.021+0000 7f3c3f7fe640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 0x7f3c280b6160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c34000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.024+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3c30014030 con 0x7f3c40101690 2024-07-24T21:45:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.262+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c4002e000 con 0x7f3c40101690 2024-07-24T21:45:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.263+0000 7f3c3d7fa640 1 -- 172.21.15.60:0/3388187840 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f3c300ab4c0 con 0x7f3c40101690 2024-07-24T21:45:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:19.265 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:45:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:45:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 msgr2=0x7f3c280b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 0x7f3c280b6160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c34000bc0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 msgr2=0x7f3c40198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c970 tx=0x7f3c3000ce30 comp rx=0 tx=0).stop 2024-07-24T21:45:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 shutdown_connections 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c280b3ca0 0x7f3c280b6160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c40107a20 0x7f3c4019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c401070a0 0x7f3c40199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 --2- 172.21.15.60:0/3388187840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c40101690 0x7f3c40198bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 >> 172.21.15.60:0/3388187840 conn(0x7f3c400fd100 msgr2=0x7f3c400ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 shutdown_connections 2024-07-24T21:45:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:19.265+0000 7f3c4675e640 1 -- 172.21.15.60:0/3388187840 wait complete. 2024-07-24T21:45:19.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:19 smithi123 ceph-mon[37143]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:19.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:19 smithi086 ceph-mon[37128]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:19 smithi060 ceph-mon[33000]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-24T21:45:19.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:45:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:20.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3388187840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:20.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3388187840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:20.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3388187840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:21.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:21 smithi060 ceph-mon[33000]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-24T21:45:21.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:21 smithi123 ceph-mon[37143]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-24T21:45:21.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:21 smithi086 ceph-mon[37128]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-24T21:45:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 -- 172.21.15.60:0/2113879785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac1079f0 msgr2=0x7fc8ac10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/2113879785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac10e1f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009ca0 tx=0x7fc8a402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 -- 172.21.15.60:0/2113879785 shutdown_connections 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/2113879785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac10e1f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/2113879785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/2113879785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac101660 0x7fc8ac101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 -- 172.21.15.60:0/2113879785 >> 172.21.15.60:0/2113879785 conn(0x7fc8ac0fd0f0 msgr2=0x7fc8ac0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 -- 172.21.15.60:0/2113879785 shutdown_connections 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.864+0000 7fc8b10fc640 1 -- 172.21.15.60:0/2113879785 wait complete. 2024-07-24T21:45:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 Processor -- start 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 -- start start 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112800 con 0x7fc8ac101660 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112680 con 0x7fc8ac1079f0 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8b10fc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac112980 con 0x7fc8ac107070 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53012/0 (socket says 172.21.15.60:53012) 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 -- 172.21.15.60:0/3778876726 learned_addr learned my addr 172.21.15.60:0/3778876726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8ab59e640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8a3fff640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 -- 172.21.15.60:0/3778876726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 msgr2=0x7fc8ac19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 -- 172.21.15.60:0/3778876726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac1079f0 msgr2=0x7fc8ac1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8aad9d640 1 -- 172.21.15.60:0/3778876726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1a0ff0 con 0x7fc8ac101660 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.865+0000 7fc8a3fff640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8aad9d640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c890 tx=0x7fc89c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c004220 con 0x7fc8ac101660 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c0043c0 con 0x7fc8ac101660 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1a12e0 con 0x7fc8ac101660 2024-07-24T21:45:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c004a00 con 0x7fc8ac101660 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.866+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a7b70 con 0x7fc8ac101660 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.867+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac101a40 con 0x7fc8ac101660 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.868+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc89c0079b0 con 0x7fc8ac101660 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.869+0000 7fc8a8d99640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 0x7fc8980b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.869+0000 7fc8a3fff640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 0x7fc8980b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.869+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc89c0e3a70 con 0x7fc8ac101660 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.869+0000 7fc8a3fff640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 0x7fc8980b6160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc894006fd0 tx=0x7fc894008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:21.873+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc89c014030 con 0x7fc8ac101660 2024-07-24T21:45:22.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.105+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8ac02e000 con 0x7fc8ac101660 2024-07-24T21:45:22.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.105+0000 7fc8a8d99640 1 -- 172.21.15.60:0/3778876726 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fc89c0aafa0 con 0x7fc8ac101660 2024-07-24T21:45:22.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:22.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:45:22.108 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 msgr2=0x7fc8980b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 0x7fc8980b6160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc894006fd0 tx=0x7fc894008040 comp rx=0 tx=0).stop 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 msgr2=0x7fc8ac19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c890 tx=0x7fc89c00cd50 comp rx=0 tx=0).stop 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 shutdown_connections 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8980b3ca0 0x7fc8980b6160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8ac1079f0 0x7fc8ac1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:22.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8ac107070 0x7fc8ac19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:22.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 --2- 172.21.15.60:0/3778876726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8ac101660 0x7fc8ac19af40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:22.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 >> 172.21.15.60:0/3778876726 conn(0x7fc8ac0fd0f0 msgr2=0x7fc8ac1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:22.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 shutdown_connections 2024-07-24T21:45:22.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:22.108+0000 7fc8b10fc640 1 -- 172.21.15.60:0/3778876726 wait complete. 2024-07-24T21:45:22.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3778876726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:22.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3778876726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:22.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3778876726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:22.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:45:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:23.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:23 smithi060 ceph-mon[33000]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-24T21:45:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:23 smithi123 ceph-mon[37143]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-24T21:45:23.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:23 smithi086 ceph-mon[37128]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 -- 172.21.15.60:0/2865148201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c1079e0 msgr2=0x7f923c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 --2- 172.21.15.60:0/2865148201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c1079e0 0x7f923c10e1e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9234009900 tx=0x7f923402a410 comp rx=0 tx=0).stop 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 -- 172.21.15.60:0/2865148201 shutdown_connections 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 --2- 172.21.15.60:0/2865148201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c1079e0 0x7f923c10e1e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 --2- 172.21.15.60:0/2865148201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 --2- 172.21.15.60:0/2865148201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c101650 0x7f923c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 -- 172.21.15.60:0/2865148201 >> 172.21.15.60:0/2865148201 conn(0x7f923c0fd0c0 msgr2=0x7f923c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 -- 172.21.15.60:0/2865148201 shutdown_connections 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.678+0000 7f9244321640 1 -- 172.21.15.60:0/2865148201 wait complete. 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 Processor -- start 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 -- start start 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c10f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 0x7f923c10f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f923c112780 con 0x7f923c101650 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f923c112600 con 0x7f923c107060 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f9244321640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f923c112900 con 0x7f923c1079e0 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f92420bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:24.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f92420bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53026/0 (socket says 172.21.15.60:53026) 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.679+0000 7f92420bd640 1 -- 172.21.15.60:0/470395546 learned_addr learned my addr 172.21.15.60:0/470395546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92418bc640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c10f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92428be640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 0x7f923c10f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 -- 172.21.15.60:0/470395546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 msgr2=0x7f923c10f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 0x7f923c10f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 -- 172.21.15.60:0/470395546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 msgr2=0x7f923c10f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c10f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 -- 172.21.15.60:0/470395546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c1a7030 con 0x7f923c101650 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92428be640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 0x7f923c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92418bc640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f92420bd640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f922c00cce0 tx=0x7f922c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c00f650 con 0x7f923c101650 2024-07-24T21:45:24.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f922c00fca0 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c1a7320 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c018b40 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.680+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c1a7860 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.681+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c101a30 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.684+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f922c018d00 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.685+0000 7f923b7fe640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 0x7f92240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.685+0000 7f92418bc640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 0x7f92240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.685+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f922c0e45d0 con 0x7f923c101650 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.685+0000 7f92418bc640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 0x7f92240b60e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9230005fd0 tx=0x7f9230005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:24.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.687+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f922c014030 con 0x7f923c101650 2024-07-24T21:45:24.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.884+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f923c1a7c80 con 0x7f923c101650 2024-07-24T21:45:24.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.885+0000 7f923b7fe640 1 -- 172.21.15.60:0/470395546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23071 (secure 0 0 0) 0x7f922c0abb30 con 0x7f923c101650 2024-07-24T21:45:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:24.888 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:44:37.734936+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:45:24.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.887+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 msgr2=0x7f92240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.887+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 0x7f92240b60e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9230005fd0 tx=0x7f9230005850 comp rx=0 tx=0).stop 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.887+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 msgr2=0x7f923c10ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.887+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f922c00cce0 tx=0x7f922c00a620 comp rx=0 tx=0).stop 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 shutdown_connections 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f92240b3c20 0x7f92240b60e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923c1079e0 0x7f923c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c107060 0x7f923c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 --2- 172.21.15.60:0/470395546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f923c101650 0x7f923c10ecf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 >> 172.21.15.60:0/470395546 conn(0x7f923c0fd0c0 msgr2=0x7f923c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 shutdown_connections 2024-07-24T21:45:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:24.888+0000 7f9244321640 1 -- 172.21.15.60:0/470395546 wait complete. 2024-07-24T21:45:25.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:25 smithi060 ceph-mon[33000]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:25.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:25 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/470395546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:45:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:25 smithi123 ceph-mon[37143]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:25.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:25 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/470395546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:45:25.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:45:25.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:25 smithi086 ceph-mon[37128]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:25.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:25 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/470395546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 -- 172.21.15.60:0/4272729473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea24101690 msgr2=0x7fea24101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 --2- 172.21.15.60:0/4272729473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea24101690 0x7fea24101a70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fea20009900 tx=0x7fea2002a410 comp rx=0 tx=0).stop 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 -- 172.21.15.60:0/4272729473 shutdown_connections 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 --2- 172.21.15.60:0/4272729473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 0x7fea2410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 --2- 172.21.15.60:0/4272729473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea241070a0 0x7fea241074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 --2- 172.21.15.60:0/4272729473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea24101690 0x7fea24101a70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 -- 172.21.15.60:0/4272729473 >> 172.21.15.60:0/4272729473 conn(0x7fea240fd100 msgr2=0x7fea240ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 -- 172.21.15.60:0/4272729473 shutdown_connections 2024-07-24T21:45:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.359+0000 7fea2c682640 1 -- 172.21.15.60:0/4272729473 wait complete. 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.360+0000 7fea2c682640 1 Processor -- start 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.360+0000 7fea2c682640 1 -- start start 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.360+0000 7fea2c682640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 0x7fea24197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.360+0000 7fea2c682640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.360+0000 7fea2c682640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 0x7fea2419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2c682640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea241116b0 con 0x7fea241070a0 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2c682640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111530 con 0x7fea24101690 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2c682640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111830 con 0x7fea24107a20 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60486/0 (socket says 172.21.15.60:60486) 2024-07-24T21:45:27.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 -- 172.21.15.60:0/3025148903 learned_addr learned my addr 172.21.15.60:0/3025148903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2ac1f640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 0x7fea2419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2a41e640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 0x7fea24197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 -- 172.21.15.60:0/3025148903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 msgr2=0x7fea2419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 0x7fea2419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 -- 172.21.15.60:0/3025148903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 msgr2=0x7fea24197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 0x7fea24197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 -- 172.21.15.60:0/3025148903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2419dc50 con 0x7fea241070a0 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2a41e640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 0x7fea24197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea29c1d640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d950 tx=0x7fea1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14017070 con 0x7fea241070a0 2024-07-24T21:45:27.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea14009c30 con 0x7fea241070a0 2024-07-24T21:45:27.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2419df40 con 0x7fea241070a0 2024-07-24T21:45:27.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.361+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1400f620 con 0x7fea241070a0 2024-07-24T21:45:27.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.362+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a4680 con 0x7fea241070a0 2024-07-24T21:45:27.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.363+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24101a70 con 0x7fea241070a0 2024-07-24T21:45:27.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.364+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fea14010460 con 0x7fea241070a0 2024-07-24T21:45:27.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.365+0000 7fea1b7fe640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 0x7fea100b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:27.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.365+0000 7fea2a41e640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 0x7fea100b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:27.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.365+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fea140e3a00 con 0x7fea241070a0 2024-07-24T21:45:27.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.365+0000 7fea2a41e640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 0x7fea100b60e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea20031440 tx=0x7fea20005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:27.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.368+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fea140aaed0 con 0x7fea241070a0 2024-07-24T21:45:27.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:27 smithi123 ceph-mon[37143]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:27.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:27 smithi086 ceph-mon[37128]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:27.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:27 smithi060 ceph-mon[33000]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.603+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea24198cd0 con 0x7fea241070a0 2024-07-24T21:45:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.604+0000 7fea1b7fe640 1 -- 172.21.15.60:0/3025148903 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fea24198cd0 con 0x7fea241070a0 2024-07-24T21:45:27.605 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:27.605 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:45:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 msgr2=0x7fea100b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 0x7fea100b60e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea20031440 tx=0x7fea20005a20 comp rx=0 tx=0).stop 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 msgr2=0x7fea241980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d950 tx=0x7fea1400de10 comp rx=0 tx=0).stop 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 shutdown_connections 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea100b3c20 0x7fea100b60e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea24107a20 0x7fea2419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea241070a0 0x7fea241980e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 --2- 172.21.15.60:0/3025148903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea24101690 0x7fea24197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:27.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 >> 172.21.15.60:0/3025148903 conn(0x7fea240fd100 msgr2=0x7fea240fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:27.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 shutdown_connections 2024-07-24T21:45:27.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:27.606+0000 7fea2c682640 1 -- 172.21.15.60:0/3025148903 wait complete. 2024-07-24T21:45:28.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:28 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3025148903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:28.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-24T21:43:14.079489+0000', 'modified': '2024-07-24T21:43:32.191585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24610, 'mds_1': 24605, 'mds_2': 24598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24560': {'gid': 24560, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/1999114136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1999114136}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1999114136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/549557535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 549557535}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 549557535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/3784109187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3784109187}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3784109187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/2950411419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2950411419}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2950411419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.86:6839/1688764220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 1688764220}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 1688764220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.123:6839/3718566677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 3718566677}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 3718566677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24610, 'qdb_cluster': [24610, 24605, 24598]} 2024-07-24T21:45:28.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-24T21:45:28.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.c status 2024-07-24T21:45:28.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:28 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3025148903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:28.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:28 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3025148903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:29.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:29 smithi060 ceph-mon[33000]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:29.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:29 smithi123 ceph-mon[37143]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:29.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:29 smithi086 ceph-mon[37128]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:30.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.107+0000 7ffbdc63d640 1 -- 172.21.15.60:0/2493647062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd41081a0 msgr2=0x7ffbd41085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.107+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/2493647062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd41081a0 0x7ffbd41085e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009900 tx=0x7ffbc402a410 comp rx=0 tx=0).stop 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- 172.21.15.60:0/2493647062 shutdown_connections 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/2493647062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd4108b20 0x7ffbd410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/2493647062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd41081a0 0x7ffbd41085e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/2493647062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 0x7ffbd4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- 172.21.15.60:0/2493647062 >> 172.21.15.60:0/2493647062 conn(0x7ffbd40fe200 msgr2=0x7ffbd4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- 172.21.15.60:0/2493647062 shutdown_connections 2024-07-24T21:45:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- 172.21.15.60:0/2493647062 wait complete. 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 Processor -- start 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- start start 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 0x7ffbd4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd41081a0 0x7ffbd40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4112740 con 0x7ffbd4108b20 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.108+0000 7ffbdc63d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41125c0 con 0x7ffbd4102790 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdc63d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41128c0 con 0x7ffbd41081a0 2024-07-24T21:45:30.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60502/0 (socket says 172.21.15.60:60502) 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 -- 172.21.15.60:0/407111501 learned_addr learned my addr 172.21.15.60:0/407111501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd41081a0 0x7ffbd40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 0x7ffbd4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 -- 172.21.15.60:0/407111501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd41081a0 msgr2=0x7ffbd40753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd41081a0 0x7ffbd40753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 -- 172.21.15.60:0/407111501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 msgr2=0x7ffbd4074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 0x7ffbd4074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 -- 172.21.15.60:0/407111501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41a70b0 con 0x7ffbd4108b20 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdabda640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eaa0 tx=0x7ffbcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbcb7fe640 1 -- 172.21.15.60:0/407111501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00cdb0 con 0x7ffbd4108b20 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbcb7fe640 1 -- 172.21.15.60:0/407111501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc005020 con 0x7ffbd4108b20 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41a73a0 con 0x7ffbd4108b20 2024-07-24T21:45:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbcb7fe640 1 -- 172.21.15.60:0/407111501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0116d0 con 0x7ffbd4108b20 2024-07-24T21:45:30.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.109+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a78b0 con 0x7ffbd4108b20 2024-07-24T21:45:30.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbcb7fe640 1 -- 172.21.15.60:0/407111501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ffbcc0051d0 con 0x7ffbd4108b20 2024-07-24T21:45:30.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 Processor -- start 2024-07-24T21:45:30.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 -- start start 2024-07-24T21:45:30.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f20718f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20718fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f207105720 con 0x55f20719b430 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2070f4c20 con 0x55f207190b60 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2070f8c60 con 0x55f20718f600 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.112+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60512/0 (socket says 172.21.15.60:60512) 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2617053381 learned_addr learned my addr 172.21.15.60:0/2617053381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f20718f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20718fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2617053381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 msgr2=0x55f20718fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20718fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2617053381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 msgr2=0x55f20718f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f20718f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2617053381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20707e1d0 con 0x55f20719b430 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402a930 tx=0x7ffbc4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2cc883e3a6e38 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbcb7fe640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 0x7ffbc00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 0x7ffbc00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbc8ff9640 1 -- 172.21.15.60:0/2617053381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035070 con 0x55f20719b430 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbc8ff9640 1 -- 172.21.15.60:0/2617053381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc4045070 con 0x55f20719b430 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbcb7fe640 1 -- 172.21.15.60:0/407111501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbcc0e3f20 con 0x7ffbd4108b20 2024-07-24T21:45:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.113+0000 7ffbc8ff9640 1 -- 172.21.15.60:0/2617053381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4003b20 con 0x55f20719b430 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2617053381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 msgr2=0x55f20719d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402a930 tx=0x7ffbc4005b50 comp rx=0 tx=0).stop 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 0x7ffbc00b60e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55f207192240 tx=0x7ffbd000b4d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2617053381 shutdown_connections 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f20719d820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20718fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2617053381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f20718f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2617053381 >> 172.21.15.60:0/2617053381 conn(0x55f20712d6a0 msgr2=0x55f20718cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2617053381 shutdown_connections 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.114+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2617053381 wait complete. 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 -- start start 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 Processor -- start 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20719abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f206f585f0 con 0x55f20719b430 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2070974a0 con 0x55f207190b60 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f20719e1f0 con 0x55f20718f600 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbde9ee740 10 client.0 _ll_register_callbacks cb 0x55f2070ed0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbdabda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbdabda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60524/0 (socket says 172.21.15.60:60524) 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbdabda640 1 -- 172.21.15.60:0/3952330931 learned_addr learned my addr 172.21.15.60:0/3952330931 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20719abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.115+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 -- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 msgr2=0x55f20719abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20719abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 -- 172.21.15.60:0/3952330931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 msgr2=0x55f2071980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 -- 172.21.15.60:0/3952330931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20708a530 con 0x55f20719b430 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbdabda640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0a7fc0 tx=0x7ffbcc0aa9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42621af06425e410 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01f040 con 0x55f20719b430 2024-07-24T21:45:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbcc0a9fb0 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc4045070 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc0b3e00 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbde9ee740 10 client.15129 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f206e55540 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc4045070 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.116+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffbcc00f040 con 0x55f20719b430 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 10 client.15129 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 10 client.15129 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 20 client.15129 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f2071abde0 0x55f2071ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 4 client.15129 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f207107670 con 0x55f2071abde0 2024-07-24T21:45:30.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f2071abde0 0x55f2071ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f2071abde0 0x55f2071ad250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d0540991393984b server_cookie=cc00edd5f1fca39b in_seq=0 out_seq=0 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.117+0000 7ffbbdffb640 10 client.15129 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbbdffb640 1 -- 172.21.15.60:0/3952330931 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f207107670 con 0x55f2071abde0 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbbdffb640 10 client.15129 handle_command_reply: tid=0 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 client.15129 shutdown 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 msgr2=0x55f207198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0a7fc0 tx=0x7ffbcc0aa9c0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 shutdown_connections 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f2071abde0 0x55f2071ad250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20719b430 0x55f207198600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbbdffb640 0 client.15129 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/3952330931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20718f600 0x55f20719abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 >> 172.21.15.60:0/3952330931 conn(0x55f20712d6a0 msgr2=0x55f207192a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 shutdown_connections 2024-07-24T21:45:30.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 1 -- 172.21.15.60:0/3952330931 wait complete. 2024-07-24T21:45:30.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.118+0000 7ffbde9ee740 20 client.15129 trim_cache size 0 max 16384 2024-07-24T21:45:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 Processor -- start 2024-07-24T21:45:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 -- start start 2024-07-24T21:45:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f20718d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f2071ab590 con 0x55f20718f600 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2071abc60 con 0x55f207190b60 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.137+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2071ac330 con 0x55f20712d6a0 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60528/0 (socket says 172.21.15.60:60528) 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2264156345 learned_addr learned my addr 172.21.15.60:0/2264156345 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbdabda640 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f20718d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2264156345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 msgr2=0x55f20718d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f20718d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2264156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 msgr2=0x55f2071995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2264156345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20719db90 con 0x55f20718f600 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4033000 tx=0x7ffbc4003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9cf923c4e51c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbbffff640 1 -- 172.21.15.60:0/2264156345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035040 con 0x55f20718f600 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbbffff640 1 -- 172.21.15.60:0/2264156345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc4045020 con 0x55f20718f600 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.138+0000 7ffbbffff640 1 -- 172.21.15.60:0/2264156345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40322a0 con 0x55f20718f600 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2264156345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 msgr2=0x55f207193280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4033000 tx=0x7ffbc4003700 comp rx=0 tx=0).stop 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2264156345 shutdown_connections 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f207193280 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2264156345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f20718d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2264156345 >> 172.21.15.60:0/2264156345 conn(0x55f207188280 msgr2=0x55f20718c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2264156345 shutdown_connections 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2264156345 wait complete. 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 -- start start 2024-07-24T21:45:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.139+0000 7ffbde9ee740 1 Processor -- start 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f2071b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f2071ad0d0 con 0x55f20718f600 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2071aca00 con 0x55f207190b60 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2071ad7a0 con 0x55f20712d6a0 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbde9ee740 10 client.0 _ll_register_callbacks cb 0x7ffbcc00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60542/0 (socket says 172.21.15.60:60542) 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2784938065 learned_addr learned my addr 172.21.15.60:0/2784938065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f2071b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbdabda640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 msgr2=0x55f2071b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f2071b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2784938065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 msgr2=0x55f2071b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 -- 172.21.15.60:0/2784938065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2071b5b00 con 0x55f20718f600 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbdabda640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbd9bd8640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005c40 tx=0x7ffbc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9bec8c1b1fba11 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.140+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035030 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbc40048c0 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc4045020 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbc4058070 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 10 client.15135 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f207194d80 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc4045020 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc40654a0 con 0x55f20718f600 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 10 client.15135 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 10 client.15135 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 20 client.15135 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f20719b430 0x55f20721a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 4 client.15135 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:45:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "status"}) -- 0x55f207103110 con 0x55f20719b430 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.141+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f20719b430 0x55f20721a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbda3d9640 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f20719b430 0x55f20721a140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4714504870f59df7 server_cookie=8e4b31d31c3dbd7e in_seq=0 out_seq=0 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbbdffb640 10 client.15135 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbbdffb640 1 -- 172.21.15.60:0/2784938065 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f207103110 con 0x55f20719b430 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbbdffb640 10 client.15135 handle_command_reply: tid=0 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 client.15135 shutdown 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 msgr2=0x55f2071b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005c40 tx=0x7ffbc4000bc0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 shutdown_connections 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f20719b430 0x55f20721a140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f207190b60 0x55f2071b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f20718f600 0x55f2071b4b70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 --2- 172.21.15.60:0/2784938065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f20712d6a0 0x55f2071b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 >> 172.21.15.60:0/2784938065 conn(0x55f207188280 msgr2=0x55f207199570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbbdffb640 0 client.15135 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 shutdown_connections 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 1 -- 172.21.15.60:0/2784938065 wait complete. 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.142+0000 7ffbde9ee740 20 client.15135 trim_cache size 0 max 16384 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24598, 2024-07-24T21:45:30.150 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 122.783986825, 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 165.77678948100001 2024-07-24T21:45:30.151 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 msgr2=0x7ffbc00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 0x7ffbc00b60e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55f207192240 tx=0x7ffbd000b4d0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 msgr2=0x7ffbd4075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eaa0 tx=0x7ffbcc00ef60 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 shutdown_connections 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbc00b3c20 0x7ffbc00b60e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbd4108b20 0x7ffbd4075910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbd41081a0 0x7ffbd40753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 --2- 172.21.15.60:0/407111501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd4102790 0x7ffbd4074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 >> 172.21.15.60:0/407111501 conn(0x7ffbd40fe200 msgr2=0x7ffbd41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 shutdown_connections 2024-07-24T21:45:30.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:30.143+0000 7ffbdc63d640 1 -- 172.21.15.60:0/407111501 wait complete. 2024-07-24T21:45:30.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.b status 2024-07-24T21:45:31.079 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:31.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:31 smithi060 ceph-mon[33000]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:31.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:31 smithi123 ceph-mon[37143]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:31.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:31 smithi086 ceph-mon[37128]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 -- 172.21.15.60:0/3727453680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6498107a20 msgr2=0x7f649810e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 --2- 172.21.15.60:0/3727453680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6498107a20 0x7f649810e220 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6490009ca0 tx=0x7f649002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 -- 172.21.15.60:0/3727453680 shutdown_connections 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 --2- 172.21.15.60:0/3727453680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6498107a20 0x7f649810e220 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 --2- 172.21.15.60:0/3727453680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64981070a0 0x7f64981074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.685+0000 7f649d308640 1 --2- 172.21.15.60:0/3727453680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498101690 0x7f6498101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.686+0000 7f649d308640 1 -- 172.21.15.60:0/3727453680 >> 172.21.15.60:0/3727453680 conn(0x7f64980fd100 msgr2=0x7f64980ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.686+0000 7f649d308640 1 -- 172.21.15.60:0/3727453680 shutdown_connections 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.686+0000 7f649d308640 1 -- 172.21.15.60:0/3727453680 wait complete. 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.686+0000 7f649d308640 1 Processor -- start 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.686+0000 7f649d308640 1 -- start start 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 0x7f6498199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 0x7f649819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6498111650 con 0x7f64981070a0 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64981114d0 con 0x7f6498101690 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f649d308640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64981117d0 con 0x7f6498107a20 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60546/0 (socket says 172.21.15.60:60546) 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 -- 172.21.15.60:0/2723736385 learned_addr learned my addr 172.21.15.60:0/2723736385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64977fe640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 0x7f649819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f6496ffd640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 0x7f6498199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 -- 172.21.15.60:0/2723736385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 msgr2=0x7f649819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 0x7f649819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 -- 172.21.15.60:0/2723736385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 msgr2=0x7f6498199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 0x7f6498199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 -- 172.21.15.60:0/2723736385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64981a4e10 con 0x7f64981070a0 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f6496ffd640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 0x7f6498199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64977fe640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 0x7f649819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.687+0000 7f64967fc640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c920 tx=0x7f648c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.688+0000 7f647ffff640 1 -- 172.21.15.60:0/2723736385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c007c70 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.688+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64981a5100 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.688+0000 7f647ffff640 1 -- 172.21.15.60:0/2723736385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648c007e10 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.688+0000 7f647ffff640 1 -- 172.21.15.60:0/2723736385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c018820 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.688+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981a5640 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.690+0000 7f649f6b9740 1 Processor -- start 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.690+0000 7f647ffff640 1 -- 172.21.15.60:0/2723736385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f648c00f450 con 0x7f64981070a0 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.690+0000 7f649f6b9740 1 -- start start 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e3120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e324c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e288740 con 0x55cf9e312620 2024-07-24T21:45:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e277c40 con 0x55cf9e313b50 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f649f6b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e27bc80 con 0x55cf9e322860 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60554/0 (socket says 172.21.15.60:60554) 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 -- 172.21.15.60:0/2270910291 learned_addr learned my addr 172.21.15.60:0/2270910291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f64967fc640 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e324c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f64977fe640 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e3120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 -- 172.21.15.60:0/2270910291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 msgr2=0x55cf9e324c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e324c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 -- 172.21.15.60:0/2270910291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 msgr2=0x55cf9e3120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e3120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 -- 172.21.15.60:0/2270910291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9e2011f0 con 0x55cf9e312620 2024-07-24T21:45:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.691+0000 7f6496ffd640 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f648800bd40 tx=0x7f648800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd02af83a2aa913 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f647ffff640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 0x7f64800b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f647d7fa640 1 -- 172.21.15.60:0/2270910291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488015070 con 0x55cf9e312620 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f647d7fa640 1 -- 172.21.15.60:0/2270910291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648800be30 con 0x55cf9e312620 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f647d7fa640 1 -- 172.21.15.60:0/2270910291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64880106e0 con 0x55cf9e312620 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f64977fe640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 0x7f64800b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f647ffff640 1 -- 172.21.15.60:0/2723736385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f648c0e3ed0 con 0x7f64981070a0 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 -- 172.21.15.60:0/2270910291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 msgr2=0x55cf9e312a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f64977fe640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 0x7f64800b60e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649002f440 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f648800bd40 tx=0x7f648800f730 comp rx=0 tx=0).stop 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 -- 172.21.15.60:0/2270910291 shutdown_connections 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e324c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 --2- 172.21.15.60:0/2270910291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e3120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.692+0000 7f649f6b9740 1 -- 172.21.15.60:0/2270910291 >> 172.21.15.60:0/2270910291 conn(0x55cf9e2b06c0 msgr2=0x55cf9e30ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- 172.21.15.60:0/2270910291 shutdown_connections 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- 172.21.15.60:0/2270910291 wait complete. 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- start start 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 Processor -- start 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 0x55cf9e321eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e313b50 0x55cf9e31f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e0db5f0 con 0x55cf9e322860 2024-07-24T21:45:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e21a4c0 con 0x55cf9e312620 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e325650 con 0x55cf9e313b50 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f649f6b9740 10 client.0 _ll_register_callbacks cb 0x55cf9e2700c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f64977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f64977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60556/0 (socket says 172.21.15.60:60556) 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f64977fe640 1 -- 172.21.15.60:0/1231452460 learned_addr learned my addr 172.21.15.60:0/1231452460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.693+0000 7f64967fc640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e313b50 0x55cf9e31f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 0x55cf9e321eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 -- 172.21.15.60:0/1231452460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e313b50 msgr2=0x55cf9e31f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e313b50 0x55cf9e31f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 -- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 msgr2=0x55cf9e321eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 0x55cf9e321eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 -- 172.21.15.60:0/1231452460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9e20d550 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 0x55cf9e321eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f64977fe640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f64900317a0 tx=0x7f64900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b880b421316400a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f647aff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649004f070 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f648800be30 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6490004240 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6490004480 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f649f6b9740 10 client.15147 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf9dfd8540 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.694+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f648800be30 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6490044040 con 0x55cf9e322860 2024-07-24T21:45:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 10 client.15147 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 20 client.15147 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 4 client.15147 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9e28a690 con 0x55cf9e333250 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3346c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c14bf79dbfc7e60 server_cookie=d3f534f63efedca9 in_seq=0 out_seq=0 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.695+0000 7f647aff5640 10 client.15147 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f647aff5640 1 -- 172.21.15.60:0/1231452460 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf9e28a690 con 0x55cf9e333250 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f647aff5640 10 client.15147 handle_command_reply: tid=0 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 client.15147 shutdown 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 msgr2=0x55cf9e31f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f64900317a0 tx=0x7f64900095c0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 shutdown_connections 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3346c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e322860 0x55cf9e31f8e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e313b50 0x55cf9e31f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f647aff5640 0 client.15147 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1231452460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e312620 0x55cf9e321eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.696+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 >> 172.21.15.60:0/1231452460 conn(0x55cf9e2b06c0 msgr2=0x55cf9e315ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.697+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 shutdown_connections 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.697+0000 7f649f6b9740 1 -- 172.21.15.60:0/1231452460 wait complete. 2024-07-24T21:45:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.697+0000 7f649f6b9740 20 client.15147 trim_cache size 0 max 16384 2024-07-24T21:45:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 Processor -- start 2024-07-24T21:45:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 -- start start 2024-07-24T21:45:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e31dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e31e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e332a00 con 0x55cf9e312620 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e3330d0 con 0x55cf9e313b50 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f649f6b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e3337a0 con 0x55cf9e322860 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60562/0 (socket says 172.21.15.60:60562) 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.716+0000 7f6496ffd640 1 -- 172.21.15.60:0/488964338 learned_addr learned my addr 172.21.15.60:0/488964338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f64977fe640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e31e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f64967fc640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e31dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 -- 172.21.15.60:0/488964338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 msgr2=0x55cf9e31e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e31e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 -- 172.21.15.60:0/488964338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 msgr2=0x55cf9e31dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e31dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 -- 172.21.15.60:0/488964338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9e31d8a0 con 0x55cf9e312620 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f64967fc640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e31dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f6496ffd640 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f648800fc30 tx=0x7f6488000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddaf1d60b0dcd81 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f647cff9640 1 -- 172.21.15.60:0/488964338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488015040 con 0x55cf9e312620 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f647cff9640 1 -- 172.21.15.60:0/488964338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488011440 con 0x55cf9e312620 2024-07-24T21:45:32.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.717+0000 7f647cff9640 1 -- 172.21.15.60:0/488964338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64880048e0 con 0x55cf9e312620 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- 172.21.15.60:0/488964338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 msgr2=0x55cf9e312a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f648800fc30 tx=0x7f6488000bc0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- 172.21.15.60:0/488964338 shutdown_connections 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e31e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e31dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 --2- 172.21.15.60:0/488964338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e312a00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- 172.21.15.60:0/488964338 >> 172.21.15.60:0/488964338 conn(0x55cf9e2b06c0 msgr2=0x55cf9e31f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- 172.21.15.60:0/488964338 shutdown_connections 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- 172.21.15.60:0/488964338 wait complete. 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 -- start start 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.718+0000 7f649f6b9740 1 Processor -- start 2024-07-24T21:45:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e33c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e33cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e334540 con 0x55cf9e312620 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e333e70 con 0x55cf9e313b50 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e334c10 con 0x55cf9e322860 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 10 client.0 _ll_register_callbacks cb 0x7f6490009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60578/0 (socket says 172.21.15.60:60578) 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 -- 172.21.15.60:0/1241023540 learned_addr learned my addr 172.21.15.60:0/1241023540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f64967fc640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e33c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 -- 172.21.15.60:0/1241023540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 msgr2=0x55cf9e33cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e33cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 -- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 msgr2=0x55cf9e33c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e33c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 -- 172.21.15.60:0/1241023540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9e33d5d0 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f6496ffd640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f64880058c0 tx=0x7f648800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1365c174ada0c62 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f647aff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488037020 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6488005040 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.719+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6488011440 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6488042050 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 10 client.15153 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cf9e39da20 con 0x55cf9e312620 2024-07-24T21:45:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488011440 con 0x55cf9e312620 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6488004cb0 con 0x55cf9e312620 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 10 client.15153 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 20 client.15153 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 4 client.15153 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "status"}) -- 0x55cf9e286130 con 0x55cf9e333250 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.720+0000 7f64967fc640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f64967fc640 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3a0c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbd6c78ab4349d0c server_cookie=4e74c818f027f94d in_seq=0 out_seq=0 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f647aff5640 10 client.15153 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f647aff5640 1 -- 172.21.15.60:0/1241023540 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cf9e286130 con 0x55cf9e333250 2024-07-24T21:45:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f647aff5640 10 client.15153 handle_command_reply: tid=0 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 client.15153 shutdown 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 msgr2=0x55cf9e30fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f64880058c0 tx=0x7f648800b670 comp rx=0 tx=0).stop 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 shutdown_connections 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55cf9e333250 0x55cf9e3a0c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf9e322860 0x55cf9e33cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f647aff5640 0 client.15153 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf9e313b50 0x55cf9e33c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 --2- 172.21.15.60:0/1241023540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cf9e312620 0x55cf9e30fe80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 >> 172.21.15.60:0/1241023540 conn(0x55cf9e2b06c0 msgr2=0x55cf9e313f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 shutdown_connections 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 1 -- 172.21.15.60:0/1241023540 wait complete. 2024-07-24T21:45:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.721+0000 7f649f6b9740 20 client.15153 trim_cache size 0 max 16384 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24605, 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 126.368094493, 2024-07-24T21:45:32.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:45:32.730 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:45:32.730 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:45:32.730 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 153.50849383900001 2024-07-24T21:45:32.730 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:45:32.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 msgr2=0x7f64800b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 0x7f64800b60e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649002f440 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 msgr2=0x7f6498199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c920 tx=0x7f648c00cde0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 shutdown_connections 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64800b3c20 0x7f64800b60e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6498107a20 0x7f649819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64981070a0 0x7f6498199f70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 --2- 172.21.15.60:0/2723736385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498101690 0x7f6498199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 >> 172.21.15.60:0/2723736385 conn(0x7f64980fd100 msgr2=0x7f6498103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 shutdown_connections 2024-07-24T21:45:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:32.722+0000 7f649d308640 1 -- 172.21.15.60:0/2723736385 wait complete. 2024-07-24T21:45:33.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:33 smithi123 ceph-mon[37143]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:33.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:33 smithi060 ceph-mon[33000]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:33.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.i status 2024-07-24T21:45:33.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:33 smithi086 ceph-mon[37128]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 -- 172.21.15.60:0/1086251052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 msgr2=0x7fb83c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 --2- 172.21.15.60:0/1086251052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c102b70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009900 tx=0x7fb82c02a3f0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 -- 172.21.15.60:0/1086251052 shutdown_connections 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 --2- 172.21.15.60:0/1086251052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c108b20 0x7fb83c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 --2- 172.21.15.60:0/1086251052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c1081a0 0x7fb83c1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 --2- 172.21.15.60:0/1086251052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c102b70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 -- 172.21.15.60:0/1086251052 >> 172.21.15.60:0/1086251052 conn(0x7fb83c0fe200 msgr2=0x7fb83c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 -- 172.21.15.60:0/1086251052 shutdown_connections 2024-07-24T21:45:35.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.378+0000 7fb841398640 1 -- 172.21.15.60:0/1086251052 wait complete. 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 Processor -- start 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 -- start start 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 0x7fb83c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 0x7fb83c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c112720 con 0x7fb83c102790 2024-07-24T21:45:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1125a0 con 0x7fb83c108b20 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb841398640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1128a0 con 0x7fb83c1081a0 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb83affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb83affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60606/0 (socket says 172.21.15.60:60606) 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.379+0000 7fb83affd640 1 -- 172.21.15.60:0/3955485646 learned_addr learned my addr 172.21.15.60:0/3955485646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 0x7fb83c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 0x7fb83c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 -- 172.21.15.60:0/3955485646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 msgr2=0x7fb83c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 0x7fb83c199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 -- 172.21.15.60:0/3955485646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 msgr2=0x7fb83c19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 0x7fb83c19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 -- 172.21.15.60:0/3955485646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c19ecc0 con 0x7fb83c102790 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 0x7fb83c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 0x7fb83c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb83affd640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006010 tx=0x7fb82c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb823fff640 1 -- 172.21.15.60:0/3955485646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c03f030 con 0x7fb83c102790 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb823fff640 1 -- 172.21.15.60:0/3955485646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c003800 con 0x7fb83c102790 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c19ef50 con 0x7fb83c102790 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb823fff640 1 -- 172.21.15.60:0/3955485646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035070 con 0x7fb83c102790 2024-07-24T21:45:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.380+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a5670 con 0x7fb83c102790 2024-07-24T21:45:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb823fff640 1 -- 172.21.15.60:0/3955485646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fb82c003cb0 con 0x7fb83c102790 2024-07-24T21:45:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 Processor -- start 2024-07-24T21:45:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 -- start start 2024-07-24T21:45:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e99430 0x55c072e9b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c072e03720 con 0x55c072e8eb60 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c072df2c20 con 0x55c072e8d600 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb843749740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c072df6c60 con 0x55c072e99430 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60620/0 (socket says 172.21.15.60:60620) 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb83b7fe640 1 -- 172.21.15.60:0/2198623562 learned_addr learned my addr 172.21.15.60:0/2198623562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.383+0000 7fb83affd640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e99430 0x55c072e9b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 -- 172.21.15.60:0/2198623562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e99430 msgr2=0x55c072e9b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e99430 0x55c072e9b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 -- 172.21.15.60:0/2198623562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 msgr2=0x55c072e8da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e8da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 -- 172.21.15.60:0/2198623562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c072d7c1d0 con 0x55c072e8eb60 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83affd640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb823fff640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 0x7fb8240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb83400cb90 tx=0x7fb83400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee73e00939272454 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb83affd640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 0x7fb8240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb8217fa640 1 -- 172.21.15.60:0/2198623562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834004220 con 0x55c072e8eb60 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb823fff640 1 -- 172.21.15.60:0/3955485646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb82c1047e0 con 0x7fb83c102790 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb8217fa640 1 -- 172.21.15.60:0/2198623562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340043c0 con 0x55c072e8eb60 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.384+0000 7fb8217fa640 1 -- 172.21.15.60:0/2198623562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834004ce0 con 0x55c072e8eb60 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- 172.21.15.60:0/2198623562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 msgr2=0x55c072e8d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb83affd640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 0x7fb8240b60e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb82c102530 tx=0x7fb82c02a3f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb83400cb90 tx=0x7fb83400e720 comp rx=0 tx=0).stop 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- 172.21.15.60:0/2198623562 shutdown_connections 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e99430 0x55c072e9b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 --2- 172.21.15.60:0/2198623562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072e8d0c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- 172.21.15.60:0/2198623562 >> 172.21.15.60:0/2198623562 conn(0x55c072e2b6a0 msgr2=0x55c072e8aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- 172.21.15.60:0/2198623562 shutdown_connections 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- 172.21.15.60:0/2198623562 wait complete. 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.385+0000 7fb843749740 1 -- start start 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 Processor -- start 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e98be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8eb60 0x55c072e960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c072c565f0 con 0x55c072e99430 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c072d954a0 con 0x55c072e8d600 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c072e9c250 con 0x55c072e8eb60 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb843749740 10 client.0 _ll_register_callbacks cb 0x55c072deb0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60636/0 (socket says 172.21.15.60:60636) 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1468418504 learned_addr learned my addr 172.21.15.60:0/1468418504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8eb60 0x55c072e960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e98be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8eb60 msgr2=0x55c072e960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8eb60 0x55c072e960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1468418504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 msgr2=0x55c072e98be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e98be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.386+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1468418504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c072d88530 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb834004610 tx=0x7fb834005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e08251ed98b76a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401a460 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb83401a600 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8340043c0 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb83401a800 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb843749740 10 client.15165 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c072b53540 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340043c0 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb834029020 con 0x55c072e99430 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.387+0000 7fb843749740 10 client.15165 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb843749740 10 client.15165 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:45:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb843749740 20 client.15165 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072ea9af0 0x55c072eaaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb843749740 4 client.15165 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c072e05670 con 0x55c072ea9af0 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072ea9af0 0x55c072eaaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072ea9af0 0x55c072eaaf60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdd779aaa3b63be server_cookie=4a0c1ae81945d19d in_seq=0 out_seq=0 2024-07-24T21:45:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.388+0000 7fb81eff5640 10 client.15165 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb81eff5640 1 -- 172.21.15.60:0/1468418504 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c072e05670 con 0x55c072ea9af0 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb81eff5640 10 client.15165 handle_command_reply: tid=0 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 client.15165 shutdown 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 msgr2=0x55c072e96610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb834004610 tx=0x7fb834005ce0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 shutdown_connections 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072ea9af0 0x55c072eaaf60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e96610 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8eb60 0x55c072e960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb81eff5640 0 client.15165 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 --2- 172.21.15.60:0/1468418504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8d600 0x55c072e98be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 >> 172.21.15.60:0/1468418504 conn(0x55c072e2b6a0 msgr2=0x55c072e90a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 shutdown_connections 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.389+0000 7fb843749740 1 -- 172.21.15.60:0/1468418504 wait complete. 2024-07-24T21:45:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.390+0000 7fb843749740 20 client.15165 trim_cache size 0 max 16384 2024-07-24T21:45:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.408+0000 7fb843749740 1 Processor -- start 2024-07-24T21:45:35.410 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:35 smithi060 ceph-mon[33000]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.408+0000 7fb843749740 1 -- start start 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072e8cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 0x55c072e91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c072ea92a0 con 0x55c072e99430 2024-07-24T21:45:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c072ea9970 con 0x55c072e8eb60 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb843749740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c072eaa040 con 0x55c072e8d600 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60640/0 (socket says 172.21.15.60:60640) 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1117572286 learned_addr learned my addr 172.21.15.60:0/1117572286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 0x55c072e91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072e8cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1117572286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 msgr2=0x55c072e8cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072e8cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1117572286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 msgr2=0x55c072e91280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 0x55c072e91280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 -- 172.21.15.60:0/1117572286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c072eaa4f0 con 0x55c072e99430 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 0x55c072e91280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb834001eb0 tx=0x7fb8340049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ddfafe1f9ade7a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.409+0000 7fb820ff9640 1 -- 172.21.15.60:0/1117572286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83403b070 con 0x55c072e99430 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb820ff9640 1 -- 172.21.15.60:0/1117572286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834020030 con 0x55c072e99430 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb820ff9640 1 -- 172.21.15.60:0/1117572286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834010b80 con 0x55c072e99430 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 -- 172.21.15.60:0/1117572286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 msgr2=0x55c072e917c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb834001eb0 tx=0x7fb8340049e0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 -- 172.21.15.60:0/1117572286 shutdown_connections 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e99430 0x55c072e917c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e8eb60 0x55c072e91280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 --2- 172.21.15.60:0/1117572286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072e8cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 -- 172.21.15.60:0/1117572286 >> 172.21.15.60:0/1117572286 conn(0x55c072e2b6a0 msgr2=0x55c072e94950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 -- 172.21.15.60:0/1117572286 shutdown_connections 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.410+0000 7fb843749740 1 -- 172.21.15.60:0/1117572286 wait complete. 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 -- start start 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 Processor -- start 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072eac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e99430 0x55c072eb31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c072eaade0 con 0x55c072e8eb60 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c072eaa710 con 0x55c072e99430 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c072eab4b0 con 0x55c072e8d600 2024-07-24T21:45:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb843749740 10 client.0 _ll_register_callbacks cb 0x7fb834014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60656/0 (socket says 172.21.15.60:60656) 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 -- 172.21.15.60:0/3041697209 learned_addr learned my addr 172.21.15.60:0/3041697209 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072eac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e99430 0x55c072eb31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 -- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 msgr2=0x55c072eac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072eac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 -- 172.21.15.60:0/3041697209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e99430 msgr2=0x55c072eb31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.411+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e99430 0x55c072eb31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb83a7fc640 1 -- 172.21.15.60:0/3041697209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c072eb38b0 con 0x55c072e8eb60 2024-07-24T21:45:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072eac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb83a7fc640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb830000f80 tx=0x7fb830012900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1d3e116b38af2a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb81eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830005450 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb8300055f0 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb834020030 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8300057f0 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb843749740 10 client.15171 fetch_fsmap learned FSMap version 25 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c072f14d90 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.412+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834020030 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb83001a290 con 0x55c072e8eb60 2024-07-24T21:45:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 10 client.15171 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 10 client.15171 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 20 client.15171 populate_metadata read hostname 'smithi060' 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072f199d0 0x55c072f186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 4 client.15171 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "status"}) -- 0x55c072e95930 con 0x55c072f199d0 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072f199d0 0x55c072f186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb83b7fe640 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072f199d0 0x55c072f186e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07ead7df768b4b7 server_cookie=9b915dd15b09be2d in_seq=0 out_seq=0 2024-07-24T21:45:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb81eff5640 10 client.15171 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb81eff5640 1 -- 172.21.15.60:0/3041697209 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c072e95930 con 0x55c072f199d0 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.413+0000 7fb81eff5640 10 client.15171 handle_command_reply: tid=0 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 client.15171 shutdown 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 msgr2=0x55c072eac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb830000f80 tx=0x7fb830012900 comp rx=0 tx=0).stop 2024-07-24T21:45:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 shutdown_connections 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c072f199d0 0x55c072f186e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c072e99430 0x55c072eb31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c072e8eb60 0x55c072eac880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 --2- 172.21.15.60:0/3041697209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c072e8d600 0x55c072eac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 >> 172.21.15.60:0/3041697209 conn(0x55c072e2b6a0 msgr2=0x55c072e97480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb81eff5640 0 client.15171 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 shutdown_connections 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 1 -- 172.21.15.60:0/3041697209 wait complete. 2024-07-24T21:45:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb843749740 20 client.15171 trim_cache size 0 max 16384 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24610, 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:45:35.426 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 141.12368602699999, 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 166.18616594100001 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 msgr2=0x7fb8240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 0x7fb8240b60e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb82c102530 tx=0x7fb82c02a3f0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 msgr2=0x7fb83c198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.414+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006010 tx=0x7fb82c0042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 shutdown_connections 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8240b3c20 0x7fb8240b60e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb83c108b20 0x7fb83c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb83c1081a0 0x7fb83c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 --2- 172.21.15.60:0/3955485646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83c102790 0x7fb83c198c10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 >> 172.21.15.60:0/3955485646 conn(0x7fb83c0fe200 msgr2=0x7fb83c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 shutdown_connections 2024-07-24T21:45:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:35.415+0000 7fb841398640 1 -- 172.21.15.60:0/3955485646 wait complete. 2024-07-24T21:45:35.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:35 smithi123 ceph-mon[37143]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:35.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:35 smithi086 ceph-mon[37128]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:45:36.087 INFO:tasks.cephadm:Setup complete, yielding 2024-07-24T21:45:36.087 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-24T21:45:36.097 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:45:36.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph orch status' 2024-07-24T21:45:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:37.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:37 smithi123 ceph-mon[37143]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:37.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:37 smithi086 ceph-mon[37128]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:37.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:37 smithi060 ceph-mon[33000]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 -- 172.21.15.60:0/927081012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8108180 msgr2=0x7f16d81085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 --2- 172.21.15.60:0/927081012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8108180 0x7f16d81085c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-07-24T21:45:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 -- 172.21.15.60:0/927081012 shutdown_connections 2024-07-24T21:45:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 --2- 172.21.15.60:0/927081012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 0x7f16d810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 --2- 172.21.15.60:0/927081012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8108180 0x7f16d81085c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 --2- 172.21.15.60:0/927081012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8102770 0x7f16d8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 -- 172.21.15.60:0/927081012 >> 172.21.15.60:0/927081012 conn(0x7f16d80fe1c0 msgr2=0x7f16d81005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 -- 172.21.15.60:0/927081012 shutdown_connections 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.068+0000 7f16dcf40640 1 -- 172.21.15.60:0/927081012 wait complete. 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 Processor -- start 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 -- start start 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 0x7f16d81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 0x7f16d819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8112700 con 0x7f16d8102770 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8112580 con 0x7f16d8108180 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16dcf40640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8112880 con 0x7f16d8108b00 2024-07-24T21:45:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36068/0 (socket says 172.21.15.60:36068) 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 0x7f16d81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.069+0000 7f16d659c640 1 -- 172.21.15.60:0/272751279 learned_addr learned my addr 172.21.15.60:0/272751279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d6d9d640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 0x7f16d819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 -- 172.21.15.60:0/272751279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 msgr2=0x7f16d819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 0x7f16d819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 -- 172.21.15.60:0/272751279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 msgr2=0x7f16d81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 0x7f16d81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 -- 172.21.15.60:0/272751279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d819ec40 con 0x7f16d8102770 2024-07-24T21:45:38.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16cffff640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 0x7f16d81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16d659c640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009de0 tx=0x7f16c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c801f070 con 0x7f16d8102770 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c8011750 con 0x7f16d8102770 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d819ef30 con 0x7f16d8102770 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c801a740 con 0x7f16d8102770 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.070+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d8074f40 con 0x7f16d8102770 2024-07-24T21:45:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.071+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d8103a20 con 0x7f16d8102770 2024-07-24T21:45:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.073+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f16c8022070 con 0x7f16d8102770 2024-07-24T21:45:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.074+0000 7f16cf7fe640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 0x7f16c40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.074+0000 7f16cffff640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 0x7f16c40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.074+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f16c80e7c40 con 0x7f16d8102770 2024-07-24T21:45:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.074+0000 7f16cffff640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 0x7f16c40b6160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16c0031440 tx=0x7f16c0005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.077+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f16c80aeb20 con 0x7f16d8102770 2024-07-24T21:45:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.292+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f16d810a910 con 0x7f16c40b3ca0 2024-07-24T21:45:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.293+0000 7f16cf7fe640 1 -- 172.21.15.60:0/272751279 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f16d810a910 con 0x7f16c40b3ca0 2024-07-24T21:45:38.294 INFO:teuthology.orchestra.run.smithi060.stdout:Backend: cephadm 2024-07-24T21:45:38.294 INFO:teuthology.orchestra.run.smithi060.stdout:Available: Yes 2024-07-24T21:45:38.294 INFO:teuthology.orchestra.run.smithi060.stdout:Paused: No 2024-07-24T21:45:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 msgr2=0x7f16c40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 0x7f16c40b6160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16c0031440 tx=0x7f16c0005b50 comp rx=0 tx=0).stop 2024-07-24T21:45:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 msgr2=0x7f16d8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009de0 tx=0x7f16c800c6a0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 shutdown_connections 2024-07-24T21:45:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16c40b3ca0 0x7f16c40b6160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16d8108b00 0x7f16d819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16d8108180 0x7f16d81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 --2- 172.21.15.60:0/272751279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d8102770 0x7f16d8198b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 >> 172.21.15.60:0/272751279 conn(0x7f16d80fe1c0 msgr2=0x7f16d81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 shutdown_connections 2024-07-24T21:45:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:38.295+0000 7f16dcf40640 1 -- 172.21.15.60:0/272751279 wait complete. 2024-07-24T21:45:38.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-24T21:45:39.207 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:39.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:39 smithi060 ceph-mon[33000]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:39.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:39 smithi060 ceph-mon[33000]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:39 smithi123 ceph-mon[37143]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:39.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:39 smithi123 ceph-mon[37143]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:39 smithi086 ceph-mon[37128]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:39 smithi086 ceph-mon[37128]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 -- 172.21.15.60:0/1104649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 msgr2=0x7fc1c8101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/1104649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c8101a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009ca0 tx=0x7fc1c402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 -- 172.21.15.60:0/1104649983 shutdown_connections 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/1104649983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c8107a20 0x7fc1c810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/1104649983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81070a0 0x7fc1c81074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/1104649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c8101a70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 -- 172.21.15.60:0/1104649983 >> 172.21.15.60:0/1104649983 conn(0x7fc1c80fd100 msgr2=0x7fc1c80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 -- 172.21.15.60:0/1104649983 shutdown_connections 2024-07-24T21:45:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.867+0000 7fc1cfc42640 1 -- 172.21.15.60:0/1104649983 wait complete. 2024-07-24T21:45:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 Processor -- start 2024-07-24T21:45:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 -- start start 2024-07-24T21:45:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81070a0 0x7fc1c80fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8107a20 0x7fc1c80fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8111720 con 0x7fc1c8101690 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81115a0 con 0x7fc1c8107a20 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cfc42640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81118a0 con 0x7fc1c81070a0 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cd9de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cd9de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36082/0 (socket says 172.21.15.60:36082) 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.868+0000 7fc1cd9de640 1 -- 172.21.15.60:0/2208076463 learned_addr learned my addr 172.21.15.60:0/2208076463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1ce1df640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8107a20 0x7fc1c80fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd1dd640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81070a0 0x7fc1c80fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 -- 172.21.15.60:0/2208076463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81070a0 msgr2=0x7fc1c80fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81070a0 0x7fc1c80fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 -- 172.21.15.60:0/2208076463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8107a20 msgr2=0x7fc1c80fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8107a20 0x7fc1c80fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 -- 172.21.15.60:0/2208076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c81a71b0 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cd9de640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc1c402ade0 tx=0x7fc1c4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003d60 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c81a7440 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c40044f0 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c402fd10 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.869+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81a7a50 con 0x7fc1c8101690 2024-07-24T21:45:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.870+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8101af0 con 0x7fc1c8101690 2024-07-24T21:45:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.872+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc1c40046a0 con 0x7fc1c8101690 2024-07-24T21:45:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.873+0000 7fc1baffd640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 0x7fc1b40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.873+0000 7fc1cd1dd640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 0x7fc1b40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.873+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc1c4105520 con 0x7fc1c8101690 2024-07-24T21:45:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.873+0000 7fc1cd1dd640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 0x7fc1b40b6160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80ff0f0 tx=0x7fc1bc009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:40.876+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc1c40cc9f0 con 0x7fc1c8101690 2024-07-24T21:45:41.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.078+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc1c8062f80 con 0x7fc1b40b3ca0 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.091+0000 7fc1baffd640 1 -- 172.21.15.60:0/2208076463 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fc1c8062f80 con 0x7fc1b40b3ca0 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stdout:mds.a smithi060 running (2m) 25s ago 2m 16.0M - 19.0.0-5197-g86385b1d 2e906353c5e5 270371cb0ea4 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stdout:mds.b smithi086 running (2m) 28s ago 2m 19.5M - 19.0.0-5197-g86385b1d 2e906353c5e5 a498ea4c2ad8 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stdout:mds.c smithi123 running (2m) 28s ago 2m 19.6M - 19.0.0-5197-g86385b1d 2e906353c5e5 7c6e24d83f58 2024-07-24T21:45:41.092 INFO:teuthology.orchestra.run.smithi060.stdout:mds.d smithi060 running (3m) 25s ago 3m 16.7M - 19.0.0-5197-g86385b1d 2e906353c5e5 63a2dbb60cf7 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.e smithi086 running (3m) 28s ago 3m 21.3M - 19.0.0-5197-g86385b1d 2e906353c5e5 98e573c1dcd3 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.f smithi123 running (2m) 28s ago 2m 20.0M - 19.0.0-5197-g86385b1d 2e906353c5e5 4541a7c5f08f 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.g smithi060 running (2m) 25s ago 2m 19.3M - 19.0.0-5197-g86385b1d 2e906353c5e5 c4ec78250afc 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.h smithi086 running (2m) 28s ago 2m 19.8M - 19.0.0-5197-g86385b1d 2e906353c5e5 605db696520f 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.i smithi123 running (2m) 28s ago 2m 28.2M - 19.0.0-5197-g86385b1d 2e906353c5e5 79d23c86058f 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.j smithi060 running (3m) 25s ago 3m 14.1M - 19.0.0-5197-g86385b1d 2e906353c5e5 7cced47b312f 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.k smithi086 running (3m) 28s ago 3m 17.3M - 19.0.0-5197-g86385b1d 2e906353c5e5 72d2c88eba95 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mds.l smithi123 running (3m) 28s ago 3m 16.7M - 19.0.0-5197-g86385b1d 2e906353c5e5 4648084d62fc 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.x smithi060 *:9283,8765 running (10m) 25s ago 10m 516M - 19.0.0-5197-g86385b1d 2e906353c5e5 8c6b9dd4242e 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.y smithi086 *:8443,8765 running (8m) 28s ago 8m 457M - 19.0.0-5197-g86385b1d 2e906353c5e5 54c370e8ba49 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.z smithi123 *:8443,8765 running (8m) 28s ago 8m 454M - 19.0.0-5197-g86385b1d 2e906353c5e5 9c73f2b52023 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a smithi060 running (10m) 25s ago 11m 54.2M 2048M 19.0.0-5197-g86385b1d 2e906353c5e5 fece5d008e50 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b smithi086 running (9m) 28s ago 9m 44.7M 2048M 19.0.0-5197-g86385b1d 2e906353c5e5 4327c010b8af 2024-07-24T21:45:41.093 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c smithi123 running (9m) 28s ago 9m 43.3M 2048M 19.0.0-5197-g86385b1d 2e906353c5e5 cd1607c2f832 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 smithi060 running (7m) 25s ago 7m 56.3M 4273M 19.0.0-5197-g86385b1d 2e906353c5e5 2e4143d28ecc 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 smithi086 running (7m) 28s ago 7m 57.7M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 34fd2c55bf93 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 smithi123 running (7m) 28s ago 7m 52.0M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 3d7ca62017c7 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 smithi060 running (6m) 25s ago 6m 54.8M 4273M 19.0.0-5197-g86385b1d 2e906353c5e5 dee4812d1905 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 smithi086 running (6m) 28s ago 6m 46.3M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 8df6f89a86a8 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 smithi123 running (5m) 28s ago 5m 53.8M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 60d6aae85ffb 2024-07-24T21:45:41.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 smithi060 running (5m) 25s ago 5m 46.5M 4273M 19.0.0-5197-g86385b1d 2e906353c5e5 7e0b132b8094 2024-07-24T21:45:41.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 smithi086 running (5m) 28s ago 5m 56.6M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 46158284376b 2024-07-24T21:45:41.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 smithi123 running (4m) 28s ago 4m 55.9M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 f85605850ded 2024-07-24T21:45:41.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 smithi060 running (4m) 25s ago 4m 52.3M 4273M 19.0.0-5197-g86385b1d 2e906353c5e5 6d335ff6ad2b 2024-07-24T21:45:41.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 smithi086 running (3m) 28s ago 3m 47.3M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 f8bcd1364602 2024-07-24T21:45:41.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 smithi123 running (3m) 28s ago 3m 45.8M 4272M 19.0.0-5197-g86385b1d 2e906353c5e5 b6f021a26669 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.093+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 msgr2=0x7fc1b40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.093+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 0x7fc1b40b6160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80ff0f0 tx=0x7fc1bc009110 comp rx=0 tx=0).stop 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.093+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 msgr2=0x7fc1c80fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.093+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc1c402ade0 tx=0x7fc1c4003970 comp rx=0 tx=0).stop 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 shutdown_connections 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b40b3ca0 0x7fc1b40b6160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c8107a20 0x7fc1c80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:41.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81070a0 0x7fc1c80fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:41.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 --2- 172.21.15.60:0/2208076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8101690 0x7fc1c80fd600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:41.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 >> 172.21.15.60:0/2208076463 conn(0x7fc1c80fd100 msgr2=0x7fc1c806b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:41.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 shutdown_connections 2024-07-24T21:45:41.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:41.094+0000 7fc1cfc42640 1 -- 172.21.15.60:0/2208076463 wait complete. 2024-07-24T21:45:41.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:41 smithi060 ceph-mon[33000]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:41.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:41 smithi123 ceph-mon[37143]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:41.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:41 smithi086 ceph-mon[37128]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:45:41.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-24T21:45:42.020 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:42.470 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:42 smithi060 ceph-mon[33000]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:42.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:42 smithi123 ceph-mon[37143]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:42.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:42 smithi086 ceph-mon[37128]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:43 smithi123 ceph-mon[37143]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:43.495 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:43 smithi060 ceph-mon[33000]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:43 smithi086 ceph-mon[37128]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:45:43.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 -- 172.21.15.60:0/3976977385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 msgr2=0x7fbffc10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:43.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 --2- 172.21.15.60:0/3976977385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc10f330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009ca0 tx=0x7fbff402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 -- 172.21.15.60:0/3976977385 shutdown_connections 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 --2- 172.21.15.60:0/3976977385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc10f330 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 --2- 172.21.15.60:0/3976977385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1081b0 0x7fbffc1085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 --2- 172.21.15.60:0/3976977385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1027a0 0x7fbffc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 -- 172.21.15.60:0/3976977385 >> 172.21.15.60:0/3976977385 conn(0x7fbffc0fe1d0 msgr2=0x7fbffc1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 -- 172.21.15.60:0/3976977385 shutdown_connections 2024-07-24T21:45:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.696+0000 7fc0029d8640 1 -- 172.21.15.60:0/3976977385 wait complete. 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 Processor -- start 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 -- start start 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 0x7fbffc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1081b0 0x7fbffc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc112830 con 0x7fbffc108b30 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1126b0 con 0x7fbffc1081b0 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc0029d8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1129b0 con 0x7fbffc1027a0 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc000f75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc000f75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36098/0 (socket says 172.21.15.60:36098) 2024-07-24T21:45:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.697+0000 7fc000f75640 1 -- 172.21.15.60:0/666671542 learned_addr learned my addr 172.21.15.60:0/666671542 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbffb7fe640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1081b0 0x7fbffc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbffbfff640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 0x7fbffc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 -- 172.21.15.60:0/666671542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 msgr2=0x7fbffc198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 0x7fbffc198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 -- 172.21.15.60:0/666671542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1081b0 msgr2=0x7fbffc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1081b0 0x7fbffc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 -- 172.21.15.60:0/666671542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc19eed0 con 0x7fbffc108b30 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbffbfff640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 0x7fbffc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc000f75640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005cc0 tx=0x7fbff402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4003cf0 con 0x7fbffc108b30 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff40044f0 con 0x7fbffc108b30 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc19f160 con 0x7fbffc108b30 2024-07-24T21:45:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a5880 con 0x7fbffc108b30 2024-07-24T21:45:43.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.698+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff402fc30 con 0x7fbffc108b30 2024-07-24T21:45:43.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.699+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc102c00 con 0x7fbffc108b30 2024-07-24T21:45:43.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.701+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fbff40046a0 con 0x7fbffc108b30 2024-07-24T21:45:43.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.701+0000 7fbff97fa640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 0x7fbfe40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:43.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.701+0000 7fbffbfff640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 0x7fbfe40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:43.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.702+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbff4105580 con 0x7fbffc108b30 2024-07-24T21:45:43.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.702+0000 7fbffbfff640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 0x7fbfe40b6160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009930 tx=0x7fbfec008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:43.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.705+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbff40ccab0 con 0x7fbffc108b30 2024-07-24T21:45:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.907+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbffc062f80 con 0x7fbfe40b3ca0 2024-07-24T21:45:43.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.913+0000 7fbff97fa640 1 -- 172.21.15.60:0/666671542 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbffc062f80 con 0x7fbfe40b3ca0 2024-07-24T21:45:43.914 INFO:teuthology.orchestra.run.smithi060.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-24T21:45:43.914 INFO:teuthology.orchestra.run.smithi060.stdout:mds.all 12/12 31s ago 3m smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-07-24T21:45:43.914 INFO:teuthology.orchestra.run.smithi060.stdout:mgr 3/3 31s ago 9m smithi060=x;smithi086=y;smithi123=z;count:3 2024-07-24T21:45:43.915 INFO:teuthology.orchestra.run.smithi060.stdout:mon 3/3 31s ago 9m smithi060:172.21.15.60=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2024-07-24T21:45:43.915 INFO:teuthology.orchestra.run.smithi060.stdout:osd 12 31s ago - 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 msgr2=0x7fbfe40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 0x7fbfe40b6160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009930 tx=0x7fbfec008040 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 msgr2=0x7fbffc19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005cc0 tx=0x7fbff402f470 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 shutdown_connections 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbfe40b3ca0 0x7fbfe40b6160 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbffc108b30 0x7fbffc19e700 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbffc1081b0 0x7fbffc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 --2- 172.21.15.60:0/666671542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbffc1027a0 0x7fbffc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 >> 172.21.15.60:0/666671542 conn(0x7fbffc0fe1d0 msgr2=0x7fbffc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 shutdown_connections 2024-07-24T21:45:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:43.916+0000 7fc0029d8640 1 -- 172.21.15.60:0/666671542 wait complete. 2024-07-24T21:45:44.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-24T21:45:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:45.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:45 smithi060 ceph-mon[33000]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:45.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:45 smithi060 ceph-mon[33000]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:45 smithi123 ceph-mon[37143]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:45 smithi123 ceph-mon[37143]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:45 smithi086 ceph-mon[37128]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:45 smithi086 ceph-mon[37128]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:46.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 -- 172.21.15.60:0/3038092446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 msgr2=0x7f938c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:46.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 --2- 172.21.15.60:0/3038092446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c102b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f937c009900 tx=0x7f937c02a410 comp rx=0 tx=0).stop 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 -- 172.21.15.60:0/3038092446 shutdown_connections 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 --2- 172.21.15.60:0/3038092446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c108b20 0x7f938c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 --2- 172.21.15.60:0/3038092446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c1081a0 0x7f938c1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 --2- 172.21.15.60:0/3038092446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c102b70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 -- 172.21.15.60:0/3038092446 >> 172.21.15.60:0/3038092446 conn(0x7f938c0fe200 msgr2=0x7f938c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 -- 172.21.15.60:0/3038092446 shutdown_connections 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 -- 172.21.15.60:0/3038092446 wait complete. 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.455+0000 7f939150e640 1 Processor -- start 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 -- start start 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c1081a0 0x7f938c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:46.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 0x7f938c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f938c112720 con 0x7f938c102790 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1125a0 con 0x7f938c108b20 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f939150e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1128a0 con 0x7f938c1081a0 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33396/0 (socket says 172.21.15.60:33396) 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 -- 172.21.15.60:0/2639019753 learned_addr learned my addr 172.21.15.60:0/2639019753 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938b7fe640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 0x7f938c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938a7fc640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c1081a0 0x7f938c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 -- 172.21.15.60:0/2639019753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c1081a0 msgr2=0x7f938c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c1081a0 0x7f938c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 -- 172.21.15.60:0/2639019753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 msgr2=0x7f938c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 0x7f938c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938affd640 1 -- 172.21.15.60:0/2639019753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c19ece0 con 0x7f938c102790 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.456+0000 7f938b7fe640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 0x7f938c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f938affd640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f937c005940 tx=0x7f937c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c03f030 con 0x7f938c102790 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f937c003aa0 con 0x7f938c102790 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c19ef70 con 0x7f938c102790 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c035070 con 0x7f938c102790 2024-07-24T21:45:46.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.457+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c1a5690 con 0x7f938c102790 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.458+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c102b70 con 0x7f938c102790 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.459+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f937c042030 con 0x7f938c102790 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.460+0000 7f9373fff640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 0x7f93740b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.460+0000 7f938a7fc640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 0x7f93740b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.460+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f937c104770 con 0x7f938c102790 2024-07-24T21:45:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.461+0000 7f938a7fc640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 0x7f93740b60e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f938c107790 tx=0x7f9380005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.463+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f937c0cbca0 con 0x7f938c102790 2024-07-24T21:45:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.666+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f938c062f80 con 0x7f93740b3c20 2024-07-24T21:45:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.667+0000 7f9373fff640 1 -- 172.21.15.60:0/2639019753 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f938c062f80 con 0x7f93740b3c20 2024-07-24T21:45:46.668 INFO:teuthology.orchestra.run.smithi060.stdout:HOST ADDR LABELS STATUS 2024-07-24T21:45:46.668 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 172.21.15.60 2024-07-24T21:45:46.669 INFO:teuthology.orchestra.run.smithi060.stdout:smithi086 172.21.15.86 2024-07-24T21:45:46.669 INFO:teuthology.orchestra.run.smithi060.stdout:smithi123 172.21.15.123 2024-07-24T21:45:46.669 INFO:teuthology.orchestra.run.smithi060.stdout:3 hosts in cluster 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 msgr2=0x7f93740b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 0x7f93740b60e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f938c107790 tx=0x7f9380005f90 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 msgr2=0x7f938c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f937c005940 tx=0x7f937c004270 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 shutdown_connections 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93740b3c20 0x7f93740b60e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c108b20 0x7f938c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c1081a0 0x7f938c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 --2- 172.21.15.60:0/2639019753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938c102790 0x7f938c198c30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.669+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 >> 172.21.15.60:0/2639019753 conn(0x7f938c0fe200 msgr2=0x7f938c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.670+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 shutdown_connections 2024-07-24T21:45:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:46.670+0000 7f939150e640 1 -- 172.21.15.60:0/2639019753 wait complete. 2024-07-24T21:45:47.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:47 smithi060 ceph-mon[33000]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-24T21:45:47.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:47 smithi060 ceph-mon[33000]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:47.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-24T21:45:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:47 smithi123 ceph-mon[37143]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-24T21:45:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:47 smithi123 ceph-mon[37143]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:47.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:47 smithi086 ceph-mon[37128]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-24T21:45:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:47 smithi086 ceph-mon[37128]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.209+0000 7f67226a9640 1 -- 172.21.15.60:0/2746284141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 msgr2=0x7f671c10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.209+0000 7f67226a9640 1 --2- 172.21.15.60:0/2746284141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c10ef10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6714009ca0 tx=0x7f671402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 -- 172.21.15.60:0/2746284141 shutdown_connections 2024-07-24T21:45:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 --2- 172.21.15.60:0/2746284141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c10ef10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 --2- 172.21.15.60:0/2746284141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103f50 0x7f671c10c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 --2- 172.21.15.60:0/2746284141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103550 0x7f671c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 -- 172.21.15.60:0/2746284141 >> 172.21.15.60:0/2746284141 conn(0x7f671c0fd0c0 msgr2=0x7f671c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 -- 172.21.15.60:0/2746284141 shutdown_connections 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 -- 172.21.15.60:0/2746284141 wait complete. 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 Processor -- start 2024-07-24T21:45:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.210+0000 7f67226a9640 1 -- start start 2024-07-24T21:45:49.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103550 0x7f671c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103f50 0x7f671c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f671c112760 con 0x7f671c10cb00 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1125e0 con 0x7f671c103f50 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f67226a9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1128e0 con 0x7f671c103550 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33410/0 (socket says 172.21.15.60:33410) 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 -- 172.21.15.60:0/669176137 learned_addr learned my addr 172.21.15.60:0/669176137 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f671bfff640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103550 0x7f671c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f671b7fe640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103f50 0x7f671c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 -- 172.21.15.60:0/669176137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103550 msgr2=0x7f671c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:49.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103550 0x7f671c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 -- 172.21.15.60:0/669176137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103f50 msgr2=0x7f671c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103f50 0x7f671c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 -- 172.21.15.60:0/669176137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c19ed00 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.211+0000 7f6720c46640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f67140319f0 tx=0x7f6714031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.212+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714003bf0 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.212+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c19ef90 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.212+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6714003d90 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.212+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671402fc00 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.212+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1a5730 con 0x7f671c10cb00 2024-07-24T21:45:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.213+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c103930 con 0x7f671c10cb00 2024-07-24T21:45:49.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.214+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f6714004610 con 0x7f671c10cb00 2024-07-24T21:45:49.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.215+0000 7f67197fa640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 0x7f67040b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:49.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.215+0000 7f671bfff640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 0x7f67040b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:49.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.216+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f67141054e0 con 0x7f671c10cb00 2024-07-24T21:45:49.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.216+0000 7f671bfff640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 0x7f67040b60e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f670c00bbe0 tx=0x7f670c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:49.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.218+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f67140cc9b0 con 0x7f671c10cb00 2024-07-24T21:45:49.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.421+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f671c062f80 con 0x7f67040b3c20 2024-07-24T21:45:49.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:49 smithi060 ceph-mon[33000]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:45:49.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.423+0000 7f67197fa640 1 -- 172.21.15.60:0/669176137 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f671c062f80 con 0x7f67040b3c20 2024-07-24T21:45:49.424 INFO:teuthology.orchestra.run.smithi060.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-24T21:45:49.424 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-07-24T21:45:49.424 INFO:teuthology.orchestra.run.smithi060.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-24T21:45:49.425 INFO:teuthology.orchestra.run.smithi060.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-24T21:45:49.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 msgr2=0x7f67040b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:49.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 0x7f67040b60e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f670c00bbe0 tx=0x7f670c00b390 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 msgr2=0x7f671c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f67140319f0 tx=0x7f6714031a20 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 shutdown_connections 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f67040b3c20 0x7f67040b60e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f671c10cb00 0x7f671c19e530 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c103f50 0x7f671c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 --2- 172.21.15.60:0/669176137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f671c103550 0x7f671c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 >> 172.21.15.60:0/669176137 conn(0x7f671c0fd0c0 msgr2=0x7f671c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 shutdown_connections 2024-07-24T21:45:49.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:49.425+0000 7f67226a9640 1 -- 172.21.15.60:0/669176137 wait complete. 2024-07-24T21:45:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:49 smithi123 ceph-mon[37143]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:45:49.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:49 smithi086 ceph-mon[37128]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-24T21:45:50.113 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-24T21:45:50.122 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:45:50.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-24T21:45:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:50.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:50 smithi123 ceph-mon[37143]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:50.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:50 smithi060 ceph-mon[33000]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:50.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:50 smithi086 ceph-mon[37128]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:45:51.430 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:51 smithi060 ceph-mon[33000]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-24T21:45:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:51 smithi123 ceph-mon[37143]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-24T21:45:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:51 smithi086 ceph-mon[37128]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 -- 172.21.15.60:0/699000643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8075850 msgr2=0x7fc6a8112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 --2- 172.21.15.60:0/699000643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8075850 0x7fc6a8112fb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc698009900 tx=0x7fc69802a3f0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 -- 172.21.15.60:0/699000643 shutdown_connections 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 --2- 172.21.15.60:0/699000643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81134f0 0x7fc6a81158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 --2- 172.21.15.60:0/699000643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8075850 0x7fc6a8112fb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 --2- 172.21.15.60:0/699000643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8074f30 0x7fc6a8075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 -- 172.21.15.60:0/699000643 >> 172.21.15.60:0/699000643 conn(0x7fc6a80fe200 msgr2=0x7fc6a8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 -- 172.21.15.60:0/699000643 shutdown_connections 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.945+0000 7fc6af185640 1 -- 172.21.15.60:0/699000643 wait complete. 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 Processor -- start 2024-07-24T21:45:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 -- start start 2024-07-24T21:45:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 0x7fc6a819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8075850 0x7fc6a819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8118d70 con 0x7fc6a81134f0 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8118bf0 con 0x7fc6a8075850 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6af185640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8118ef0 con 0x7fc6a8074f30 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6ad722640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6ad722640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33436/0 (socket says 172.21.15.60:33436) 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6ad722640 1 -- 172.21.15.60:0/2771378739 learned_addr learned my addr 172.21.15.60:0/2771378739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc69ffff640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8075850 0x7fc6a819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.946+0000 7fc6acf21640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 0x7fc6a819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 -- 172.21.15.60:0/2771378739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 msgr2=0x7fc6a819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 0x7fc6a819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 -- 172.21.15.60:0/2771378739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8075850 msgr2=0x7fc6a819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8075850 0x7fc6a819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 -- 172.21.15.60:0/2771378739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81a5310 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6acf21640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 0x7fc6a819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6ad722640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000ccb0 tx=0x7fc6a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000f450 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a000faa0 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81a5600 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0018b00 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.947+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81abcc0 con 0x7fc6a81134f0 2024-07-24T21:45:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.948+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8075310 con 0x7fc6a81134f0 2024-07-24T21:45:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.949+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fc6a000fc50 con 0x7fc6a81134f0 2024-07-24T21:45:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.950+0000 7fc69dffb640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 0x7fc6900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.951+0000 7fc6acf21640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 0x7fc6900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.951+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc6a00e4470 con 0x7fc6a81134f0 2024-07-24T21:45:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.951+0000 7fc6acf21640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 0x7fc6900b60e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400abe0 tx=0x7fc6a400a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:51.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:51.955+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc6a0014030 con 0x7fc6a81134f0 2024-07-24T21:45:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.191+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc6a81abf90 con 0x7fc6a81134f0 2024-07-24T21:45:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.191+0000 7fc69dffb640 1 -- 172.21.15.60:0/2771378739 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3087 (secure 0 0 0) 0x7fc6a00ab9a0 con 0x7fc6a81134f0 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:e25 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:btime 2024-07-24T21:43:32:377013+0000 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:legacy client fscid: 1 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:Filesystem 'cephfs' (1) 2024-07-24T21:45:52.193 INFO:teuthology.orchestra.run.smithi060.stdout:fs_name cephfs 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 25 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-07-24T21:43:14.079489+0000 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-07-24T21:43:32.191585+0000 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:tableserver 0 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:root 0 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:session_timeout 300 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:session_autoclose 300 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:max_file_size 1099511627776 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:max_xattr_size 65536 2024-07-24T21:45:52.194 INFO:teuthology.orchestra.run.smithi060.stdout:required_client_features {} 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure 0 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure_osd_epoch 0 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:max_mds 3 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:in 0,1,2 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:up {0=24610,1=24605,2=24598} 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:failed 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:damaged 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:stopped 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:data_pools [3] 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:metadata_pool 2 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:inline_data disabled 2024-07-24T21:45:52.195 INFO:teuthology.orchestra.run.smithi060.stdout:balancer 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:bal_rank_mask -1 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:standby_count_wanted 0 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:qdb_cluster leader: 24610 members: 24610,24605,24598 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.i{0:24610} state up:active seq 8 addr [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.f{0:24586} state up:standby-replay seq 1 addr [v2:172.21.15.123:6834/3784109187,v1:172.21.15.123:6835/3784109187] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b{1:24605} state up:active seq 8 addr [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.h{1:24578} state up:standby-replay seq 1 addr [v2:172.21.15.86:6836/549557535,v1:172.21.15.86:6837/549557535] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c{2:24598} state up:active seq 12 addr [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.e{2:24560} state up:standby-replay seq 1 addr [v2:172.21.15.86:6834/1999114136,v1:172.21.15.86:6835/1999114136] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:Standby daemons: 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:52.196 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.j{-1:14709} state up:standby seq 1 addr [v2:172.21.15.60:6834/1887502837,v1:172.21.15.60:6835/1887502837] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.197 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d{-1:14715} state up:standby seq 1 addr [v2:172.21.15.60:6836/2896838776,v1:172.21.15.60:6837/2896838776] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.197 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.g{-1:14760} state up:standby seq 1 addr [v2:172.21.15.60:6838/3968790109,v1:172.21.15.60:6839/3968790109] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.197 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a{-1:14772} state up:standby seq 1 addr [v2:172.21.15.60:6840/2105555291,v1:172.21.15.60:6841/2105555291] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.197 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.k{-1:24536} state up:standby seq 1 addr [v2:172.21.15.86:6832/3131020456,v1:172.21.15.86:6833/3131020456] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.197 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.l{-1:24556} state up:standby seq 1 addr [v2:172.21.15.123:6832/1405683512,v1:172.21.15.123:6833/1405683512] compat {c=[1],r=[1],i=[1fff]}] 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 msgr2=0x7fc6900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 0x7fc6900b60e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400abe0 tx=0x7fc6a400a5c0 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 msgr2=0x7fc6a81a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000ccb0 tx=0x7fc6a000a620 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 shutdown_connections 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6900b3c20 0x7fc6900b60e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81134f0 0x7fc6a81a4b40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8075850 0x7fc6a819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 --2- 172.21.15.60:0/2771378739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8074f30 0x7fc6a819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 >> 172.21.15.60:0/2771378739 conn(0x7fc6a80fe200 msgr2=0x7fc6a80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 shutdown_connections 2024-07-24T21:45:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:52.194+0000 7fc6af185640 1 -- 172.21.15.60:0/2771378739 wait complete. 2024-07-24T21:45:52.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-24T21:45:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:53.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:53 smithi123 ceph-mon[37143]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-24T21:45:53.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:53 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2771378739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-24T21:45:53.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:53 smithi060 ceph-mon[33000]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-24T21:45:53.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:53 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2771378739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-24T21:45:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:53 smithi086 ceph-mon[37128]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-24T21:45:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:53 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2771378739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 -- 172.21.15.60:0/2547559552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103f90 msgr2=0x7f035010c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 --2- 172.21.15.60:0/2547559552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103f90 0x7f035010c480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0338009900 tx=0x7f033802a410 comp rx=0 tx=0).stop 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 -- 172.21.15.60:0/2547559552 shutdown_connections 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 --2- 172.21.15.60:0/2547559552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f035010cad0 0x7f035010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 --2- 172.21.15.60:0/2547559552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103f90 0x7f035010c480 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 --2- 172.21.15.60:0/2547559552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0350103670 0x7f0350103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 -- 172.21.15.60:0/2547559552 >> 172.21.15.60:0/2547559552 conn(0x7f03500fd0f0 msgr2=0x7f03500ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 -- 172.21.15.60:0/2547559552 shutdown_connections 2024-07-24T21:45:54.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.807+0000 7f0354d3b640 1 -- 172.21.15.60:0/2547559552 wait complete. 2024-07-24T21:45:54.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 Processor -- start 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 -- start start 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0350103f90 0x7f03501991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 0x7f035019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0350112710 con 0x7f0350103670 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0350112590 con 0x7f0350103f90 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.808+0000 7f0354d3b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0350112890 con 0x7f035010cad0 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33452/0 (socket says 172.21.15.60:33452) 2024-07-24T21:45:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 -- 172.21.15.60:0/1264403714 learned_addr learned my addr 172.21.15.60:0/1264403714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0345d9b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0350103f90 0x7f03501991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034ed9d640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 0x7f035019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 -- 172.21.15.60:0/1264403714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 msgr2=0x7f035019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 0x7f035019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 -- 172.21.15.60:0/1264403714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0350103f90 msgr2=0x7f03501991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0350103f90 0x7f03501991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 -- 172.21.15.60:0/1264403714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035019ed50 con 0x7f0350103670 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034ed9d640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 0x7f035019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f034e59c640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017030 con 0x7f0350103670 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0340004020 con 0x7f0350103670 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035019f040 con 0x7f0350103670 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400047a0 con 0x7f0350103670 2024-07-24T21:45:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.809+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a5700 con 0x7f0350103670 2024-07-24T21:45:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.810+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350103a50 con 0x7f0350103670 2024-07-24T21:45:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.812+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f0340009a30 con 0x7f0350103670 2024-07-24T21:45:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.813+0000 7f0347fff640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 0x7f033c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.813+0000 7f0345d9b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 0x7f033c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.813+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f03400e3ea0 con 0x7f0350103670 2024-07-24T21:45:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.813+0000 7f0345d9b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 0x7f033c0b60e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f0338031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:54.815+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f03400ab3d0 con 0x7f0350103670 2024-07-24T21:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.011+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f035002e000 con 0x7f0350103670 2024-07-24T21:45:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.012+0000 7f0347fff640 1 -- 172.21.15.60:0/1264403714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5727 (secure 0 0 0) 0x7f03400aacd0 con 0x7f0350103670 2024-07-24T21:45:55.013 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 87 2024-07-24T21:45:55.013 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T21:45:55.013 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-07-24T21:34:36.682798+0000 2024-07-24T21:45:55.013 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-07-24T21:44:37.734936+0000 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 26 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client luminous 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-24T21:45:55.014 INFO:teuthology.orchestra.run.smithi060.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 12 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 9 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/2629637807,v1:172.21.15.60:6803/2629637807] [v2:172.21.15.60:6804/2629637807,v1:172.21.15.60:6805/2629637807] exists,up c5cc77b0-13c2-4164-822c-c458ae4cac91 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 14 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/2205288191,v1:172.21.15.86:6801/2205288191] [v2:172.21.15.86:6802/2205288191,v1:172.21.15.86:6803/2205288191] exists,up 0a9d0f0b-703f-4b07-b1c3-6c12a00261d6 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 19 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/3586156659,v1:172.21.15.123:6801/3586156659] [v2:172.21.15.123:6802/3586156659,v1:172.21.15.123:6803/3586156659] exists,up c9f91f69-71de-4bb0-81b0-a3a5da7c0733 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 27 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/1240167260,v1:172.21.15.60:6811/1240167260] [v2:172.21.15.60:6812/1240167260,v1:172.21.15.60:6813/1240167260] exists,up 824da9ca-8ade-4664-8ec4-084f409bcacc 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/1082160365,v1:172.21.15.86:6809/1082160365] [v2:172.21.15.86:6810/1082160365,v1:172.21.15.86:6811/1082160365] exists,up bfc77006-c095-442e-8894-7d5901632cc0 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/44992122,v1:172.21.15.123:6809/44992122] [v2:172.21.15.123:6810/44992122,v1:172.21.15.123:6811/44992122] exists,up a841523e-44a1-4112-81ab-b61d962ed02e 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 45 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/2857305108,v1:172.21.15.60:6819/2857305108] [v2:172.21.15.60:6820/2857305108,v1:172.21.15.60:6821/2857305108] exists,up cdea9ac1-bcee-44d9-be88-5b587cb1f7bf 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 51 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/3700904835,v1:172.21.15.86:6817/3700904835] [v2:172.21.15.86:6818/3700904835,v1:172.21.15.86:6819/3700904835] exists,up bc920fda-05a3-4fde-92f9-efdfb7776621 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/2616635066,v1:172.21.15.123:6817/2616635066] [v2:172.21.15.123:6818/2616635066,v1:172.21.15.123:6819/2616635066] exists,up 16f5ae1a-e0de-4d8d-82da-211c1a11328a 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 up in weight 1 up_from 64 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/451392111,v1:172.21.15.60:6827/451392111] [v2:172.21.15.60:6828/451392111,v1:172.21.15.60:6829/451392111] exists,up a27b6df3-b9da-4c55-b541-2dbce29fcf6f 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 up in weight 1 up_from 69 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/2411147999,v1:172.21.15.86:6825/2411147999] [v2:172.21.15.86:6826/2411147999,v1:172.21.15.86:6827/2411147999] exists,up ce9b27b5-9ff8-4004-b2b7-fa97986a66cd 2024-07-24T21:45:55.015 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 up in weight 1 up_from 75 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/1138852346,v1:172.21.15.123:6825/1138852346] [v2:172.21.15.123:6826/1138852346,v1:172.21.15.123:6827/1138852346] exists,up 384e3184-0630-4bce-bb27-ca026eb0597d 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.5 [8,11] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.b [8,11] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.e [2,1] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.1d [2,4] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.0 [4,8] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.12 [3,9] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.17 [4,6,5,7] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.21 [5,9] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.28 [3,6] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.36 [4,9] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.3a [4,8] 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/895848880 expires 2024-07-25T21:36:35.954660+0000 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/458230597 expires 2024-07-25T21:36:35.954660+0000 2024-07-24T21:45:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/764504102 expires 2024-07-25T21:36:35.954660+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2052402589 expires 2024-07-25T21:34:59.599034+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/287976863 expires 2024-07-25T21:34:59.599034+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/2613800927 expires 2024-07-25T21:34:59.599034+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3417400078 expires 2024-07-25T21:34:59.599034+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/317343788 expires 2024-07-25T21:36:35.954660+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/751678544 expires 2024-07-25T21:35:24.132100+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/458230597 expires 2024-07-25T21:36:35.954660+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3626499665 expires 2024-07-25T21:35:24.132100+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/485987782 expires 2024-07-25T21:35:24.132100+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/2613800927 expires 2024-07-25T21:34:59.599034+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/485987782 expires 2024-07-25T21:35:24.132100+0000 2024-07-24T21:45:55.017 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3095475402 expires 2024-07-25T21:35:24.132100+0000 2024-07-24T21:45:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 msgr2=0x7f033c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 0x7f033c0b60e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f0338031400 comp rx=0 tx=0).stop 2024-07-24T21:45:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 msgr2=0x7f0350198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).stop 2024-07-24T21:45:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 shutdown_connections 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f033c0b3c20 0x7f033c0b60e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f035010cad0 0x7f035019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0350103f90 0x7f03501991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 --2- 172.21.15.60:0/1264403714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0350103670 0x7f0350198ca0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 >> 172.21.15.60:0/1264403714 conn(0x7f03500fd0f0 msgr2=0x7f03500fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 shutdown_connections 2024-07-24T21:45:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:55.014+0000 7f0354d3b640 1 -- 172.21.15.60:0/1264403714 wait complete. 2024-07-24T21:45:55.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:55 smithi060 ceph-mon[33000]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:55.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:55 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1264403714' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-24T21:45:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:55 smithi123 ceph-mon[37143]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:55 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1264403714' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-24T21:45:55.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:55 smithi086 ceph-mon[37128]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-24T21:45:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:55 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1264403714' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-24T21:45:55.741 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-24T21:45:55.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:45:56.019 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:57.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:57 smithi123 ceph-mon[37143]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:57.514 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:57 smithi060 ceph-mon[33000]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:57 smithi086 ceph-mon[37128]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:45:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 -- 172.21.15.60:0/855555211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 msgr2=0x7fb1e01074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 --2- 172.21.15.60:0/855555211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e01074b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009900 tx=0x7fb1c802a410 comp rx=0 tx=0).stop 2024-07-24T21:45:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 -- 172.21.15.60:0/855555211 shutdown_connections 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 --2- 172.21.15.60:0/855555211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e01079f0 0x7fb1e010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 --2- 172.21.15.60:0/855555211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e01074b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 --2- 172.21.15.60:0/855555211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e0101660 0x7fb1e0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 -- 172.21.15.60:0/855555211 >> 172.21.15.60:0/855555211 conn(0x7fb1e00fd0b0 msgr2=0x7fb1e00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 -- 172.21.15.60:0/855555211 shutdown_connections 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.598+0000 7fb1e536d640 1 -- 172.21.15.60:0/855555211 wait complete. 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 Processor -- start 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 -- start start 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 0x7fb1e0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 0x7fb1e019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e0111670 con 0x7fb1e0107070 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01114f0 con 0x7fb1e0101660 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1e536d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01117f0 con 0x7fb1e01079f0 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56064/0 (socket says 172.21.15.60:56064) 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 -- 172.21.15.60:0/627240142 learned_addr learned my addr 172.21.15.60:0/627240142 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1deffd640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 0x7fb1e0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 -- 172.21.15.60:0/627240142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 msgr2=0x7fb1e019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1df7fe640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 0x7fb1e019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 0x7fb1e019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 -- 172.21.15.60:0/627240142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 msgr2=0x7fb1e0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 0x7fb1e0197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.599+0000 7fb1d7fff640 1 -- 172.21.15.60:0/627240142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e019dc70 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1deffd640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 0x7fb1e0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1df7fe640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 0x7fb1e019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1d7fff640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006010 tx=0x7fb1c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c803f030 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c8003800 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e019df00 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8035070 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.600+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e01a4620 con 0x7fb1e0107070 2024-07-24T21:45:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.601+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e0101a40 con 0x7fb1e0107070 2024-07-24T21:45:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.604+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fb1c8003cb0 con 0x7fb1e0107070 2024-07-24T21:45:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.604+0000 7fb1dcff9640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 0x7fb1cc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:45:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.605+0000 7fb1deffd640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 0x7fb1cc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:45:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.605+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb1c81049d0 con 0x7fb1e0107070 2024-07-24T21:45:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.605+0000 7fb1deffd640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 0x7fb1cc0b6160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0006fd0 tx=0x7fb1d0008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:45:57.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.607+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb1c80cbf00 con 0x7fb1e0107070 2024-07-24T21:45:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.842+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1e01097d0 con 0x7fb1e0107070 2024-07-24T21:45:57.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.843+0000 7fb1dcff9640 1 -- 172.21.15.60:0/627240142 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fb1c80cb800 con 0x7fb1e0107070 2024-07-24T21:45:57.844 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:45:57.845 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:45:57.845 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:45:57.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 msgr2=0x7fb1cc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:57.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 0x7fb1cc0b6160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0006fd0 tx=0x7fb1d0008040 comp rx=0 tx=0).stop 2024-07-24T21:45:57.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 msgr2=0x7fb1e0198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:45:57.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006010 tx=0x7fb1c80042f0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 shutdown_connections 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb1cc0b3ca0 0x7fb1cc0b6160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1e01079f0 0x7fb1e019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1e0107070 0x7fb1e0198100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 --2- 172.21.15.60:0/627240142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e0101660 0x7fb1e0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:45:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 >> 172.21.15.60:0/627240142 conn(0x7fb1e00fd0b0 msgr2=0x7fb1e00ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:45:57.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 shutdown_connections 2024-07-24T21:45:57.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:45:57.845+0000 7fb1e536d640 1 -- 172.21.15.60:0/627240142 wait complete. 2024-07-24T21:45:58.468 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:58 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/627240142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:58.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:58 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/627240142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:58.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:45:58.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:58 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/627240142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:45:58.768 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:45:59.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:45:59 smithi123 ceph-mon[37143]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:45:59 smithi086 ceph-mon[37128]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:45:59.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:45:59 smithi060 ceph-mon[33000]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 -- 172.21.15.60:0/994418361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 msgr2=0x7ffb1c10ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 --2- 172.21.15.60:0/994418361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c10ef20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009ca0 tx=0x7ffb1402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 -- 172.21.15.60:0/994418361 shutdown_connections 2024-07-24T21:46:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 --2- 172.21.15.60:0/994418361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c10ef20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 --2- 172.21.15.60:0/994418361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 0x7ffb1c10c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 --2- 172.21.15.60:0/994418361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 0x7ffb1c103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 -- 172.21.15.60:0/994418361 >> 172.21.15.60:0/994418361 conn(0x7ffb1c0fd0c0 msgr2=0x7ffb1c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 -- 172.21.15.60:0/994418361 shutdown_connections 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.395+0000 7ffb22688640 1 -- 172.21.15.60:0/994418361 wait complete. 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 Processor -- start 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 -- start start 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 0x7ffb1c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 0x7ffb1c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c112810 con 0x7ffb1c10cac0 2024-07-24T21:46:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c112690 con 0x7ffb1c103660 2024-07-24T21:46:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb22688640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c112990 con 0x7ffb1c103f80 2024-07-24T21:46:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56088/0 (socket says 172.21.15.60:56088) 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 -- 172.21.15.60:0/1082350324 learned_addr learned my addr 172.21.15.60:0/1082350324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb1b7fe640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 0x7ffb1c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb1bfff640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 0x7ffb1c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 -- 172.21.15.60:0/1082350324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 msgr2=0x7ffb1c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 0x7ffb1c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 -- 172.21.15.60:0/1082350324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 msgr2=0x7ffb1c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 0x7ffb1c198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.396+0000 7ffb20c25640 1 -- 172.21.15.60:0/1082350324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c19ee30 con 0x7ffb1c10cac0 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb20c25640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005cc0 tx=0x7ffb1402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14003cf0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c19f0c0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb140044f0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1402fc30 con 0x7ffb1c10cac0 2024-07-24T21:46:00.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.397+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1a5860 con 0x7ffb1c10cac0 2024-07-24T21:46:00.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.398+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c103ac0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.400+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ffb140046a0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.401+0000 7ffb197fa640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 0x7ffb040b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:00.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.401+0000 7ffb1bfff640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 0x7ffb040b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:00.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.401+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffb14105560 con 0x7ffb1c10cac0 2024-07-24T21:46:00.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.401+0000 7ffb1bfff640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 0x7ffb040b60e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009990 tx=0x7ffb0c009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:00.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.404+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ffb140cca90 con 0x7ffb1c10cac0 2024-07-24T21:46:00.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.640+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb1c199eb0 con 0x7ffb1c10cac0 2024-07-24T21:46:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.641+0000 7ffb197fa640 1 -- 172.21.15.60:0/1082350324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7ffb140cc390 con 0x7ffb1c10cac0 2024-07-24T21:46:00.642 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:00.642 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"btime":"2024-07-24T21:43:32:377013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:43:32.191585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:46:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-07-24T21:46:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 msgr2=0x7ffb040b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 0x7ffb040b60e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009990 tx=0x7ffb0c009110 comp rx=0 tx=0).stop 2024-07-24T21:46:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 msgr2=0x7ffb1c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb14005cc0 tx=0x7ffb1402f470 comp rx=0 tx=0).stop 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 shutdown_connections 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffb040b3c20 0x7ffb040b60e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb1c10cac0 0x7ffb1c19e660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb1c103f80 0x7ffb1c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 --2- 172.21.15.60:0/1082350324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1c103660 0x7ffb1c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 >> 172.21.15.60:0/1082350324 conn(0x7ffb1c0fd0c0 msgr2=0x7ffb1c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 shutdown_connections 2024-07-24T21:46:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:00.643+0000 7ffb22688640 1 -- 172.21.15.60:0/1082350324 wait complete. 2024-07-24T21:46:01.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:01 smithi060 ceph-mon[33000]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:01.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:01 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1082350324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:01.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:46:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:01 smithi123 ceph-mon[37143]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:01 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1082350324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:01 smithi086 ceph-mon[37128]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:01.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:01 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1082350324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:01.725 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:03.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 -- 172.21.15.60:0/2585006541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e4108b20 msgr2=0x7f46e410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:03.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 --2- 172.21.15.60:0/2585006541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e4108b20 0x7f46e410f320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ca0 tx=0x7f46dc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 -- 172.21.15.60:0/2585006541 shutdown_connections 2024-07-24T21:46:03.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 --2- 172.21.15.60:0/2585006541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e4108b20 0x7f46e410f320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 --2- 172.21.15.60:0/2585006541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e41081a0 0x7f46e41085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 --2- 172.21.15.60:0/2585006541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4102790 0x7f46e4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 -- 172.21.15.60:0/2585006541 >> 172.21.15.60:0/2585006541 conn(0x7f46e40fe200 msgr2=0x7f46e4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 -- 172.21.15.60:0/2585006541 shutdown_connections 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 -- 172.21.15.60:0/2585006541 wait complete. 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.414+0000 7f46ec446640 1 Processor -- start 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 -- start start 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4102790 0x7f46e4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4108b20 0x7f46e419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f46e4112740 con 0x7f46e41081a0 2024-07-24T21:46:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41125c0 con 0x7f46e4108b20 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ec446640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46e41128c0 con 0x7f46e4102790 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56104/0 (socket says 172.21.15.60:56104) 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 -- 172.21.15.60:0/2049759276 learned_addr learned my addr 172.21.15.60:0/2049759276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ea1e2640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4102790 0x7f46e4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46ea9e3640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4108b20 0x7f46e419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 -- 172.21.15.60:0/2049759276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4102790 msgr2=0x7f46e4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4102790 0x7f46e4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 -- 172.21.15.60:0/2049759276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4108b20 msgr2=0x7f46e419e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4108b20 0x7f46e419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.415+0000 7f46e99e1640 1 -- 172.21.15.60:0/2049759276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e419ecc0 con 0x7f46e41081a0 2024-07-24T21:46:03.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46e99e1640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46d400c890 tx=0x7f46d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4004220 con 0x7f46e41081a0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d40043c0 con 0x7f46e41081a0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e419efb0 con 0x7f46e41081a0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4004a00 con 0x7f46e41081a0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.416+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e41a5670 con 0x7f46e41081a0 2024-07-24T21:46:03.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.417+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e4102b70 con 0x7f46e41081a0 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.419+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f46d40079b0 con 0x7f46e41081a0 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.420+0000 7f46db7fe640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 0x7f46d00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.420+0000 7f46ea1e2640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 0x7f46d00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.420+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f46d40e3ee0 con 0x7f46e41081a0 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.420+0000 7f46ea1e2640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 0x7f46d00b60e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46e000bbe0 tx=0x7f46e000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:03.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.423+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f46d4014030 con 0x7f46e41081a0 2024-07-24T21:46:03.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:03 smithi123 ceph-mon[37143]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:03.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:03 smithi086 ceph-mon[37128]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:03 smithi060 ceph-mon[33000]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.617+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46e410a370 con 0x7f46e41081a0 2024-07-24T21:46:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.618+0000 7f46db7fe640 1 -- 172.21.15.60:0/2049759276 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23071 (secure 0 0 0) 0x7f46d40ab410 con 0x7f46e41081a0 2024-07-24T21:46:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:03.620 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:44:37.734936+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:46:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 msgr2=0x7f46d00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 0x7f46d00b60e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46e000bbe0 tx=0x7f46e000b390 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 msgr2=0x7f46e4199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46d400c890 tx=0x7f46d400cd50 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 shutdown_connections 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f46d00b3c20 0x7f46d00b60e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e4108b20 0x7f46e419e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e41081a0 0x7f46e4199150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 --2- 172.21.15.60:0/2049759276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4102790 0x7f46e4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 >> 172.21.15.60:0/2049759276 conn(0x7f46e40fe200 msgr2=0x7f46e41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 shutdown_connections 2024-07-24T21:46:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:03.620+0000 7f46ec446640 1 -- 172.21.15.60:0/2049759276 wait complete. 2024-07-24T21:46:04.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-24T21:43:14.079489+0000', 'modified': '2024-07-24T21:43:32.191585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24610, 'mds_1': 24605, 'mds_2': 24598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24560': {'gid': 24560, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/1999114136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1999114136}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1999114136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/549557535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 549557535}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 549557535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/3784109187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3784109187}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3784109187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/2950411419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2950411419}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2950411419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.86:6839/1688764220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 1688764220}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 1688764220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.123:6839/3718566677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 3718566677}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 3718566677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24610, 'qdb_cluster': [24610, 24605, 24598]} 2024-07-24T21:46:04.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-24T21:46:04.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.c status 2024-07-24T21:46:04.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:04 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2049759276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:04.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:04 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2049759276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:04.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:04 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2049759276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:04.608 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:05.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:05 smithi060 ceph-mon[33000]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:05.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:05 smithi123 ceph-mon[37143]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:05.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:05 smithi086 ceph-mon[37128]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:06.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 -- 172.21.15.60:0/3107855990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae4103fc0 msgr2=0x7f7ae410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3107855990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae4103fc0 0x7f7ae410c4b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009900 tx=0x7f7ad802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 -- 172.21.15.60:0/3107855990 shutdown_connections 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3107855990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae410cb00 0x7f7ae410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3107855990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae4103fc0 0x7f7ae410c4b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3107855990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 0x7f7ae4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 -- 172.21.15.60:0/3107855990 >> 172.21.15.60:0/3107855990 conn(0x7f7ae40fd100 msgr2=0x7f7ae40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 -- 172.21.15.60:0/3107855990 shutdown_connections 2024-07-24T21:46:06.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.302+0000 7f7aea68b640 1 -- 172.21.15.60:0/3107855990 wait complete. 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.303+0000 7f7aea68b640 1 Processor -- start 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.303+0000 7f7aea68b640 1 -- start start 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 0x7f7ae4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae4103fc0 0x7f7ae4101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae4112750 con 0x7f7ae410cb00 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae41125d0 con 0x7f7ae4103fc0 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae41128d0 con 0x7f7ae41036a0 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43974/0 (socket says 172.21.15.60:43974) 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 -- 172.21.15.60:0/3422600576 learned_addr learned my addr 172.21.15.60:0/3422600576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae4103fc0 0x7f7ae4101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 0x7f7ae4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 -- 172.21.15.60:0/3422600576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 msgr2=0x7f7ae4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 0x7f7ae4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 -- 172.21.15.60:0/3422600576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae4103fc0 msgr2=0x7f7ae4101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae4103fc0 0x7f7ae4101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 -- 172.21.15.60:0/3422600576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae41a7100 con 0x7f7ae410cb00 2024-07-24T21:46:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00eaa0 tx=0x7f7adc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae17fa640 1 -- 172.21.15.60:0/3422600576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00cdb0 con 0x7f7ae410cb00 2024-07-24T21:46:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7ae17fa640 1 -- 172.21.15.60:0/3422600576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7adc005020 con 0x7f7ae410cb00 2024-07-24T21:46:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.304+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae41a73f0 con 0x7f7ae410cb00 2024-07-24T21:46:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.305+0000 7f7ae17fa640 1 -- 172.21.15.60:0/3422600576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0116d0 con 0x7f7ae410cb00 2024-07-24T21:46:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.305+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae41a7900 con 0x7f7ae410cb00 2024-07-24T21:46:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 Processor -- start 2024-07-24T21:46:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7ae17fa640 1 -- 172.21.15.60:0/3422600576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f7adc0051d0 con 0x7f7ae410cb00 2024-07-24T21:46:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 -- start start 2024-07-24T21:46:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e77450 0x5651f1e79840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5651f1de1740 con 0x5651f1e6cb80 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651f1dd0c40 con 0x5651f1e77450 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.307+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651f1dd4c80 con 0x5651f1e6b620 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43982/0 (socket says 172.21.15.60:43982) 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 -- 172.21.15.60:0/2535553695 learned_addr learned my addr 172.21.15.60:0/2535553695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e77450 0x5651f1e79840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 -- 172.21.15.60:0/2535553695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 msgr2=0x5651f1e6ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e6ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 -- 172.21.15.60:0/2535553695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e77450 msgr2=0x5651f1e79840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e77450 0x5651f1e79840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 -- 172.21.15.60:0/2535553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f1d5a1f0 con 0x5651f1e6cb80 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae17fa640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 0x7f7acc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e6ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406bcf0 tx=0x7f7ad8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebe0fd5050cf088 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 0x7f7acc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.308+0000 7f7ae17fa640 1 -- 172.21.15.60:0/3422600576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7adc0e3f00 con 0x7f7ae410cb00 2024-07-24T21:46:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.309+0000 7f7aca7fc640 1 -- 172.21.15.60:0/2535553695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8044020 con 0x5651f1e6cb80 2024-07-24T21:46:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.309+0000 7f7aca7fc640 1 -- 172.21.15.60:0/2535553695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad8031870 con 0x5651f1e6cb80 2024-07-24T21:46:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.309+0000 7f7aca7fc640 1 -- 172.21.15.60:0/2535553695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8035040 con 0x5651f1e6cb80 2024-07-24T21:46:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2535553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 msgr2=0x5651f1e6b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 0x7f7acc0b6160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5651f1e6e260 tx=0x7f7ad400c700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406bcf0 tx=0x7f7ad8009640 comp rx=0 tx=0).stop 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2535553695 shutdown_connections 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e77450 0x5651f1e79840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e6ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2535553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6cb80 0x5651f1e6b0e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2535553695 >> 172.21.15.60:0/2535553695 conn(0x5651f1e096c0 msgr2=0x5651f1e68f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2535553695 shutdown_connections 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2535553695 wait complete. 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 -- start start 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.310+0000 7f7aeca3c740 1 Processor -- start 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5651f1c345f0 con 0x5651f1e77450 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651f1d734c0 con 0x5651f1e6cb80 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e7a210 con 0x5651f1e6b620 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7aeca3c740 10 client.0 _ll_register_callbacks cb 0x5651f1dc90c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43998/0 (socket says 172.21.15.60:43998) 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 -- 172.21.15.60:0/1210315039 learned_addr learned my addr 172.21.15.60:0/1210315039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 -- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 msgr2=0x5651f1e76c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e76c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 -- 172.21.15.60:0/1210315039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 msgr2=0x5651f1e740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.311+0000 7f7ae8c28640 1 -- 172.21.15.60:0/1210315039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f1d66550 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0a7fa0 tx=0x7f7adc0ab4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16094aaf201e1b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc01f040 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7adc0a9f90 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad8031870 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7adc0b3de0 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7aeca3c740 10 client.15243 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651f1b31540 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad8031870 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.312+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc00f040 con 0x5651f1e77450 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 10 client.15243 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 10 client.15243 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 20 client.15243 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1e87e10 0x5651f1e89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 4 client.15243 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651f1de3690 con 0x5651f1e87e10 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1e87e10 0x5651f1e89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1e87e10 0x5651f1e89280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cb4bd8b693dddfb server_cookie=9f1371d6eee25bdd in_seq=0 out_seq=0 2024-07-24T21:46:06.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.313+0000 7f7ac37fe640 10 client.15243 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7ac37fe640 1 -- 172.21.15.60:0/1210315039 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651f1de3690 con 0x5651f1e87e10 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7ac37fe640 10 client.15243 handle_command_reply: tid=0 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 client.15243 shutdown 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 msgr2=0x5651f1e74630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0a7fa0 tx=0x7f7adc0ab4b0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 shutdown_connections 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1e87e10 0x5651f1e89280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e74630 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7ac37fe640 0 client.15243 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/1210315039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.314+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 >> 172.21.15.60:0/1210315039 conn(0x5651f1e096c0 msgr2=0x5651f1e6eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.315+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 shutdown_connections 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.315+0000 7f7aeca3c740 1 -- 172.21.15.60:0/1210315039 wait complete. 2024-07-24T21:46:06.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.315+0000 7f7aeca3c740 20 client.15243 trim_cache size 0 max 16384 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 Processor -- start 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 -- start start 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e68ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e77450 0x5651f1e75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e875c0 con 0x5651f1e6b620 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e87c90 con 0x5651f1e6cb80 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e88360 con 0x5651f1e77450 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44010/0 (socket says 172.21.15.60:44010) 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 -- 172.21.15.60:0/3103191156 learned_addr learned my addr 172.21.15.60:0/3103191156 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e77450 0x5651f1e75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 -- 172.21.15.60:0/3103191156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e77450 msgr2=0x5651f1e75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e77450 0x5651f1e75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 -- 172.21.15.60:0/3103191156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 msgr2=0x5651f1e68ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e68ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e68ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 -- 172.21.15.60:0/3103191156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f1e69e00 con 0x5651f1e6b620 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.334+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5651f1e74d80 tx=0x7f7ad4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb2a91a362eea55 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7ac9ffb640 1 -- 172.21.15.60:0/3103191156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4011df0 con 0x5651f1e6b620 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7ac9ffb640 1 -- 172.21.15.60:0/3103191156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad4020110 con 0x5651f1e6b620 2024-07-24T21:46:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7ac9ffb640 1 -- 172.21.15.60:0/3103191156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4005020 con 0x5651f1e6b620 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 -- 172.21.15.60:0/3103191156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 msgr2=0x5651f1e89280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5651f1e74d80 tx=0x7f7ad4013000 comp rx=0 tx=0).stop 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 -- 172.21.15.60:0/3103191156 shutdown_connections 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e77450 0x5651f1e75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e68ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/3103191156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e6b620 0x5651f1e89280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 -- 172.21.15.60:0/3103191156 >> 172.21.15.60:0/3103191156 conn(0x5651f1e096c0 msgr2=0x5651f1e75610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 -- 172.21.15.60:0/3103191156 shutdown_connections 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.335+0000 7f7aeca3c740 1 -- 172.21.15.60:0/3103191156 wait complete. 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 -- start start 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 Processor -- start 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e70a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e89100 con 0x5651f1e77450 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e88a30 con 0x5651f1e6cb80 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651f1e897d0 con 0x5651f1e6b620 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7aeca3c740 10 client.0 _ll_register_callbacks cb 0x7f7adc0a6bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7ae8c28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44016/0 (socket says 172.21.15.60:44016) 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7ae8c28640 1 -- 172.21.15.60:0/2041181232 learned_addr learned my addr 172.21.15.60:0/2041181232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.336+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e70a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 -- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 msgr2=0x5651f1e70530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e70530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 -- 172.21.15.60:0/2041181232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 msgr2=0x5651f1e70a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e70a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 -- 172.21.15.60:0/2041181232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f1e71a50 con 0x5651f1e77450 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae37fe640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e70a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ae8c28640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00cfd0 tx=0x7f7adc005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0a606baa57e41b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0ca060 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7adc0a9a00 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad4020110 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7adc01f040 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7aeca3c740 10 client.15249 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651f1ef2f20 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad4020110 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc08d080 con 0x5651f1e77450 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.337+0000 7f7aeca3c740 10 client.15249 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 20 client.15249 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1ef7af0 0x5651f1ef6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 4 client.15249 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "status"}) -- 0x5651f1ddf130 con 0x5651f1ef7af0 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1ef7af0 0x5651f1ef6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7ae3fff640 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1ef7af0 0x5651f1ef6800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94b6738859dea30b server_cookie=735eaa344a77cf43 in_seq=0 out_seq=0 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7ac37fe640 10 client.15249 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7ac37fe640 1 -- 172.21.15.60:0/2041181232 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5651f1ddf130 con 0x5651f1ef7af0 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7ac37fe640 10 client.15249 handle_command_reply: tid=0 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.338+0000 7f7aeca3c740 1 client.15249 shutdown 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 msgr2=0x5651f1e71220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00cfd0 tx=0x7f7adc005e10 comp rx=0 tx=0).stop 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 shutdown_connections 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5651f1ef7af0 0x5651f1ef6800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651f1e77450 0x5651f1e71220 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651f1e6cb80 0x5651f1e70a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 --2- 172.21.15.60:0/2041181232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651f1e6b620 0x5651f1e70530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 >> 172.21.15.60:0/2041181232 conn(0x5651f1e096c0 msgr2=0x5651f1e75cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7ac37fe640 0 client.15249 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 shutdown_connections 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 1 -- 172.21.15.60:0/2041181232 wait complete. 2024-07-24T21:46:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aeca3c740 20 client.15249 trim_cache size 0 max 16384 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24598, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 158.98045063399999, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:46:06.347 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 201.97325368700001 2024-07-24T21:46:06.348 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 msgr2=0x7f7acc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 0x7f7acc0b6160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5651f1e6e260 tx=0x7f7ad400c700 comp rx=0 tx=0).stop 2024-07-24T21:46:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 msgr2=0x7f7ae4101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.339+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00eaa0 tx=0x7f7adc00ef60 comp rx=0 tx=0).stop 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 shutdown_connections 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7acc0b3ca0 0x7f7acc0b6160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae410cb00 0x7f7ae4101dd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ae4103fc0 0x7f7ae4101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 --2- 172.21.15.60:0/3422600576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ae41036a0 0x7f7ae4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 >> 172.21.15.60:0/3422600576 conn(0x7f7ae40fd100 msgr2=0x7f7ae40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 shutdown_connections 2024-07-24T21:46:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:06.340+0000 7f7aea68b640 1 -- 172.21.15.60:0/3422600576 wait complete. 2024-07-24T21:46:07.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.b status 2024-07-24T21:46:07.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:07 smithi060 ceph-mon[33000]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:07 smithi123 ceph-mon[37143]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:07.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:07 smithi086 ceph-mon[37128]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 -- 172.21.15.60:0/3046557735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b410ede0 msgr2=0x7fd1b41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3046557735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b410ede0 0x7fd1b41111d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009ca0 tx=0x7fd1ac02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 -- 172.21.15.60:0/3046557735 shutdown_connections 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3046557735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b410ede0 0x7fd1b41111d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3046557735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101b50 0x7fd1b410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3046557735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b4101230 0x7fd1b4101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 -- 172.21.15.60:0/3046557735 >> 172.21.15.60:0/3046557735 conn(0x7fd1b40fd100 msgr2=0x7fd1b40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 -- 172.21.15.60:0/3046557735 shutdown_connections 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.036+0000 7fd1bba28640 1 -- 172.21.15.60:0/3046557735 wait complete. 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 Processor -- start 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 -- start start 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101230 0x7fd1b419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b410ede0 0x7fd1b41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4114960 con 0x7fd1b4101b50 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41147e0 con 0x7fd1b4101230 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1bba28640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4114ae0 con 0x7fd1b410ede0 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.037+0000 7fd1b8fc3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44040/0 (socket says 172.21.15.60:44040) 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 -- 172.21.15.60:0/3545943043 learned_addr learned my addr 172.21.15.60:0/3545943043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101230 0x7fd1b419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b410ede0 0x7fd1b41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 -- 172.21.15.60:0/3545943043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b410ede0 msgr2=0x7fd1b41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b410ede0 0x7fd1b41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 -- 172.21.15.60:0/3545943043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101230 msgr2=0x7fd1b419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101230 0x7fd1b419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 -- 172.21.15.60:0/3545943043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a0ec0 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800d950 tx=0x7fd1a800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b27fc640 1 -- 172.21.15.60:0/3545943043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8017070 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b27fc640 1 -- 172.21.15.60:0/3545943043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a800de80 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1b27fc640 1 -- 172.21.15.60:0/3545943043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800f620 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a11b0 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.038+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a7a40 con 0x7fd1b4101b50 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1bddd9740 1 Processor -- start 2024-07-24T21:46:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1b27fc640 1 -- 172.21.15.60:0/3545943043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fd1a8009c30 con 0x7fd1b4101b50 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1bddd9740 1 -- start start 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb61a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.041+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfad7720 con 0x55b0cfb62b60 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfac6c20 con 0x55b0cfb6d430 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfacac60 con 0x55b0cfb61600 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44056/0 (socket says 172.21.15.60:44056) 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/1465767931 learned_addr learned my addr 172.21.15.60:0/1465767931 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb61a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/1465767931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 msgr2=0x55b0cfb61a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb61a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/1465767931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 msgr2=0x55b0cfb6f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/1465767931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0cfa501d0 con 0x55b0cfb62b60 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b27fc640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 0x7fd19c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005cc0 tx=0x7fd1ac0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9522bbae6216f076 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 0x7fd19c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd1b27fc640 1 -- 172.21.15.60:0/3545943043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd1a80e3900 con 0x7fd1b4101b50 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.042+0000 7fd19b7fe640 1 -- 172.21.15.60:0/1465767931 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac031450 con 0x55b0cfb62b60 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd19b7fe640 1 -- 172.21.15.60:0/1465767931 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x55b0cfb62b60 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd19b7fe640 1 -- 172.21.15.60:0/1465767931 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0044e0 con 0x55b0cfb62b60 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 -- 172.21.15.60:0/1465767931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 msgr2=0x55b0cfb610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 0x7fd19c0b6160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55b0cfb64240 tx=0x7fd1a400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005cc0 tx=0x7fd1ac0099c0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 -- 172.21.15.60:0/1465767931 shutdown_connections 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/1465767931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb62b60 0x55b0cfb610c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 -- 172.21.15.60:0/1465767931 >> 172.21.15.60:0/1465767931 conn(0x55b0cfaff6a0 msgr2=0x55b0cfb5eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 -- 172.21.15.60:0/1465767931 shutdown_connections 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.043+0000 7fd1bddd9740 1 -- 172.21.15.60:0/1465767931 wait complete. 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 -- start start 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 Processor -- start 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf92a5f0 con 0x55b0cfb6d430 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfa694a0 con 0x55b0cfb62b60 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb701f0 con 0x55b0cfb61600 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1bddd9740 10 client.0 _ll_register_callbacks cb 0x55b0cfabf0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44072/0 (socket says 172.21.15.60:44072) 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/2647354059 learned_addr learned my addr 172.21.15.60:0/2647354059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.044+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/2647354059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 msgr2=0x55b0cfb6cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb6cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 msgr2=0x55b0cfb6a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/2647354059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0cfa5c530 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac003f40 tx=0x7fd1ac003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cbcca596a336f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac04a040 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ac036040 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd1ac031450 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1ac05c050 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1bddd9740 10 client.15261 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0cf827540 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.045+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1ac04d070 con 0x55b0cfb6d430 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 10 client.15261 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:46:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 20 client.15261 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfb7d8d0 0x55b0cfb7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 4 client.15261 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0cfad9670 con 0x55b0cfb7d8d0 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfb7d8d0 0x55b0cfb7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd198ff9640 10 client.15261 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.046+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfb7d8d0 0x55b0cfb7ed40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf8d4508292c1fa2 server_cookie=5e04eddcb3263fec in_seq=0 out_seq=0 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd198ff9640 1 -- 172.21.15.60:0/2647354059 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0cfad9670 con 0x55b0cfb7d8d0 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd198ff9640 10 client.15261 handle_command_reply: tid=0 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 client.15261 shutdown 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 msgr2=0x55b0cfb6a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac003f40 tx=0x7fd1ac003d60 comp rx=0 tx=0).stop 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 shutdown_connections 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfb7d8d0 0x55b0cfb7ed40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb6a5f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/2647354059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb6cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd198ff9640 0 client.15261 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.047+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 >> 172.21.15.60:0/2647354059 conn(0x55b0cfaff6a0 msgr2=0x55b0cfb64a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.048+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 shutdown_connections 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.048+0000 7fd1bddd9740 1 -- 172.21.15.60:0/2647354059 wait complete. 2024-07-24T21:46:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.048+0000 7fd1bddd9740 20 client.15261 trim_cache size 0 max 16384 2024-07-24T21:46:09.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.067+0000 7fd1bddd9740 1 Processor -- start 2024-07-24T21:46:09.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.067+0000 7fd1bddd9740 1 -- start start 2024-07-24T21:46:09.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.067+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb60e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.067+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.067+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7d080 con 0x55b0cfb6d430 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7d750 con 0x55b0cfb62b60 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7de20 con 0x55b0cfb61600 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44076/0 (socket says 172.21.15.60:44076) 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/858538988 learned_addr learned my addr 172.21.15.60:0/858538988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb60e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/858538988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 msgr2=0x55b0cfb60e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb60e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/858538988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 msgr2=0x55b0cfb6a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 -- 172.21.15.60:0/858538988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0cfb685b0 con 0x55b0cfb6d430 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009910 tx=0x7fd1ac005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4434b2d12ebdeff9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.068+0000 7fd19affd640 1 -- 172.21.15.60:0/858538988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac003970 con 0x55b0cfb6d430 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd19affd640 1 -- 172.21.15.60:0/858538988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x55b0cfb6d430 2024-07-24T21:46:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd19affd640 1 -- 172.21.15.60:0/858538988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0316b0 con 0x55b0cfb6d430 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 -- 172.21.15.60:0/858538988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 msgr2=0x55b0cfb5ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009910 tx=0x7fd1ac005c50 comp rx=0 tx=0).stop 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 -- 172.21.15.60:0/858538988 shutdown_connections 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb6d430 0x55b0cfb5ee00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb62b60 0x55b0cfb6a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/858538988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb61600 0x55b0cfb60e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 -- 172.21.15.60:0/858538988 >> 172.21.15.60:0/858538988 conn(0x55b0cfaff6a0 msgr2=0x55b0cfb6b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 -- 172.21.15.60:0/858538988 shutdown_connections 2024-07-24T21:46:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.069+0000 7fd1bddd9740 1 -- 172.21.15.60:0/858538988 wait complete. 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 -- start start 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 Processor -- start 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb62b60 0x55b0cfb864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7ebc0 con 0x55b0cfb61600 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7e4f0 con 0x55b0cfb6d430 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0cfb7f290 con 0x55b0cfb62b60 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1b97c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1bddd9740 10 client.0 _ll_register_callbacks cb 0x7fd1a400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.070+0000 7fd1b97c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44084/0 (socket says 172.21.15.60:44084) 2024-07-24T21:46:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 -- 172.21.15.60:0/763431545 learned_addr learned my addr 172.21.15.60:0/763431545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb62b60 0x55b0cfb864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b9fc5640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 -- 172.21.15.60:0/763431545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb62b60 msgr2=0x55b0cfb864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb62b60 0x55b0cfb864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 -- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 msgr2=0x55b0cfb86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 -- 172.21.15.60:0/763431545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0cfb87240 con 0x55b0cfb61600 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1b97c4640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55b0cfb7fb80 tx=0x7fd1a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d966bfdf6be64498 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4005050 con 0x55b0cfb61600 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ac036040 con 0x55b0cfb61600 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd1a40051f0 con 0x55b0cfb61600 2024-07-24T21:46:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1a4036050 con 0x55b0cfb61600 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1bddd9740 10 client.15267 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.071+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0cfb669d0 con 0x55b0cfb61600 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x55b0cfb61600 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1a4028610 con 0x55b0cfb61600 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 10 client.15267 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 20 client.15267 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfbf5b20 0x55b0cfbf4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 4 client.15267 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "status"}) -- 0x55b0cfad5110 con 0x55b0cfbf5b20 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.072+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfbf5b20 0x55b0cfbf4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1b8fc3640 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfbf5b20 0x55b0cfbf4830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96305ecdaa813726 server_cookie=3a515748fb2e5538 in_seq=0 out_seq=0 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd198ff9640 10 client.15267 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd198ff9640 1 -- 172.21.15.60:0/763431545 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b0cfad5110 con 0x55b0cfbf5b20 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd198ff9640 10 client.15267 handle_command_reply: tid=0 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 client.15267 shutdown 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 msgr2=0x55b0cfb85f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55b0cfb7fb80 tx=0x7fd1a4015000 comp rx=0 tx=0).stop 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 shutdown_connections 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55b0cfbf5b20 0x55b0cfbf4830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cfb6d430 0x55b0cfb86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0cfb62b60 0x55b0cfb864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 --2- 172.21.15.60:0/763431545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0cfb61600 0x55b0cfb85f90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 >> 172.21.15.60:0/763431545 conn(0x55b0cfaff6a0 msgr2=0x55b0cfb64a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd198ff9640 0 client.15267 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 shutdown_connections 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 1 -- 172.21.15.60:0/763431545 wait complete. 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.073+0000 7fd1bddd9740 20 client.15267 trim_cache size 0 max 16384 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24605, 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:46:09.080 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 162.71989949100001, 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 189.86029873300001 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 msgr2=0x7fd19c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 0x7fd19c0b6160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55b0cfb64240 tx=0x7fd1a400a2d0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 msgr2=0x7fd1b419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800d950 tx=0x7fd1a800dd50 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 shutdown_connections 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd19c0b3ca0 0x7fd19c0b6160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1b410ede0 0x7fd1b41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b4101b50 0x7fd1b419b350 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 --2- 172.21.15.60:0/3545943043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4101230 0x7fd1b419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 >> 172.21.15.60:0/3545943043 conn(0x7fd1b40fd100 msgr2=0x7fd1b4110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 shutdown_connections 2024-07-24T21:46:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:09.074+0000 7fd1bba28640 1 -- 172.21.15.60:0/3545943043 wait complete. 2024-07-24T21:46:09.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:09 smithi060 ceph-mon[33000]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:09 smithi123 ceph-mon[37143]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:09.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:09 smithi086 ceph-mon[37128]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:09.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.i status 2024-07-24T21:46:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:11.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:11 smithi060 ceph-mon[33000]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:11 smithi123 ceph-mon[37143]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:11.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:11 smithi086 ceph-mon[37128]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- 172.21.15.60:0/3202265478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6098108b20 msgr2=0x7f609810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 --2- 172.21.15.60:0/3202265478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6098108b20 0x7f609810f320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6090009ca0 tx=0x7f609002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- 172.21.15.60:0/3202265478 shutdown_connections 2024-07-24T21:46:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 --2- 172.21.15.60:0/3202265478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6098108b20 0x7f609810f320 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 --2- 172.21.15.60:0/3202265478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60981081a0 0x7f60981085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 --2- 172.21.15.60:0/3202265478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 0x7f6098102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- 172.21.15.60:0/3202265478 >> 172.21.15.60:0/3202265478 conn(0x7f60980fe200 msgr2=0x7f6098100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- 172.21.15.60:0/3202265478 shutdown_connections 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- 172.21.15.60:0/3202265478 wait complete. 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 Processor -- start 2024-07-24T21:46:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.559+0000 7f60a06f7640 1 -- start start 2024-07-24T21:46:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 0x7f609819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6098108b20 0x7f60981a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60981127a0 con 0x7f60981081a0 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112620 con 0x7f6098108b20 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f60a06f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112920 con 0x7f6098102790 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44098/0 (socket says 172.21.15.60:44098) 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 -- 172.21.15.60:0/4180213387 learned_addr learned my addr 172.21.15.60:0/4180213387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609e493640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 0x7f609819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 -- 172.21.15.60:0/4180213387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 msgr2=0x7f609819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609ec94640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6098108b20 0x7f60981a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 0x7f609819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 -- 172.21.15.60:0/4180213387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6098108b20 msgr2=0x7f60981a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6098108b20 0x7f60981a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 -- 172.21.15.60:0/4180213387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60981a0ea0 con 0x7f60981081a0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.560+0000 7f609dc92640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f608800c920 tx=0x7f608800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.561+0000 7f608f7fe640 1 -- 172.21.15.60:0/4180213387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088007bb0 con 0x7f60981081a0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.561+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60981a1190 con 0x7f60981081a0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.561+0000 7f608f7fe640 1 -- 172.21.15.60:0/4180213387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6088007d50 con 0x7f60981081a0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.561+0000 7f608f7fe640 1 -- 172.21.15.60:0/4180213387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088018730 con 0x7f60981081a0 2024-07-24T21:46:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.561+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60981a78d0 con 0x7f60981081a0 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.563+0000 7f60a2aa8740 1 Processor -- start 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 -- start start 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3a620 0x558b24c3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c48840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b24bb0740 con 0x558b24c3bb80 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b24b9fc40 con 0x558b24c46450 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b24ba3c80 con 0x558b24c3a620 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44106/0 (socket says 172.21.15.60:44106) 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 -- 172.21.15.60:0/2234297268 learned_addr learned my addr 172.21.15.60:0/2234297268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f608f7fe640 1 -- 172.21.15.60:0/4180213387 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f608800f450 con 0x7f60981081a0 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609e493640 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3a620 0x558b24c3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 -- 172.21.15.60:0/2234297268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3a620 msgr2=0x558b24c3aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3a620 0x558b24c3aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 -- 172.21.15.60:0/2234297268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 msgr2=0x558b24c48840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c48840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.564+0000 7f609ec94640 1 -- 172.21.15.60:0/2234297268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b24b291f0 con 0x558b24c3bb80 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f609ec94640 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6090009930 tx=0x7f6090009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de40b7fc5cded8f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f608cff9640 1 -- 172.21.15.60:0/2234297268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090031df0 con 0x558b24c3bb80 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f608cff9640 1 -- 172.21.15.60:0/2234297268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090036070 con 0x558b24c3bb80 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f608cff9640 1 -- 172.21.15.60:0/2234297268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090004260 con 0x558b24c3bb80 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f608f7fe640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 0x7f60840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f609e493640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 0x7f60840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.565+0000 7f608f7fe640 1 -- 172.21.15.60:0/4180213387 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f60880e3df0 con 0x7f60981081a0 2024-07-24T21:46:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- 172.21.15.60:0/2234297268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 msgr2=0x558b24c3a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6090009930 tx=0x7f6090009960 comp rx=0 tx=0).stop 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- 172.21.15.60:0/2234297268 shutdown_connections 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c48840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3a620 0x558b24c3aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/2234297268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c3a0e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- 172.21.15.60:0/2234297268 >> 172.21.15.60:0/2234297268 conn(0x558b24bd86c0 msgr2=0x558b24c37f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- 172.21.15.60:0/2234297268 shutdown_connections 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f609e493640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 0x7f60840b60e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x558b24c3d260 tx=0x7f6094009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- 172.21.15.60:0/2234297268 wait complete. 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 -- start start 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 Processor -- start 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.566+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b24a035f0 con 0x558b24c3a620 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b24b424c0 con 0x558b24c46450 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b24c49210 con 0x558b24c3bb80 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f60a2aa8740 10 client.0 _ll_register_callbacks cb 0x558b24b980c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44108/0 (socket says 172.21.15.60:44108) 2024-07-24T21:46:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 -- 172.21.15.60:0/3195773481 learned_addr learned my addr 172.21.15.60:0/3195773481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609dc92640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609ec94640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 -- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 msgr2=0x558b24c43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 -- 172.21.15.60:0/3195773481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 msgr2=0x558b24c43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 -- 172.21.15.60:0/3195773481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b24b35550 con 0x558b24c3a620 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609dc92640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609ec94640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f609e493640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6094009560 tx=0x7f60940047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deb72b0c1c13183 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f6081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094025070 con 0x558b24c3a620 2024-07-24T21:46:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6094013670 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6090036070 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.567+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6094013870 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 10 client.15279 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b24900540 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090036070 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6094020070 con 0x558b24c3a620 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 10 client.15279 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 20 client.15279 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24c56f40 0x558b24c583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 4 client.15279 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b24bb2690 con 0x558b24c56f40 2024-07-24T21:46:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.568+0000 7f609dc92640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24c56f40 0x558b24c583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.569+0000 7f609dc92640 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24c56f40 0x558b24c583b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1623dcfec074b713 server_cookie=b021e42e419282cd in_seq=0 out_seq=0 2024-07-24T21:46:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.569+0000 7f6081ffb640 10 client.15279 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.569+0000 7f6081ffb640 1 -- 172.21.15.60:0/3195773481 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b24bb2690 con 0x558b24c56f40 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.569+0000 7f6081ffb640 10 client.15279 handle_command_reply: tid=0 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.569+0000 7f60a2aa8740 1 client.15279 shutdown 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 msgr2=0x558b24c45c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6094009560 tx=0x7f60940047d0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 shutdown_connections 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24c56f40 0x558b24c583b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c46450 0x558b24c43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/3195773481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3a620 0x558b24c45c30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 >> 172.21.15.60:0/3195773481 conn(0x558b24bd86c0 msgr2=0x558b24c3dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f6081ffb640 0 client.15279 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 shutdown_connections 2024-07-24T21:46:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 1 -- 172.21.15.60:0/3195773481 wait complete. 2024-07-24T21:46:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.570+0000 7f60a2aa8740 20 client.15279 trim_cache size 0 max 16384 2024-07-24T21:46:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.588+0000 7f60a2aa8740 1 Processor -- start 2024-07-24T21:46:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 -- start start 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c39dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b24c566f0 con 0x558b24c46450 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b24c56dc0 con 0x558b24c3a620 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b24c57490 con 0x558b24c3bb80 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44114/0 (socket says 172.21.15.60:44114) 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 -- 172.21.15.60:0/1612602615 learned_addr learned my addr 172.21.15.60:0/1612602615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609e493640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c39dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609dc92640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 -- 172.21.15.60:0/1612602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 msgr2=0x558b24c33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 -- 172.21.15.60:0/1612602615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 msgr2=0x558b24c39dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c39dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 -- 172.21.15.60:0/1612602615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b24c41620 con 0x558b24c46450 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609dc92640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.589+0000 7f609ec94640 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6090044000 tx=0x7f6090005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be74bbe2ca027e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f6083fff640 1 -- 172.21.15.60:0/1612602615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090031450 con 0x558b24c46450 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f6083fff640 1 -- 172.21.15.60:0/1612602615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090036070 con 0x558b24c46450 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f6083fff640 1 -- 172.21.15.60:0/1612602615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090004510 con 0x558b24c46450 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 -- 172.21.15.60:0/1612602615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 msgr2=0x558b24c43090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6090044000 tx=0x7f6090005bf0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 -- 172.21.15.60:0/1612602615 shutdown_connections 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c46450 0x558b24c43090 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c3bb80 0x558b24c33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/1612602615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c39dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 -- 172.21.15.60:0/1612602615 >> 172.21.15.60:0/1612602615 conn(0x558b24bd86c0 msgr2=0x558b24c3e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 -- 172.21.15.60:0/1612602615 shutdown_connections 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.590+0000 7f60a2aa8740 1 -- 172.21.15.60:0/1612602615 wait complete. 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 -- start start 2024-07-24T21:46:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 Processor -- start 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 0x558b24c36f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b24c58230 con 0x558b24c3bb80 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b24c57b60 con 0x558b24c3a620 2024-07-24T21:46:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b24c58900 con 0x558b24c46450 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f60a2aa8740 10 client.0 _ll_register_callbacks cb 0x7f6094009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44126/0 (socket says 172.21.15.60:44126) 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 -- 172.21.15.60:0/457010629 learned_addr learned my addr 172.21.15.60:0/457010629 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609e493640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609ec94640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 0x558b24c36f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 -- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 msgr2=0x558b24c36f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 0x558b24c36f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 -- 172.21.15.60:0/457010629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 msgr2=0x558b24c60900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c60900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609dc92640 1 -- 172.21.15.60:0/457010629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b24c37780 con 0x558b24c3bb80 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.591+0000 7f609ec94640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 0x558b24c36f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f609dc92640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f60880af190 tx=0x7f60880af040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0c4276578fa037 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088084e60 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f608800f990 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6090036070 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f60880aac20 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 10 client.15285 fetch_fsmap learned FSMap version 25 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b24c5e2b0 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090036070 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60880b38d0 con 0x558b24c3bb80 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 10 client.15285 fetch_fsmap finished waiting for FSMap version 25 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 20 client.15285 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24cc5e10 0x558b24cc4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 4 client.15285 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.592+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "status"}) -- 0x558b24bae130 con 0x558b24cc5e10 2024-07-24T21:46:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f609e493640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24cc5e10 0x558b24cc4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f609e493640 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24cc5e10 0x558b24cc4b20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b41e876fb0471c9 server_cookie=60d6e5da75beaa97 in_seq=0 out_seq=0 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f6081ffb640 10 client.15285 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f6081ffb640 1 -- 172.21.15.60:0/457010629 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558b24bae130 con 0x558b24cc5e10 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f6081ffb640 10 client.15285 handle_command_reply: tid=0 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 client.15285 shutdown 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 msgr2=0x558b24c60ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f60880af190 tx=0x7f60880af040 comp rx=0 tx=0).stop 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 shutdown_connections 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b24cc5e10 0x558b24cc4b20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b24c46450 0x558b24c36f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b24c3bb80 0x558b24c60ea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 --2- 172.21.15.60:0/457010629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b24c3a620 0x558b24c60900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 >> 172.21.15.60:0/457010629 conn(0x558b24bd86c0 msgr2=0x558b24c43070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.593+0000 7f6081ffb640 0 client.15285 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 shutdown_connections 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a2aa8740 1 -- 172.21.15.60:0/457010629 wait complete. 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a2aa8740 20 client.15285 trim_cache size 0 max 16384 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24610, 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:46:11.601 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 177.303410128, 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 25, 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 202.36589021500001 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 msgr2=0x7f60840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 0x7f60840b60e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x558b24c3d260 tx=0x7f6094009110 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 msgr2=0x7f609819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f608800c920 tx=0x7f608800cd20 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 shutdown_connections 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60840b3c20 0x7f60840b60e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6098108b20 0x7f60981a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60981081a0 0x7f609819b330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 --2- 172.21.15.60:0/4180213387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6098102790 0x7f609819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 >> 172.21.15.60:0/4180213387 conn(0x7f60980fe200 msgr2=0x7f6098105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 shutdown_connections 2024-07-24T21:46:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:11.594+0000 7f60a06f7640 1 -- 172.21.15.60:0/4180213387 wait complete. 2024-07-24T21:46:12.385 INFO:teuthology.run_tasks:Running task sequential... 2024-07-24T21:46:12.397 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-24T21:46:12.397 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-24T21:46:12.397 INFO:teuthology.task.print:Enabling mgr modules 2024-07-24T21:46:12.398 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-07-24T21:46:12.398 INFO:teuthology.task.exec:Executing custom commands... 2024-07-24T21:46:12.399 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:46:12.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 -- 172.21.15.60:0/3001302898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a81015d0 msgr2=0x7fe0a81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 --2- 172.21.15.60:0/3001302898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a81015d0 0x7fe0a81019b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe098009900 tx=0x7fe09802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 -- 172.21.15.60:0/3001302898 shutdown_connections 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 --2- 172.21.15.60:0/3001302898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 0x7fe0a810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 --2- 172.21.15.60:0/3001302898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 --2- 172.21.15.60:0/3001302898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a81015d0 0x7fe0a81019b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 -- 172.21.15.60:0/3001302898 >> 172.21.15.60:0/3001302898 conn(0x7fe0a80fd030 msgr2=0x7fe0a80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 -- 172.21.15.60:0/3001302898 shutdown_connections 2024-07-24T21:46:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.510+0000 7fe0ae855640 1 -- 172.21.15.60:0/3001302898 wait complete. 2024-07-24T21:46:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 Processor -- start 2024-07-24T21:46:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 -- start start 2024-07-24T21:46:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a81015d0 0x7fe0a8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 0x7fe0a819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81115a0 con 0x7fe0a8106fd0 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8111420 con 0x7fe0a81015d0 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0ae855640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8111720 con 0x7fe0a8107950 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44154/0 (socket says 172.21.15.60:44154) 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.511+0000 7fe0a77fe640 1 -- 172.21.15.60:0/2107456665 learned_addr learned my addr 172.21.15.60:0/2107456665 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a7fff640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a81015d0 0x7fe0a8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0acdf2640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 0x7fe0a819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 -- 172.21.15.60:0/2107456665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 msgr2=0x7fe0a819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 0x7fe0a819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 -- 172.21.15.60:0/2107456665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a81015d0 msgr2=0x7fe0a8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a81015d0 0x7fe0a8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 -- 172.21.15.60:0/2107456665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a819ec70 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a77fe640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d950 tx=0x7fe09c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c017070 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09c009c30 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a819ef60 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c00f620 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.512+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a81a5620 con 0x7fe0a8106fd0 2024-07-24T21:46:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.513+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a81019b0 con 0x7fe0a8106fd0 2024-07-24T21:46:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.516+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fe09c010460 con 0x7fe0a8106fd0 2024-07-24T21:46:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.516+0000 7fe0a57fa640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 0x7fe0900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.516+0000 7fe0a7fff640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 0x7fe0900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.516+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe09c0e38d0 con 0x7fe0a8106fd0 2024-07-24T21:46:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.517+0000 7fe0a7fff640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 0x7fe0900b60e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe098031440 tx=0x7fe098005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.519+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe09c009de0 con 0x7fe0a8106fd0 2024-07-24T21:46:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:12.769+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe0a802e000 con 0x7fe0a8106fd0 2024-07-24T21:46:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.225+0000 7fe0a57fa640 1 -- 172.21.15.60:0/2107456665 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7fe09c0b30d0 con 0x7fe0a8106fd0 2024-07-24T21:46:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:module 'snap_schedule' is already enabled 2024-07-24T21:46:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 msgr2=0x7fe0900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 0x7fe0900b60e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe098031440 tx=0x7fe098005b50 comp rx=0 tx=0).stop 2024-07-24T21:46:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 msgr2=0x7fe0a8199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d950 tx=0x7fe09c00de10 comp rx=0 tx=0).stop 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 shutdown_connections 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe0900b3c20 0x7fe0900b60e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0a8107950 0x7fe0a819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0a8106fd0 0x7fe0a8199110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 --2- 172.21.15.60:0/2107456665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0a81015d0 0x7fe0a8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 >> 172.21.15.60:0/2107456665 conn(0x7fe0a80fd030 msgr2=0x7fe0a806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 shutdown_connections 2024-07-24T21:46:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:13.228+0000 7fe0ae855640 1 -- 172.21.15.60:0/2107456665 wait complete. 2024-07-24T21:46:13.248 INFO:teuthology.run_tasks:Running task kclient... 2024-07-24T21:46:13.276 INFO:tasks.kclient:Mounting kernel clients... 2024-07-24T21:46:13.276 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-24T21:46:13.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:13.276 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-07-24T21:46:13.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:13.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-07-24T21:46:13.363 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-07-24T21:46:13.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T21:46:13.365 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-24T21:46:13.365 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-24T21:46:13.365 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:13.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs ls 2024-07-24T21:46:13.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:13 smithi086 ceph-mon[37128]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:13.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:46:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:13 smithi060 ceph-mon[33000]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:46:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:13 smithi123 ceph-mon[37143]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T21:46:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-24T21:46:14.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:14 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:46:14.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:14 smithi060 ceph-mon[33000]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-07-24T21:46:14.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:14 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:46:14.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:14 smithi086 ceph-mon[37128]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-07-24T21:46:14.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:14 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2107456665' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-24T21:46:14.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:14 smithi123 ceph-mon[37143]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 -- 172.21.15.60:0/666199833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c102790 msgr2=0x7f659c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 --2- 172.21.15.60:0/666199833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c102790 0x7f659c102b70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 -- 172.21.15.60:0/666199833 shutdown_connections 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 --2- 172.21.15.60:0/666199833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c108b20 0x7f659c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 --2- 172.21.15.60:0/666199833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c1081a0 0x7f659c1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 --2- 172.21.15.60:0/666199833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c102790 0x7f659c102b70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 -- 172.21.15.60:0/666199833 >> 172.21.15.60:0/666199833 conn(0x7f659c0fe200 msgr2=0x7f659c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 -- 172.21.15.60:0/666199833 shutdown_connections 2024-07-24T21:46:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.306+0000 7f65a29f3640 1 -- 172.21.15.60:0/666199833 wait complete. 2024-07-24T21:46:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.307+0000 7f65a29f3640 1 Processor -- start 2024-07-24T21:46:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.307+0000 7f65a29f3640 1 -- start start 2024-07-24T21:46:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c102790 0x7f659c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 0x7f659c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 0x7f659c075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f659c112720 con 0x7f659c108b20 2024-07-24T21:46:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1125a0 con 0x7f659c102790 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a29f3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1128a0 con 0x7f659c1081a0 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a0f90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 0x7f659c075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 0x7f659c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a0f90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 0x7f659c075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44166/0 (socket says 172.21.15.60:44166) 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f65a0f90640 1 -- 172.21.15.60:0/1941731331 learned_addr learned my addr 172.21.15.60:0/1941731331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659bfff640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c102790 0x7f659c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 -- 172.21.15.60:0/1941731331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c102790 msgr2=0x7f659c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c102790 0x7f659c074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 -- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 msgr2=0x7f659c075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 0x7f659c075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 -- 172.21.15.60:0/1941731331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1a7010 con 0x7f659c1081a0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.308+0000 7f659b7fe640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 0x7f659c0753d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f659000e9f0 tx=0x7f659000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.309+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659000ccc0 con 0x7f659c1081a0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.309+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a7300 con 0x7f659c1081a0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.309+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65900108e0 con 0x7f659c1081a0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.309+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590018880 con 0x7f659c1081a0 2024-07-24T21:46:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.309+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a7890 con 0x7f659c1081a0 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.310+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c102b70 con 0x7f659c1081a0 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.311+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f6590010430 con 0x7f659c1081a0 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.312+0000 7f65997fa640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 0x7f65840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.312+0000 7f659bfff640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 0x7f65840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.312+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f65900e40f0 con 0x7f659c1081a0 2024-07-24T21:46:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.312+0000 7f659bfff640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 0x7f65840b60e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f658c031440 tx=0x7f658c005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.316+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f65900aafd0 con 0x7f659c1081a0 2024-07-24T21:46:15.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:15 smithi086 ceph-mon[37128]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.551+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f659c02e000 con 0x7f659c1081a0 2024-07-24T21:46:15.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:15 smithi060 ceph-mon[33000]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.551+0000 7f65997fa640 1 -- 172.21.15.60:0/1941731331 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f65900af520 con 0x7f659c1081a0 2024-07-24T21:46:15.553 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-24T21:46:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 msgr2=0x7f65840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 0x7f65840b60e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f658c031440 tx=0x7f658c005b50 comp rx=0 tx=0).stop 2024-07-24T21:46:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 msgr2=0x7f659c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 0x7f659c0753d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f659000e9f0 tx=0x7f659000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 shutdown_connections 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3c20 0x7f65840b60e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659c108b20 0x7f659c075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f659c1081a0 0x7f659c0753d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 --2- 172.21.15.60:0/1941731331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659c102790 0x7f659c074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 >> 172.21.15.60:0/1941731331 conn(0x7f659c0fe200 msgr2=0x7f659c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 shutdown_connections 2024-07-24T21:46:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:15.554+0000 7f65a29f3640 1 -- 172.21.15.60:0/1941731331 wait complete. 2024-07-24T21:46:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:15 smithi123 ceph-mon[37143]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-24T21:46:16.355 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-24T21:46:16.356 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-24T21:46:16.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:16.357 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-07-24T21:46:16.374 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42020sec preferred_lft 42020sec 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-07-24T21:46:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-07-24T21:46:16.376 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-24T21:46:16.376 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-07-24T21:46:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:16.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:16 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1941731331' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-24T21:46:16.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:16 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1941731331' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-24T21:46:16.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:16.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:16.612 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-24T21:46:16.642 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-07-24T21:46:16.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:16.644 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-07-24T21:46:16.712 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-07-24T21:46:16.712 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-24T21:46:16.712 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-24T21:46:16.712 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-24T21:46:16.712 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-24T21:46:16.713 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-07-24T21:46:16.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:16 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1941731331' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-24T21:46:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:17.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:17.071 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-07-24T21:46:17.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:17.089 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-07-24T21:46:17.145 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-24T21:46:17.145 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-07-24T21:46:17.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-24T21:46:17.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-24T21:46:17.145 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-07-24T21:46:17.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:17.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:17.275 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-24T21:46:17.275 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-24T21:46:17.276 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-07-24T21:46:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:17.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:17.421 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-24T21:46:17.421 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-07-24T21:46:17.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-07-24T21:46:17.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-07-24T21:46:17.422 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-07-24T21:46:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:17.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-24T21:46:17.507 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:17.539 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-24T21:46:17.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-24T21:46:17.540 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.595 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-24T21:46:17.596 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-24T21:46:17.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b 2024-07-24T21:46:17.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:17.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-24T21:46:17.694 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-24T21:46:17.694 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-07-24T21:46:17.694 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.86:6789,172.21.15.123:6789" 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.86:6789,172.21.15.123:6789:/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b 2024-07-24T21:46:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=646042b4-4a04-11ef-bcad-c7b262605968" will pass to kernel 2024-07-24T21:46:17.830 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-24T21:46:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-07-24T21:46:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-24T21:46:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-07-24T21:46:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-24T21:46:17.832 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-24T21:46:17.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T21:46:17.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:17.859 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-24T21:46:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-24T21:46:17.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:17.959 INFO:teuthology.orchestra.run.smithi060.stdout:client15300 2024-07-24T21:46:17.959 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:46:17.959 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-24T21:46:17.977 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-24T21:46:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-24T21:46:18.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:18.077 INFO:teuthology.orchestra.run.smithi060.stdout:client15300 2024-07-24T21:46:18.077 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-24T21:46:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-24T21:46:18.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:client15300 2024-07-24T21:46:18.136 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/646042b4-4a04-11ef-bcad-c7b262605968.client15300/status') 2024-07-24T21:46:18.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-24T21:46:18.196 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.15300 (3)192.168.0.1:0/596551106 2024-07-24T21:46:18.196 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-07-24T21:46:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-07-24T21:46:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-07-24T21:46:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:69 bytes copied, 2.4459e-05 s, 2.8 MB/s 2024-07-24T21:46:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-24T21:46:18.204 INFO:teuthology.run_tasks:Running task exec... 2024-07-24T21:46:18.215 INFO:teuthology.task.exec:Executing custom commands... 2024-07-24T21:46:18.216 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:46:18.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-07-24T21:46:18.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 -- 172.21.15.60:0/3964493472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 msgr2=0x7f20041073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:18.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 --2- 172.21.15.60:0/3964493472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f20041073f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009900 tx=0x7f1ff402a410 comp rx=0 tx=0).stop 2024-07-24T21:46:18.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 -- 172.21.15.60:0/3964493472 shutdown_connections 2024-07-24T21:46:18.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 --2- 172.21.15.60:0/3964493472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2004107930 0x7f200410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 --2- 172.21.15.60:0/3964493472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f20041073f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 --2- 172.21.15.60:0/3964493472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20041015b0 0x7f2004101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 -- 172.21.15.60:0/3964493472 >> 172.21.15.60:0/3964493472 conn(0x7f20040fcfb0 msgr2=0x7f20040ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 -- 172.21.15.60:0/3964493472 shutdown_connections 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.322+0000 7f200c4be640 1 -- 172.21.15.60:0/3964493472 wait complete. 2024-07-24T21:46:18.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 Processor -- start 2024-07-24T21:46:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 -- start start 2024-07-24T21:46:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 0x7f2004198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2004107930 0x7f200419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2004111550 con 0x7f2004106fb0 2024-07-24T21:46:18.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20041113d0 con 0x7f20041015b0 2024-07-24T21:46:18.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200c4be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20041116d0 con 0x7f2004107930 2024-07-24T21:46:18.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:18.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34302/0 (socket says 172.21.15.60:34302) 2024-07-24T21:46:18.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 -- 172.21.15.60:0/2541336611 learned_addr learned my addr 172.21.15.60:0/2541336611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200aa5b640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2004107930 0x7f200419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200a25a640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 0x7f2004198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 -- 172.21.15.60:0/2541336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2004107930 msgr2=0x7f200419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2004107930 0x7f200419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 -- 172.21.15.60:0/2541336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 msgr2=0x7f2004198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:18.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 0x7f2004198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f2009a59640 1 -- 172.21.15.60:0/2541336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200419ec60 con 0x7f2004106fb0 2024-07-24T21:46:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.323+0000 7f200a25a640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 0x7f2004198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f2009a59640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4005940 tx=0x7f1ff40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff403f030 con 0x7f2004106fb0 2024-07-24T21:46:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff4003cb0 con 0x7f2004106fb0 2024-07-24T21:46:18.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200419eef0 con 0x7f2004106fb0 2024-07-24T21:46:18.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4035070 con 0x7f2004106fb0 2024-07-24T21:46:18.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.324+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a5610 con 0x7f2004106fb0 2024-07-24T21:46:18.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.325+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2004101990 con 0x7f2004106fb0 2024-07-24T21:46:18.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.326+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f1ff4042030 con 0x7f2004106fb0 2024-07-24T21:46:18.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.327+0000 7f1ffb7fe640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 0x7f1ff00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:18.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.327+0000 7f200a25a640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 0x7f1ff00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:18.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.327+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1ff41048f0 con 0x7f2004106fb0 2024-07-24T21:46:18.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.327+0000 7f200a25a640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 0x7f1ff00b60e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20000058c0 tx=0x7f2000005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:18.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.330+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1ff40cbe20 con 0x7f2004106fb0 2024-07-24T21:46:18.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:46:18.572 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:46:18.572 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:18.570+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f20041a5940 con 0x7f2004106fb0 2024-07-24T21:46:18.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:46:18.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:46:18.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:18.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:46:18.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:46:18.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:46:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.401+0000 7f1ffb7fe640 1 -- 172.21.15.60:0/2541336611 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f1ff40cb720 con 0x7f2004106fb0 2024-07-24T21:46:19.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 msgr2=0x7f1ff00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 0x7f1ff00b60e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20000058c0 tx=0x7f2000005850 comp rx=0 tx=0).stop 2024-07-24T21:46:19.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 msgr2=0x7f2004199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4005940 tx=0x7f1ff40042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 shutdown_connections 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ff00b3c20 0x7f1ff00b60e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2004107930 0x7f200419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2004106fb0 0x7f2004199100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 --2- 172.21.15.60:0/2541336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20041015b0 0x7f2004198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 >> 172.21.15.60:0/2541336611 conn(0x7f20040fcfb0 msgr2=0x7f200406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 shutdown_connections 2024-07-24T21:46:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.404+0000 7f200c4be640 1 -- 172.21.15.60:0/2541336611 wait complete. 2024-07-24T21:46:19.419 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-24T21:46:19.432 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-24T21:46:19.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-24T21:46:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 -- 172.21.15.60:0/4030910102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 msgr2=0x7f8bc810c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 --2- 172.21.15.60:0/4030910102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc810c440 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb002a410 comp rx=0 tx=0).stop 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 -- 172.21.15.60:0/4030910102 shutdown_connections 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 --2- 172.21.15.60:0/4030910102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc810ca10 0x7f8bc810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 --2- 172.21.15.60:0/4030910102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc810c440 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 --2- 172.21.15.60:0/4030910102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc8103460 0x7f8bc8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 -- 172.21.15.60:0/4030910102 >> 172.21.15.60:0/4030910102 conn(0x7f8bc80fd030 msgr2=0x7f8bc80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 -- 172.21.15.60:0/4030910102 shutdown_connections 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.576+0000 7f8bcdded640 1 -- 172.21.15.60:0/4030910102 wait complete. 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 Processor -- start 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 -- start start 2024-07-24T21:46:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 0x7f8bc81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 0x7f8bc8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8112640 con 0x7f8bc8103e60 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81124c0 con 0x7f8bc8103460 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bcdded640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81127c0 con 0x7f8bc810ca10 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34308/0 (socket says 172.21.15.60:34308) 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.577+0000 7f8bbffff640 1 -- 172.21.15.60:0/1945917411 learned_addr learned my addr 172.21.15.60:0/1945917411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc77fe640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 0x7f8bc81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 -- 172.21.15.60:0/1945917411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 msgr2=0x7f8bc8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc7fff640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 0x7f8bc8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 0x7f8bc8101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 -- 172.21.15.60:0/1945917411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 msgr2=0x7f8bc81012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 0x7f8bc81012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 -- 172.21.15.60:0/1945917411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a7050 con 0x7f8bc8103e60 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc77fe640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 0x7f8bc81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc7fff640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 0x7f8bc8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bbffff640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0005940 tx=0x7f8bb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb003f030 con 0x7f8bc8103e60 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0003cb0 con 0x7f8bc8103e60 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0035070 con 0x7f8bc8103e60 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a72e0 con 0x7f8bc8103e60 2024-07-24T21:46:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.578+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a7820 con 0x7f8bc8103e60 2024-07-24T21:46:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.579+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc8103840 con 0x7f8bc8103e60 2024-07-24T21:46:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.581+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f8bb0042030 con 0x7f8bc8103e60 2024-07-24T21:46:19.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.581+0000 7f8bc57fa640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 0x7f8bb40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.581+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8bb01048e0 con 0x7f8bc8103e60 2024-07-24T21:46:19.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.581+0000 7f8bc77fe640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 0x7f8bb40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.582+0000 7f8bc77fe640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 0x7f8bb40b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80058c0 tx=0x7f8bb8005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:19.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.584+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8bb00cbe10 con 0x7f8bc8103e60 2024-07-24T21:46:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:19 smithi123 ceph-mon[37143]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:19 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-24T21:46:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.792+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bc81025e0 con 0x7f8bc8103e60 2024-07-24T21:46:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:19 smithi086 ceph-mon[37128]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:19 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-24T21:46:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.794+0000 7f8bc57fa640 1 -- 172.21.15.60:0/1945917411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23071 (secure 0 0 0) 0x7f8bb00d40e0 con 0x7f8bc8103e60 2024-07-24T21:46:19.795 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:19.795 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:44:37.734936+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 msgr2=0x7f8bb40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 0x7f8bb40b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80058c0 tx=0x7f8bb8005850 comp rx=0 tx=0).stop 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 msgr2=0x7f8bc8101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0005940 tx=0x7f8bb00042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 shutdown_connections 2024-07-24T21:46:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8bb40b3ca0 0x7f8bb40b6160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc810ca10 0x7f8bc8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8103e60 0x7f8bc8101820 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 --2- 172.21.15.60:0/1945917411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc8103460 0x7f8bc81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 >> 172.21.15.60:0/1945917411 conn(0x7f8bc80fd030 msgr2=0x7f8bc80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 shutdown_connections 2024-07-24T21:46:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.796+0000 7f8bcdded640 1 -- 172.21.15.60:0/1945917411 wait complete. 2024-07-24T21:46:19.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:19 smithi060 ceph-mon[33000]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-24T21:46:19.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:19 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-24T21:46:19.809 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-24T21:38:46.032062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-24T21:43:07.304504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-24T21:43:10.965950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-24T21:46:19.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- 172.21.15.60:0/3710726311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 msgr2=0x7f35e0102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 --2- 172.21.15.60:0/3710726311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0102aa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d002a3f0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- 172.21.15.60:0/3710726311 shutdown_connections 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 --2- 172.21.15.60:0/3710726311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 0x7f35e010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 --2- 172.21.15.60:0/3710726311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 0x7f35e0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 --2- 172.21.15.60:0/3710726311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0102aa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- 172.21.15.60:0/3710726311 >> 172.21.15.60:0/3710726311 conn(0x7f35e00fe140 msgr2=0x7f35e0100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- 172.21.15.60:0/3710726311 shutdown_connections 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- 172.21.15.60:0/3710726311 wait complete. 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 Processor -- start 2024-07-24T21:46:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.917+0000 7f35e6a16640 1 -- start start 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 0x7f35e0199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 0x7f35e019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0112670 con 0x7f35e01026c0 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01124f0 con 0x7f35e01080c0 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01127f0 con 0x7f35e0108a40 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34314/0 (socket says 172.21.15.60:34314) 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 -- 172.21.15.60:0/1526983111 learned_addr learned my addr 172.21.15.60:0/1526983111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35d7fff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 0x7f35e0199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e4fb3640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 0x7f35e019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 -- 172.21.15.60:0/1526983111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 msgr2=0x7f35e019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 0x7f35e019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 -- 172.21.15.60:0/1526983111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 msgr2=0x7f35e0199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 0x7f35e0199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 -- 172.21.15.60:0/1526983111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e019ebb0 con 0x7f35e01026c0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35dffff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35d0006010 tx=0x7f35d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d003f030 con 0x7f35e01026c0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d0003800 con 0x7f35e01026c0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.918+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e019ee40 con 0x7f35e01026c0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.919+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0035070 con 0x7f35e01026c0 2024-07-24T21:46:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.919+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01a5560 con 0x7f35e01026c0 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.919+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0102aa0 con 0x7f35e01026c0 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.921+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f35d0003cb0 con 0x7f35e01026c0 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.922+0000 7f35ddffb640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 0x7f35cc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.922+0000 7f35d7fff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 0x7f35cc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.922+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f35d01049c0 con 0x7f35e01026c0 2024-07-24T21:46:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.922+0000 7f35d7fff640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 0x7f35cc0b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35c8005e10 tx=0x7f35c8005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:19.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:19.924+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f35d00cbef0 con 0x7f35e01026c0 2024-07-24T21:46:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.117+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35e01a5840 con 0x7f35e01026c0 2024-07-24T21:46:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.118+0000 7f35ddffb640 1 -- 172.21.15.60:0/1526983111 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f35d00d41c0 con 0x7f35e01026c0 2024-07-24T21:46:20.119 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 msgr2=0x7f35cc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 0x7f35cc0b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35c8005e10 tx=0x7f35c8005da0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 msgr2=0x7f35e0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35d0006010 tx=0x7f35d00042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 shutdown_connections 2024-07-24T21:46:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f35cc0b3c20 0x7f35cc0b60e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35e0108a40 0x7f35e019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e01080c0 0x7f35e0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 --2- 172.21.15.60:0/1526983111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35e01026c0 0x7f35e0198b10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 >> 172.21.15.60:0/1526983111 conn(0x7f35e00fe140 msgr2=0x7f35e00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 shutdown_connections 2024-07-24T21:46:20.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.120+0000 7f35e6a16640 1 -- 172.21.15.60:0/1526983111 wait complete. 2024-07-24T21:46:20.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- 172.21.15.60:0/2171383024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c41078a0 msgr2=0x7f12c410e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/2171383024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c41078a0 0x7f12c410e0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009900 tx=0x7f12bc02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- 172.21.15.60:0/2171383024 shutdown_connections 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/2171383024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c41078a0 0x7f12c410e0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/2171383024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c4106f20 0x7f12c4107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/2171383024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 0x7f12c4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- 172.21.15.60:0/2171383024 >> 172.21.15.60:0/2171383024 conn(0x7f12c4077030 msgr2=0x7f12c4077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- 172.21.15.60:0/2171383024 shutdown_connections 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- 172.21.15.60:0/2171383024 wait complete. 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 Processor -- start 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 -- start start 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 0x7f12c41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.239+0000 7f12cb6c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c41078a0 0x7f12c419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12c4111530 con 0x7f12c4106f20 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41113b0 con 0x7f12c41019d0 2024-07-24T21:46:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41116b0 con 0x7f12c41078a0 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34326/0 (socket says 172.21.15.60:34326) 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 -- 172.21.15.60:0/1573009370 learned_addr learned my addr 172.21.15.60:0/1573009370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c9462640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 0x7f12c41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c9c63640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c41078a0 0x7f12c419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 -- 172.21.15.60:0/1573009370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c41078a0 msgr2=0x7f12c419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c41078a0 0x7f12c419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 -- 172.21.15.60:0/1573009370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 msgr2=0x7f12c41999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 0x7f12c41999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 -- 172.21.15.60:0/1573009370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c41a4de0 con 0x7f12c4106f20 2024-07-24T21:46:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c8c61640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8017030 con 0x7f12c4106f20 2024-07-24T21:46:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b8004020 con 0x7f12c4106f20 2024-07-24T21:46:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c41a50d0 con 0x7f12c4106f20 2024-07-24T21:46:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b80047a0 con 0x7f12c4106f20 2024-07-24T21:46:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.240+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c41a5670 con 0x7f12c4106f20 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.242+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c4101db0 con 0x7f12c4106f20 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.243+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f12b8009a30 con 0x7f12c4106f20 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.243+0000 7f12c27fc640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 0x7f12ac0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.244+0000 7f12c9462640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 0x7f12ac0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.244+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f12b80e36b0 con 0x7f12c4106f20 2024-07-24T21:46:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.244+0000 7f12c9462640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 0x7f12ac0b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005fd0 tx=0x7f12b4009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.247+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f12b80aabe0 con 0x7f12c4106f20 2024-07-24T21:46:20.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.440+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12c402e000 con 0x7f12c4106f20 2024-07-24T21:46:20.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.440+0000 7f12c27fc640 1 -- 172.21.15.60:0/1573009370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f12b80aa4e0 con 0x7f12c4106f20 2024-07-24T21:46:20.442 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.442+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 msgr2=0x7f12ac0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.442+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 0x7f12ac0b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005fd0 tx=0x7f12b4009680 comp rx=0 tx=0).stop 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.442+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 msgr2=0x7f12c4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.442+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800de10 comp rx=0 tx=0).stop 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 shutdown_connections 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12ac0b3ca0 0x7f12ac0b6160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c41078a0 0x7f12c419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106f20 0x7f12c4199f20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 --2- 172.21.15.60:0/1573009370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12c41019d0 0x7f12c41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 >> 172.21.15.60:0/1573009370 conn(0x7f12c4077030 msgr2=0x7f12c4104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 shutdown_connections 2024-07-24T21:46:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.443+0000 7f12cb6c6640 1 -- 172.21.15.60:0/1573009370 wait complete. 2024-07-24T21:46:20.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- 172.21.15.60:0/3019124317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee201057f0 msgr2=0x7fee2010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 --2- 172.21.15.60:0/3019124317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee201057f0 0x7fee2010f230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fee18009ca0 tx=0x7fee1802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- 172.21.15.60:0/3019124317 shutdown_connections 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 --2- 172.21.15.60:0/3019124317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee201057f0 0x7fee2010f230 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 --2- 172.21.15.60:0/3019124317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20104e50 0x7fee201052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 --2- 172.21.15.60:0/3019124317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20069fb0 0x7fee2006a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- 172.21.15.60:0/3019124317 >> 172.21.15.60:0/3019124317 conn(0x7fee20077030 msgr2=0x7fee20077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- 172.21.15.60:0/3019124317 shutdown_connections 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- 172.21.15.60:0/3019124317 wait complete. 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 Processor -- start 2024-07-24T21:46:20.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.562+0000 7fee25767640 1 -- start start 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 0x7fee20198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 0x7fee2019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112610 con 0x7fee20104e50 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112490 con 0x7fee201057f0 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112790 con 0x7fee20069fb0 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34358/0 (socket says 172.21.15.60:34358) 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 -- 172.21.15.60:0/4208133367 learned_addr learned my addr 172.21.15.60:0/4208133367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1effd640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 0x7fee20198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 0x7fee2019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 -- 172.21.15.60:0/4208133367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 msgr2=0x7fee20198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 0x7fee20198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 -- 172.21.15.60:0/4208133367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 msgr2=0x7fee2019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 0x7fee2019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 -- 172.21.15.60:0/4208133367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2019ebd0 con 0x7fee20104e50 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1effd640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 0x7fee20198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 0x7fee2019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee1e7fc640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fee1400e9f0 tx=0x7fee1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:20.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400cd80 con 0x7fee20104e50 2024-07-24T21:46:20.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140108e0 con 0x7fee20104e50 2024-07-24T21:46:20.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2019eec0 con 0x7fee20104e50 2024-07-24T21:46:20.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14018980 con 0x7fee20104e50 2024-07-24T21:46:20.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.563+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee201a5580 con 0x7fee20104e50 2024-07-24T21:46:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.564+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2006a390 con 0x7fee20104e50 2024-07-24T21:46:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.566+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fee14010430 con 0x7fee20104e50 2024-07-24T21:46:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.566+0000 7fee07fff640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 0x7fee080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.567+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fee140e46b0 con 0x7fee20104e50 2024-07-24T21:46:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.567+0000 7fee1effd640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 0x7fee080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.567+0000 7fee1effd640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 0x7fee080b60e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee100058c0 tx=0x7fee10005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.569+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fee140abbe0 con 0x7fee20104e50 2024-07-24T21:46:20.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-24T21:46:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:20 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:46:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1945917411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:20 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1526983111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:46:20.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.761+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee201a5980 con 0x7fee20104e50 2024-07-24T21:46:20.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.762+0000 7fee07fff640 1 -- 172.21.15.60:0/4208133367 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fee140ab4e0 con 0x7fee20104e50 2024-07-24T21:46:20.763 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 msgr2=0x7fee080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 0x7fee080b60e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee100058c0 tx=0x7fee10005850 comp rx=0 tx=0).stop 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 msgr2=0x7fee20199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fee1400e9f0 tx=0x7fee1400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 shutdown_connections 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3c20 0x7fee080b60e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee201057f0 0x7fee2019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20104e50 0x7fee20199070 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 --2- 172.21.15.60:0/4208133367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20069fb0 0x7fee20198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 >> 172.21.15.60:0/4208133367 conn(0x7fee20077030 msgr2=0x7fee20105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 shutdown_connections 2024-07-24T21:46:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:20.764+0000 7fee25767640 1 -- 172.21.15.60:0/4208133367 wait complete. 2024-07-24T21:46:20.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-24T21:46:20.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:20 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:46:20.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1945917411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:20.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:20 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1526983111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:46:20.776 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-24T21:46:20.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:46:20.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2541336611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-24T21:46:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:20 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:46:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1945917411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:20 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1526983111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.039 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:21.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:21 smithi060 ceph-mon[33000]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:21.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:21 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1573009370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:21 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4208133367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:21 smithi123 ceph-mon[37143]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:21 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1573009370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:21 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4208133367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:21 smithi086 ceph-mon[37128]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-24T21:46:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:21 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1573009370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T21:46:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:21 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4208133367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T21:46:22.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 -- 172.21.15.60:0/2274165558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 msgr2=0x7f1e88103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/2274165558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88103a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009900 tx=0x7f1e7802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 -- 172.21.15.60:0/2274165558 shutdown_connections 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/2274165558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e8810cac0 0x7f1e8810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/2274165558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e88103f80 0x7f1e8810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/2274165558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88103a40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 -- 172.21.15.60:0/2274165558 >> 172.21.15.60:0/2274165558 conn(0x7f1e880fd0c0 msgr2=0x7f1e880ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 -- 172.21.15.60:0/2274165558 shutdown_connections 2024-07-24T21:46:22.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.598+0000 7f1e8dff7640 1 -- 172.21.15.60:0/2274165558 wait complete. 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 Processor -- start 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 -- start start 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88103f80 0x7f1e881991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e8810cac0 0x7f1e8819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88112750 con 0x7f1e88103660 2024-07-24T21:46:22.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881125d0 con 0x7f1e8810cac0 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.599+0000 7f1e8dff7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881128d0 con 0x7f1e88103f80 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34370/0 (socket says 172.21.15.60:34370) 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 -- 172.21.15.60:0/1430711339 learned_addr learned my addr 172.21.15.60:0/1430711339 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e86ffd640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88103f80 0x7f1e881991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e87fff640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e8810cac0 0x7f1e8819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 -- 172.21.15.60:0/1430711339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88103f80 msgr2=0x7f1e881991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88103f80 0x7f1e881991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 -- 172.21.15.60:0/1430711339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e8810cac0 msgr2=0x7f1e8819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e8810cac0 0x7f1e8819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 -- 172.21.15.60:0/1430711339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8819ed10 con 0x7f1e88103660 2024-07-24T21:46:22.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e877fe640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1e78005ab0 tx=0x7f1e78003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:22.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7803f070 con 0x7f1e88103660 2024-07-24T21:46:22.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e78031d60 con 0x7f1e88103660 2024-07-24T21:46:22.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.600+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8819efa0 con 0x7f1e88103660 2024-07-24T21:46:22.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.601+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78035030 con 0x7f1e88103660 2024-07-24T21:46:22.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.601+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881a56c0 con 0x7f1e88103660 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.602+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e88103a40 con 0x7f1e88103660 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.603+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f1e78042070 con 0x7f1e88103660 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.604+0000 7f1e84ff9640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 0x7f1e700b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.604+0000 7f1e86ffd640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 0x7f1e700b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.604+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1e78104c80 con 0x7f1e88103660 2024-07-24T21:46:22.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.604+0000 7f1e86ffd640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 0x7f1e700b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ff910 tx=0x7f1e7c00b2e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:22.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.607+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1e780cc150 con 0x7f1e88103660 2024-07-24T21:46:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.842+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e881a5920 con 0x7f1e88103660 2024-07-24T21:46:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.843+0000 7f1e84ff9640 1 -- 172.21.15.60:0/1430711339 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f1e780cba50 con 0x7f1e88103660 2024-07-24T21:46:22.844 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:22.844 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:46:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 msgr2=0x7f1e700b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 0x7f1e700b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ff910 tx=0x7f1e7c00b2e0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 msgr2=0x7f1e88198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1e78005ab0 tx=0x7f1e78003cd0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 shutdown_connections 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1e700b3c20 0x7f1e700b60e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e8810cac0 0x7f1e8819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88103f80 0x7f1e881991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 --2- 172.21.15.60:0/1430711339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e88103660 0x7f1e88198c60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 >> 172.21.15.60:0/1430711339 conn(0x7f1e880fd0c0 msgr2=0x7f1e8810ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 shutdown_connections 2024-07-24T21:46:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:22.845+0000 7f1e8dff7640 1 -- 172.21.15.60:0/1430711339 wait complete. 2024-07-24T21:46:23.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:23 smithi060 ceph-mon[33000]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:46:23.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:23 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1430711339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:23.544 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-24T21:46:23.544 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-24T21:46:23.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-24T21:46:23.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 -- 172.21.15.60:0/3634855070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01080a0 msgr2=0x7efca01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:23.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 --2- 172.21.15.60:0/3634855070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01080a0 0x7efca01084e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc88009900 tx=0x7efc8802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:23.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 -- 172.21.15.60:0/3634855070 shutdown_connections 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 --2- 172.21.15.60:0/3634855070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 0x7efca010f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 --2- 172.21.15.60:0/3634855070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01080a0 0x7efca01084e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 --2- 172.21.15.60:0/3634855070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01026a0 0x7efca0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 -- 172.21.15.60:0/3634855070 >> 172.21.15.60:0/3634855070 conn(0x7efca00fe0e0 msgr2=0x7efca0100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 -- 172.21.15.60:0/3634855070 shutdown_connections 2024-07-24T21:46:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.653+0000 7efca5568640 1 -- 172.21.15.60:0/3634855070 wait complete. 2024-07-24T21:46:23.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 Processor -- start 2024-07-24T21:46:23.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 -- start start 2024-07-24T21:46:23.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:23.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01080a0 0x7efca0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:23.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 0x7efca019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:23.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efca0112670 con 0x7efca01026a0 2024-07-24T21:46:23.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efca01124f0 con 0x7efca01080a0 2024-07-24T21:46:23.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efca5568640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efca01127f0 con 0x7efca0108a20 2024-07-24T21:46:23.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.654+0000 7efc9effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:23.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34402/0 (socket says 172.21.15.60:34402) 2024-07-24T21:46:23.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 -- 172.21.15.60:0/595236880 learned_addr learned my addr 172.21.15.60:0/595236880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:23.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc967fc640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01080a0 0x7efca0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:23.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9f7fe640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 0x7efca019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:23.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 -- 172.21.15.60:0/595236880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 msgr2=0x7efca019e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:23.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 0x7efca019e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 -- 172.21.15.60:0/595236880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01080a0 msgr2=0x7efca0199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01080a0 0x7efca0199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 -- 172.21.15.60:0/595236880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca019ec70 con 0x7efca01026a0 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9effd640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc9000d950 tx=0x7efc9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90017030 con 0x7efca01026a0 2024-07-24T21:46:23.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc90004020 con 0x7efca01026a0 2024-07-24T21:46:23.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca019ef60 con 0x7efca01026a0 2024-07-24T21:46:23.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900047a0 con 0x7efca01026a0 2024-07-24T21:46:23.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.655+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca01a5800 con 0x7efca01026a0 2024-07-24T21:46:23.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.656+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca0102a80 con 0x7efca01026a0 2024-07-24T21:46:23.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.657+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7efc90009a30 con 0x7efca01026a0 2024-07-24T21:46:23.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.658+0000 7efc9cff9640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 0x7efc8c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:23.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.658+0000 7efc967fc640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 0x7efc8c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:23.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.658+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efc900e3e80 con 0x7efca01026a0 2024-07-24T21:46:23.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.659+0000 7efc967fc640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 0x7efc8c0b6160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efc88031440 tx=0x7efc88005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:23.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.661+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7efc900ab3d0 con 0x7efca01026a0 2024-07-24T21:46:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:23 smithi123 ceph-mon[37143]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:46:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:23 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1430711339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:23.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:23 smithi086 ceph-mon[37128]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T21:46:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:23 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1430711339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:23.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.854+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efca0062f80 con 0x7efc8c0b3ca0 2024-07-24T21:46:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.858+0000 7efc9cff9640 1 -- 172.21.15.60:0/595236880 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258824 (secure 0 0 0) 0x7efca0062f80 con 0x7efc8c0b3ca0 2024-07-24T21:46:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 msgr2=0x7efc8c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 0x7efc8c0b6160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efc88031440 tx=0x7efc88005b50 comp rx=0 tx=0).stop 2024-07-24T21:46:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 msgr2=0x7efca0198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc9000d950 tx=0x7efc9000de10 comp rx=0 tx=0).stop 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 shutdown_connections 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efc8c0b3ca0 0x7efc8c0b6160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca0108a20 0x7efca019e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efca01080a0 0x7efca0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 --2- 172.21.15.60:0/595236880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca01026a0 0x7efca0198bd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 >> 172.21.15.60:0/595236880 conn(0x7efca00fe0e0 msgr2=0x7efca01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 shutdown_connections 2024-07-24T21:46:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:23.862+0000 7efca5568640 1 -- 172.21.15.60:0/595236880 wait complete. 2024-07-24T21:46:23.874 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-07-24T21:46:22.084089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1136,"num_read_kb":668,"num_write":298,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404212,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537612,"statfs":{"total":1151940427776,"available":1151526514688,"internally_reserved":0,"allocated":13783040,"data_stored":7741215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":209,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:17.623813+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:17.623813+0000","last_peered":"2024-07-24T21:46:17.623813+0000","last_clean":"2024-07-24T21:46:17.623813+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:17.623813+0000","last_undegraded":"2024-07-24T21:46:17.623813+0000","last_fullsized":"2024-07-24T21:46:17.623813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":206,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:19.626008+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:19.626008+0000","last_peered":"2024-07-24T21:46:19.626008+0000","last_clean":"2024-07-24T21:46:19.626008+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:19.626008+0000","last_undegraded":"2024-07-24T21:46:19.626008+0000","last_fullsized":"2024-07-24T21:46:19.626008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":421,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.628865+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:20.628865+0000","last_peered":"2024-07-24T21:46:20.628865+0000","last_clean":"2024-07-24T21:46:20.628865+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:20.628865+0000","last_undegraded":"2024-07-24T21:46:20.628865+0000","last_fullsized":"2024-07-24T21:46:20.628865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":355,"num_read_kb":242,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":214,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.657850+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:20.657850+0000","last_peered":"2024-07-24T21:46:20.657850+0000","last_clean":"2024-07-24T21:46:20.657850+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:20.657850+0000","last_undegraded":"2024-07-24T21:46:20.657850+0000","last_fullsized":"2024-07-24T21:46:20.657850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081509+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:44:38.081509+0000","last_peered":"2024-07-24T21:44:38.081509+0000","last_clean":"2024-07-24T21:44:38.081509+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:44:38.081509+0000","last_undegraded":"2024-07-24T21:44:38.081509+0000","last_fullsized":"2024-07-24T21:44:38.081509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":204,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:15.650540+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:15.650540+0000","last_peered":"2024-07-24T21:46:15.650540+0000","last_clean":"2024-07-24T21:46:15.650540+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:15.650540+0000","last_undegraded":"2024-07-24T21:46:15.650540+0000","last_fullsized":"2024-07-24T21:46:15.650540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.312252+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:45:06.312252+0000","last_peered":"2024-07-24T21:45:06.312252+0000","last_clean":"2024-07-24T21:45:06.312252+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:45:06.312252+0000","last_undegraded":"2024-07-24T21:45:06.312252+0000","last_fullsized":"2024-07-24T21:45:06.312252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080824+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:44:38.080824+0000","last_peered":"2024-07-24T21:44:38.080824+0000","last_clean":"2024-07-24T21:44:38.080824+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:44:38.080824+0000","last_undegraded":"2024-07-24T21:44:38.080824+0000","last_fullsized":"2024-07-24T21:44:38.080824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1055,"num_read_kb":599,"num_write":173,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1138727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1072401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378718,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":540759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:23.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- 172.21.15.60:0/3191712430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 msgr2=0x7f3f3c075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 --2- 172.21.15.60:0/3191712430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c075290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009900 tx=0x7f3f2c02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- 172.21.15.60:0/3191712430 shutdown_connections 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 --2- 172.21.15.60:0/3191712430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 --2- 172.21.15.60:0/3191712430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c0757d0 0x7f3f3c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 --2- 172.21.15.60:0/3191712430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c075290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- 172.21.15.60:0/3191712430 >> 172.21.15.60:0/3191712430 conn(0x7f3f3c0fe100 msgr2=0x7f3f3c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- 172.21.15.60:0/3191712430 shutdown_connections 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- 172.21.15.60:0/3191712430 wait complete. 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 Processor -- start 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.019+0000 7f3f41e84640 1 -- start start 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c0757d0 0x7f3f3c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c1148a0 con 0x7f3f3c074eb0 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c114720 con 0x7f3f3c10ecc0 2024-07-24T21:46:24.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f41e84640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c114a20 con 0x7f3f3c0757d0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34428/0 (socket says 172.21.15.60:34428) 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 -- 172.21.15.60:0/2329465553 learned_addr learned my addr 172.21.15.60:0/2329465553 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3affd640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c0757d0 0x7f3f3c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3bfff640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 -- 172.21.15.60:0/2329465553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c0757d0 msgr2=0x7f3f3c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c0757d0 0x7f3f3c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 -- 172.21.15.60:0/2329465553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 msgr2=0x7f3f3c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 -- 172.21.15.60:0/2329465553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1a0e40 con 0x7f3f3c074eb0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3bfff640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f3b7fe640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c005940 tx=0x7f3f2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.020+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c03f030 con 0x7f3f3c074eb0 2024-07-24T21:46:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.021+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2c003cb0 con 0x7f3f3c074eb0 2024-07-24T21:46:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.021+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c1a10d0 con 0x7f3f3c074eb0 2024-07-24T21:46:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.021+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c035070 con 0x7f3f3c074eb0 2024-07-24T21:46:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.021+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c1a77f0 con 0x7f3f3c074eb0 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.021+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c075290 con 0x7f3f3c074eb0 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.023+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f3f2c042030 con 0x7f3f3c074eb0 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.024+0000 7f3f38ff9640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 0x7f3f240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.024+0000 7f3f3affd640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 0x7f3f240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.024+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3f2c104980 con 0x7f3f3c074eb0 2024-07-24T21:46:24.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.024+0000 7f3f3affd640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 0x7f3f240b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c100930 tx=0x7f3f3000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:24.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.026+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3f2c0cbe50 con 0x7f3f3c074eb0 2024-07-24T21:46:24.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.217+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f3c0ffd10 con 0x7f3f240b3c20 2024-07-24T21:46:24.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.221+0000 7f3f38ff9640 1 -- 172.21.15.60:0/2329465553 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258824 (secure 0 0 0) 0x7f3f3c0ffd10 con 0x7f3f240b3c20 2024-07-24T21:46:24.225 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:24.226 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 msgr2=0x7f3f240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 0x7f3f240b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c100930 tx=0x7f3f3000b390 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 msgr2=0x7f3f3c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c005940 tx=0x7f3f2c0042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 shutdown_connections 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3f240b3c20 0x7f3f240b60e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c10ecc0 0x7f3f3c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f3c0757d0 0x7f3f3c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 --2- 172.21.15.60:0/2329465553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f3c074eb0 0x7f3f3c19ada0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.224+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 >> 172.21.15.60:0/2329465553 conn(0x7f3f3c0fe100 msgr2=0x7f3f3c110ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:24.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.225+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 shutdown_connections 2024-07-24T21:46:24.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.225+0000 7f3f41e84640 1 -- 172.21.15.60:0/2329465553 wait complete. 2024-07-24T21:46:24.237 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-07-24T21:46:24.084544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1146,"num_read_kb":673,"num_write":300,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404212,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537612,"statfs":{"total":1151940427776,"available":1151526514688,"internally_reserved":0,"allocated":13783040,"data_stored":7741215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":42,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002961"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":214,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:22.630852+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:22.630852+0000","last_peered":"2024-07-24T21:46:22.630852+0000","last_clean":"2024-07-24T21:46:22.630852+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:22.630852+0000","last_undegraded":"2024-07-24T21:46:22.630852+0000","last_fullsized":"2024-07-24T21:46:22.630852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":206,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:19.626008+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:19.626008+0000","last_peered":"2024-07-24T21:46:19.626008+0000","last_clean":"2024-07-24T21:46:19.626008+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:19.626008+0000","last_undegraded":"2024-07-24T21:46:19.626008+0000","last_fullsized":"2024-07-24T21:46:19.626008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":421,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.628865+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:20.628865+0000","last_peered":"2024-07-24T21:46:20.628865+0000","last_clean":"2024-07-24T21:46:20.628865+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:20.628865+0000","last_undegraded":"2024-07-24T21:46:20.628865+0000","last_fullsized":"2024-07-24T21:46:20.628865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":355,"num_read_kb":242,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":214,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.657850+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:20.657850+0000","last_peered":"2024-07-24T21:46:20.657850+0000","last_clean":"2024-07-24T21:46:20.657850+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:20.657850+0000","last_undegraded":"2024-07-24T21:46:20.657850+0000","last_fullsized":"2024-07-24T21:46:20.657850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081509+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:44:38.081509+0000","last_peered":"2024-07-24T21:44:38.081509+0000","last_clean":"2024-07-24T21:44:38.081509+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:44:38.081509+0000","last_undegraded":"2024-07-24T21:44:38.081509+0000","last_fullsized":"2024-07-24T21:44:38.081509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":209,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.657293+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:20.657293+0000","last_peered":"2024-07-24T21:46:20.657293+0000","last_clean":"2024-07-24T21:46:20.657293+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:20.657293+0000","last_undegraded":"2024-07-24T21:46:20.657293+0000","last_fullsized":"2024-07-24T21:46:20.657293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:22.695443+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:45:22.695443+0000","last_peered":"2024-07-24T21:45:22.695443+0000","last_clean":"2024-07-24T21:45:22.695443+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:45:22.695443+0000","last_undegraded":"2024-07-24T21:45:22.695443+0000","last_fullsized":"2024-07-24T21:45:22.695443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080824+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:44:38.080824+0000","last_peered":"2024-07-24T21:44:38.080824+0000","last_clean":"2024-07-24T21:44:38.080824+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:44:38.080824+0000","last_undegraded":"2024-07-24T21:44:38.080824+0000","last_fullsized":"2024-07-24T21:44:38.080824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1065,"num_read_kb":604,"num_write":175,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1138727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1072401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378718,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":540759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:24.244 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-24T21:46:24.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 -- 172.21.15.60:0/4037253921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 msgr2=0x7f15d8101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 --2- 172.21.15.60:0/4037253921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d8101550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009900 tx=0x7f15d402a480 comp rx=0 tx=0).stop 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 -- 172.21.15.60:0/4037253921 shutdown_connections 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 --2- 172.21.15.60:0/4037253921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d8111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 --2- 172.21.15.60:0/4037253921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 0x7f15d810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 --2- 172.21.15.60:0/4037253921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d8101550 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.381+0000 7f15df616640 1 -- 172.21.15.60:0/4037253921 >> 172.21.15.60:0/4037253921 conn(0x7f15d80fd020 msgr2=0x7f15d80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- 172.21.15.60:0/4037253921 shutdown_connections 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- 172.21.15.60:0/4037253921 wait complete. 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 Processor -- start 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- start start 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 0x7f15d819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d81a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81148d0 con 0x7f15d8101170 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8114750 con 0x7f15d810ed20 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15df616640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8114a50 con 0x7f15d8101a90 2024-07-24T21:46:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.382+0000 7f15dd3b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34450/0 (socket says 172.21.15.60:34450) 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 -- 172.21.15.60:0/915443053 learned_addr learned my addr 172.21.15.60:0/915443053 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15ddbb3640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d81a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dcbb1640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 0x7f15d819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 -- 172.21.15.60:0/915443053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 msgr2=0x7f15d819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 0x7f15d819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 -- 172.21.15.60:0/915443053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 msgr2=0x7f15d81a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d81a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 -- 172.21.15.60:0/915443053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d81a0d70 con 0x7f15d8101170 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15ddbb3640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15dd3b2640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:24.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d403f030 con 0x7f15d8101170 2024-07-24T21:46:24.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d4003800 con 0x7f15d8101170 2024-07-24T21:46:24.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d81a1000 con 0x7f15d8101170 2024-07-24T21:46:24.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4035070 con 0x7f15d8101170 2024-07-24T21:46:24.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.383+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d81a7720 con 0x7f15d8101170 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.384+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d8101550 con 0x7f15d8101170 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.386+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f15d4003cb0 con 0x7f15d8101170 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.386+0000 7f15ce7fc640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 0x7f15c40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.386+0000 7f15dcbb1640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 0x7f15c40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.387+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f15d4104950 con 0x7f15d8101170 2024-07-24T21:46:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.387+0000 7f15dcbb1640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 0x7f15c40b60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15d81005f0 tx=0x7f15c800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:24.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.389+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f15d40cbe80 con 0x7f15d8101170 2024-07-24T21:46:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.582+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15d810fe60 con 0x7f15c40b3c20 2024-07-24T21:46:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.586+0000 7f15ce7fc640 1 -- 172.21.15.60:0/915443053 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258824 (secure 0 0 0) 0x7f15d810fe60 con 0x7f15c40b3c20 2024-07-24T21:46:24.592 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 msgr2=0x7f15c40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 0x7f15c40b60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15d81005f0 tx=0x7f15c800a2d0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 msgr2=0x7f15d819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006010 tx=0x7f15d40042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 shutdown_connections 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f15c40b3c20 0x7f15c40b60e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d810ed20 0x7f15d81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8101a90 0x7f15d819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 --2- 172.21.15.60:0/915443053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d8101170 0x7f15d819acd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:24.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.589+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 >> 172.21.15.60:0/915443053 conn(0x7f15d80fd020 msgr2=0x7f15d810f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:24.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.590+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 shutdown_connections 2024-07-24T21:46:24.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:24.590+0000 7f15df616640 1 -- 172.21.15.60:0/915443053 wait complete. 2024-07-24T21:46:24.602 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-07-24T21:46:24.084544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1146,"num_read_kb":673,"num_write":300,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404212,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537612,"statfs":{"total":1151940427776,"available":1151526514688,"internally_reserved":0,"allocated":13783040,"data_stored":7741215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":42,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002961"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":214,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:22.630852+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:22.630852+0000","last_peered":"2024-07-24T21:46:22.630852+0000","last_clean":"2024-07-24T21:46:22.630852+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:22.630852+0000","last_undegraded":"2024-07-24T21:46:22.630852+0000","last_fullsized":"2024-07-24T21:46:22.630852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":206,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:19.626008+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:19.626008+0000","last_peered":"2024-07-24T21:46:19.626008+0000","last_clean":"2024-07-24T21:46:19.626008+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:19.626008+0000","last_undegraded":"2024-07-24T21:46:19.626008+0000","last_fullsized":"2024-07-24T21:46:19.626008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":421,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.628865+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:20.628865+0000","last_peered":"2024-07-24T21:46:20.628865+0000","last_clean":"2024-07-24T21:46:20.628865+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:20.628865+0000","last_undegraded":"2024-07-24T21:46:20.628865+0000","last_fullsized":"2024-07-24T21:46:20.628865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":355,"num_read_kb":242,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":214,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.657850+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:20.657850+0000","last_peered":"2024-07-24T21:46:20.657850+0000","last_clean":"2024-07-24T21:46:20.657850+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:20.657850+0000","last_undegraded":"2024-07-24T21:46:20.657850+0000","last_fullsized":"2024-07-24T21:46:20.657850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081509+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:44:38.081509+0000","last_peered":"2024-07-24T21:44:38.081509+0000","last_clean":"2024-07-24T21:44:38.081509+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:44:38.081509+0000","last_undegraded":"2024-07-24T21:44:38.081509+0000","last_fullsized":"2024-07-24T21:44:38.081509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":209,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:20.657293+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:20.657293+0000","last_peered":"2024-07-24T21:46:20.657293+0000","last_clean":"2024-07-24T21:46:20.657293+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:20.657293+0000","last_undegraded":"2024-07-24T21:46:20.657293+0000","last_fullsized":"2024-07-24T21:46:20.657293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:22.695443+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:45:22.695443+0000","last_peered":"2024-07-24T21:45:22.695443+0000","last_clean":"2024-07-24T21:45:22.695443+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:45:22.695443+0000","last_undegraded":"2024-07-24T21:45:22.695443+0000","last_fullsized":"2024-07-24T21:45:22.695443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080824+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:44:38.080824+0000","last_peered":"2024-07-24T21:44:38.080824+0000","last_clean":"2024-07-24T21:44:38.080824+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:44:38.080824+0000","last_undegraded":"2024-07-24T21:44:38.080824+0000","last_fullsized":"2024-07-24T21:44:38.080824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1065,"num_read_kb":604,"num_write":175,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1138727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1072401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378718,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":540759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:24.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:46:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:25.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:25 smithi060 ceph-mon[33000]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:25 smithi060 ceph-mon[33000]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T21:46:25.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:25 smithi060 ceph-mon[33000]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:25 smithi060 ceph-mon[33000]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:25 smithi123 ceph-mon[37143]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:25 smithi123 ceph-mon[37143]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T21:46:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:25 smithi123 ceph-mon[37143]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:25 smithi123 ceph-mon[37143]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:25 smithi086 ceph-mon[37128]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:25 smithi086 ceph-mon[37128]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T21:46:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:25 smithi086 ceph-mon[37128]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:25 smithi086 ceph-mon[37128]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 -- 172.21.15.60:0/2622637772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb410a2f0 msgr2=0x7fddb410c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/2622637772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb410a2f0 0x7fddb410c6e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 -- 172.21.15.60:0/2622637772 shutdown_connections 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/2622637772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410cc20 0x7fddb410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/2622637772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb410a2f0 0x7fddb410c6e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/2622637772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb41011e0 0x7fddb41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 -- 172.21.15.60:0/2622637772 >> 172.21.15.60:0/2622637772 conn(0x7fddb4076940 msgr2=0x7fddb4076d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 -- 172.21.15.60:0/2622637772 shutdown_connections 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.642+0000 7fddb9ac6640 1 -- 172.21.15.60:0/2622637772 wait complete. 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 Processor -- start 2024-07-24T21:46:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 -- start start 2024-07-24T21:46:26.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:26.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410a2f0 0x7fddb41026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:26.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 0x7fddb4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:26.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4112730 con 0x7fddb41011e0 2024-07-24T21:46:26.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddb41125b0 con 0x7fddb410cc20 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb9ac6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fddb41128b0 con 0x7fddb410a2f0 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60882/0 (socket says 172.21.15.60:60882) 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb37fe640 1 -- 172.21.15.60:0/1706670413 learned_addr learned my addr 172.21.15.60:0/1706670413 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb2ffd640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410a2f0 0x7fddb41026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb3fff640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 0x7fddb4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.643+0000 7fddb37fe640 1 -- 172.21.15.60:0/1706670413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410a2f0 msgr2=0x7fddb41026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb37fe640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410a2f0 0x7fddb41026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb37fe640 1 -- 172.21.15.60:0/1706670413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 msgr2=0x7fddb4102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb37fe640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 0x7fddb4102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb37fe640 1 -- 172.21.15.60:0/1706670413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41a6fe0 con 0x7fddb41011e0 2024-07-24T21:46:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb3fff640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 0x7fddb4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb37fe640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdda400c920 tx=0x7fdda400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4007c70 con 0x7fddb41011e0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb41a72d0 con 0x7fddb41011e0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda4007e10 con 0x7fddb41011e0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4018820 con 0x7fddb41011e0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.644+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb41a77e0 con 0x7fddb41011e0 2024-07-24T21:46:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.645+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb41015c0 con 0x7fddb41011e0 2024-07-24T21:46:26.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.647+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fdda400f450 con 0x7fddb41011e0 2024-07-24T21:46:26.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.647+0000 7fddb0ff9640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 0x7fdd9c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:26.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.647+0000 7fddb2ffd640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 0x7fdd9c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:26.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.647+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fdda40e4150 con 0x7fddb41011e0 2024-07-24T21:46:26.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.648+0000 7fddb2ffd640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 0x7fdd9c0b60e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdda8031440 tx=0x7fdda8005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:26.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.650+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdda4014030 con 0x7fddb41011e0 2024-07-24T21:46:26.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.888+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddb41a7c40 con 0x7fddb41011e0 2024-07-24T21:46:26.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.888+0000 7fddb0ff9640 1 -- 172.21.15.60:0/1706670413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fdda40ab710 con 0x7fddb41011e0 2024-07-24T21:46:26.890 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:26.890 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:46:26.890 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 msgr2=0x7fdd9c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 0x7fdd9c0b60e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdda8031440 tx=0x7fdda8005e80 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 msgr2=0x7fddb41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdda400c920 tx=0x7fdda400cde0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 shutdown_connections 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdd9c0b3c20 0x7fdd9c0b60e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddb410cc20 0x7fddb4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.890+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddb410a2f0 0x7fddb41026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.891+0000 7fddb9ac6640 1 --2- 172.21.15.60:0/1706670413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb41011e0 0x7fddb41021b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.891+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 >> 172.21.15.60:0/1706670413 conn(0x7fddb4076940 msgr2=0x7fddb40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.891+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 shutdown_connections 2024-07-24T21:46:26.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:26.891+0000 7fddb9ac6640 1 -- 172.21.15.60:0/1706670413 wait complete. 2024-07-24T21:46:27.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:46:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:27 smithi123 ceph-mon[37143]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1706670413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:27.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:27 smithi086 ceph-mon[37128]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1706670413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:27.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:27 smithi060 ceph-mon[33000]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:27.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1706670413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:27.879 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:29.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 -- 172.21.15.60:0/2001438377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d41079e0 msgr2=0x7f36d410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:29.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 --2- 172.21.15.60:0/2001438377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d41079e0 0x7f36d410e1e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009ca0 tx=0x7f36cc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 -- 172.21.15.60:0/2001438377 shutdown_connections 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 --2- 172.21.15.60:0/2001438377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d41079e0 0x7f36d410e1e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 --2- 172.21.15.60:0/2001438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d4107060 0x7f36d41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 --2- 172.21.15.60:0/2001438377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4101650 0x7f36d4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 -- 172.21.15.60:0/2001438377 >> 172.21.15.60:0/2001438377 conn(0x7f36d40fd0c0 msgr2=0x7f36d40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 -- 172.21.15.60:0/2001438377 shutdown_connections 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 -- 172.21.15.60:0/2001438377 wait complete. 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.457+0000 7f36d9925640 1 Processor -- start 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 -- start start 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 0x7f36d41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:29.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 0x7f36d419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4111610 con 0x7f36d4101650 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4111490 con 0x7f36d41079e0 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d9925640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4111790 con 0x7f36d4107060 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60888/0 (socket says 172.21.15.60:60888) 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 -- 172.21.15.60:0/3091889871 learned_addr learned my addr 172.21.15.60:0/3091889871 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d27fc640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 0x7f36d41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 -- 172.21.15.60:0/3091889871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 msgr2=0x7f36d41990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 0x7f36d41990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d37fe640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 0x7f36d419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 -- 172.21.15.60:0/3091889871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 msgr2=0x7f36d419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 0x7f36d419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 -- 172.21.15.60:0/3091889871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d419ec20 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d27fc640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 0x7f36d41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d37fe640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 0x7f36d419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.458+0000 7f36d2ffd640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c920 tx=0x7f36c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.459+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4007c70 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.459+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c4007e10 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.459+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4018820 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.459+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d419ef10 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.459+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a55d0 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.460+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d4101a30 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.462+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f36c400f450 con 0x7f36d4101650 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.463+0000 7f36d8923640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 0x7f36c00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:29.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.463+0000 7f36d27fc640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 0x7f36c00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:29.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.463+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f36c40e4250 con 0x7f36d4101650 2024-07-24T21:46:29.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.463+0000 7f36d27fc640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 0x7f36c00b60e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36d4103730 tx=0x7f36bc00b5f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:29.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.466+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f36c4014030 con 0x7f36d4101650 2024-07-24T21:46:29.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.662+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36d410a240 con 0x7f36d4101650 2024-07-24T21:46:29.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:29 smithi060 ceph-mon[33000]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.664+0000 7f36d8923640 1 -- 172.21.15.60:0/3091889871 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23071 (secure 0 0 0) 0x7f36c40ab7b0 con 0x7f36d4101650 2024-07-24T21:46:29.665 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:29.665 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:44:37.734936+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.666+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 msgr2=0x7f36c00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.666+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 0x7f36c00b60e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36d4103730 tx=0x7f36bc00b5f0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.666+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 msgr2=0x7f36d4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.666+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c920 tx=0x7f36c400cde0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 shutdown_connections 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36c00b3c20 0x7f36c00b60e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d41079e0 0x7f36d419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d4107060 0x7f36d41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 --2- 172.21.15.60:0/3091889871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101650 0x7f36d4198b70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 >> 172.21.15.60:0/3091889871 conn(0x7f36d40fd0c0 msgr2=0x7f36d406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 shutdown_connections 2024-07-24T21:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:29.667+0000 7f36d9925640 1 -- 172.21.15.60:0/3091889871 wait complete. 2024-07-24T21:46:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:29 smithi123 ceph-mon[37143]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:29 smithi086 ceph-mon[37128]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:30.347 DEBUG:tasks.fwd_scrub:Yielding 2024-07-24T21:46:30.347 INFO:teuthology.run_tasks:Running task exec... 2024-07-24T21:46:30.357 INFO:teuthology.task.exec:Executing custom commands... 2024-07-24T21:46:30.357 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:46:30.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-07-24T21:46:30.358 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-24T21:46:30.358 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:46:30.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 -- 172.21.15.60:0/2590441570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec1035d0 msgr2=0x7f28ec1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 --2- 172.21.15.60:0/2590441570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec1035d0 0x7f28ec1039b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 -- 172.21.15.60:0/2590441570 shutdown_connections 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 --2- 172.21.15.60:0/2590441570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec10c9f0 0x7f28ec10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 --2- 172.21.15.60:0/2590441570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec103ef0 0x7f28ec10c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 --2- 172.21.15.60:0/2590441570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec1035d0 0x7f28ec1039b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 -- 172.21.15.60:0/2590441570 >> 172.21.15.60:0/2590441570 conn(0x7f28ec0782e0 msgr2=0x7f28ec0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 -- 172.21.15.60:0/2590441570 shutdown_connections 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.478+0000 7f28f2816640 1 -- 172.21.15.60:0/2590441570 wait complete. 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 Processor -- start 2024-07-24T21:46:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 -- start start 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 0x7f28ec198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 0x7f28ec199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112680 con 0x7f28ec10c9f0 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112500 con 0x7f28ec1035d0 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f2816640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112800 con 0x7f28ec103ef0 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f0db3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f0db3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60908/0 (socket says 172.21.15.60:60908) 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28f0db3640 1 -- 172.21.15.60:0/1286063536 learned_addr learned my addr 172.21.15.60:0/1286063536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.479+0000 7f28ebfff640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 0x7f28ec198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28eb7fe640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 0x7f28ec199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 -- 172.21.15.60:0/1286063536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 msgr2=0x7f28ec199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 0x7f28ec199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 -- 172.21.15.60:0/1286063536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 msgr2=0x7f28ec198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 0x7f28ec198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 -- 172.21.15.60:0/1286063536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec19ecc0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28eb7fe640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 0x7f28ec199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28ebfff640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 0x7f28ec198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f0db3640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f28e400eaa0 tx=0x7f28e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400cdb0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec19efb0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e4005020 con 0x7f28ec10c9f0 2024-07-24T21:46:30.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40116d0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.480+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a5670 con 0x7f28ec10c9f0 2024-07-24T21:46:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.481+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec1039b0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.484+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f28e40054a0 con 0x7f28ec10c9f0 2024-07-24T21:46:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.484+0000 7f28e97fa640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 0x7f28d40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.485+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f28e40e4400 con 0x7f28ec10c9f0 2024-07-24T21:46:30.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.485+0000 7f28ebfff640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 0x7f28d40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.485+0000 7f28ebfff640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 0x7f28d40b60e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031440 tx=0x7f28dc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.487+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f28e4016030 con 0x7f28ec10c9f0 2024-07-24T21:46:30.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:30 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3091889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:30.639 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:30.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.711+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f28ec0ff810 con 0x7f28d40b3c20 2024-07-24T21:46:30.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:30 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3091889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:30.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:30 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3091889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.867+0000 7f28e97fa640 1 -- 172.21.15.60:0/1286063536 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f28ec0ff810 con 0x7f28d40b3c20 2024-07-24T21:46:30.868 INFO:teuthology.orchestra.run.smithi060.stdout:Schedule set for path / 2024-07-24T21:46:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 msgr2=0x7f28d40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 0x7f28d40b60e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031440 tx=0x7f28dc005a20 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 msgr2=0x7f28ec19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f28e400eaa0 tx=0x7f28e400ef60 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 shutdown_connections 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28d40b3c20 0x7f28d40b60e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28ec10c9f0 0x7f28ec19e4f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28ec103ef0 0x7f28ec199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 --2- 172.21.15.60:0/1286063536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28ec1035d0 0x7f28ec198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 >> 172.21.15.60:0/1286063536 conn(0x7f28ec0782e0 msgr2=0x7f28ec0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 shutdown_connections 2024-07-24T21:46:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.869+0000 7f28f2816640 1 -- 172.21.15.60:0/1286063536 wait complete. 2024-07-24T21:46:30.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-07-24T21:46:30.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 -- 172.21.15.60:0/2213506397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 msgr2=0x7fc1c810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/2213506397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c810f230 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009ca0 tx=0x7fc1c002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 -- 172.21.15.60:0/2213506397 shutdown_connections 2024-07-24T21:46:30.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/2213506397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c810f230 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/2213506397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81080b0 0x7fc1c81084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/2213506397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81026b0 0x7fc1c8102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 -- 172.21.15.60:0/2213506397 >> 172.21.15.60:0/2213506397 conn(0x7fc1c80fe0f0 msgr2=0x7fc1c8100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 -- 172.21.15.60:0/2213506397 shutdown_connections 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.990+0000 7fc1cf04e640 1 -- 172.21.15.60:0/2213506397 wait complete. 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 Processor -- start 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 -- start start 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81026b0 0x7fc1c8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81080b0 0x7fc1c81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112680 con 0x7fc1c8108a30 2024-07-24T21:46:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112500 con 0x7fc1c81026b0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cf04e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8112800 con 0x7fc1c81080b0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cd5eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cd5eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60956/0 (socket says 172.21.15.60:60956) 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1cd5eb640 1 -- 172.21.15.60:0/4239763201 learned_addr learned my addr 172.21.15.60:0/4239763201 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.991+0000 7fc1ccdea640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81026b0 0x7fc1c8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1c7fff640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81080b0 0x7fc1c81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 -- 172.21.15.60:0/4239763201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81080b0 msgr2=0x7fc1c81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81080b0 0x7fc1c81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 -- 172.21.15.60:0/4239763201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81026b0 msgr2=0x7fc1c8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81026b0 0x7fc1c8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 -- 172.21.15.60:0/4239763201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c819ec20 con 0x7fc1c8108a30 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cd5eb640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005cc0 tx=0x7fc1c0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:30.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0003b70 con 0x7fc1c8108a30 2024-07-24T21:46:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c819eeb0 con 0x7fc1c8108a30 2024-07-24T21:46:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c00318f0 con 0x7fc1c8108a30 2024-07-24T21:46:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c002f950 con 0x7fc1c8108a30 2024-07-24T21:46:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.992+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81a55d0 con 0x7fc1c8108a30 2024-07-24T21:46:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.993+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8102a90 con 0x7fc1c8108a30 2024-07-24T21:46:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.995+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fc1c002fb10 con 0x7fc1c8108a30 2024-07-24T21:46:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.996+0000 7fc1c5ffb640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 0x7fc1b80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.996+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc1c01052d0 con 0x7fc1c8108a30 2024-07-24T21:46:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.996+0000 7fc1ccdea640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 0x7fc1b80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:30.996+0000 7fc1ccdea640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 0x7fc1b80b60e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009bd0 tx=0x7fc1bc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:31.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.000+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc1c00cc800 con 0x7fc1c8108a30 2024-07-24T21:46:31.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.226+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc1c8062f80 con 0x7fc1b80b3c20 2024-07-24T21:46:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.287+0000 7fc1c5ffb640 1 -- 172.21.15.60:0/4239763201 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc1c8062f80 con 0x7fc1b80b3c20 2024-07-24T21:46:31.288 INFO:teuthology.orchestra.run.smithi060.stdout:Retention added to path / 2024-07-24T21:46:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 msgr2=0x7fc1b80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 0x7fc1b80b60e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009bd0 tx=0x7fc1bc005f90 comp rx=0 tx=0).stop 2024-07-24T21:46:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 msgr2=0x7fc1c819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005cc0 tx=0x7fc1c0031700 comp rx=0 tx=0).stop 2024-07-24T21:46:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 shutdown_connections 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc1b80b3c20 0x7fc1b80b60e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c8108a30 0x7fc1c819e450 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1c81080b0 0x7fc1c81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 --2- 172.21.15.60:0/4239763201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1c81026b0 0x7fc1c8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 >> 172.21.15.60:0/4239763201 conn(0x7fc1c80fe0f0 msgr2=0x7fc1c81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 shutdown_connections 2024-07-24T21:46:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.289+0000 7fc1cf04e640 1 -- 172.21.15.60:0/4239763201 wait complete. 2024-07-24T21:46:31.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 -- 172.21.15.60:0/1001721178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a810ca30 msgr2=0x7f51a810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 --2- 172.21.15.60:0/1001721178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a810ca30 0x7f51a810eeb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009ca0 tx=0x7f51a002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 -- 172.21.15.60:0/1001721178 shutdown_connections 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 --2- 172.21.15.60:0/1001721178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a810ca30 0x7f51a810eeb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 --2- 172.21.15.60:0/1001721178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a8103f00 0x7f51a810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 --2- 172.21.15.60:0/1001721178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a81035e0 0x7f51a81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 -- 172.21.15.60:0/1001721178 >> 172.21.15.60:0/1001721178 conn(0x7f51a80fd030 msgr2=0x7f51a80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 -- 172.21.15.60:0/1001721178 shutdown_connections 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.411+0000 7f51af44e640 1 -- 172.21.15.60:0/1001721178 wait complete. 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 Processor -- start 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 -- start start 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 0x7f51a81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a810ca30 0x7f51a819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51a81126b0 con 0x7f51a81035e0 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8112530 con 0x7f51a8103f00 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51af44e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8112830 con 0x7f51a810ca30 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60974/0 (socket says 172.21.15.60:60974) 2024-07-24T21:46:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 -- 172.21.15.60:0/3643577331 learned_addr learned my addr 172.21.15.60:0/3643577331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad9eb640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a810ca30 0x7f51a819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ac9e9640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 0x7f51a81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 -- 172.21.15.60:0/3643577331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a810ca30 msgr2=0x7f51a819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a810ca30 0x7f51a819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 -- 172.21.15.60:0/3643577331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 msgr2=0x7f51a81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 0x7f51a81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ad1ea640 1 -- 172.21.15.60:0/3643577331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a819ec30 con 0x7f51a81035e0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.412+0000 7f51ac9e9640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 0x7f51a81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f51ad1ea640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c890 tx=0x7f51a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4004220 con 0x7f51a81035e0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a40043c0 con 0x7f51a81035e0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a819ef20 con 0x7f51a81035e0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4004a00 con 0x7f51a81035e0 2024-07-24T21:46:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.413+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81a55e0 con 0x7f51a81035e0 2024-07-24T21:46:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.414+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a81039c0 con 0x7f51a81035e0 2024-07-24T21:46:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.415+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f51a40079b0 con 0x7f51a81035e0 2024-07-24T21:46:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.417+0000 7f519e7fc640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 0x7f51940b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.417+0000 7f51ac9e9640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 0x7f51940b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.417+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f51a40e3f90 con 0x7f51a81035e0 2024-07-24T21:46:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.417+0000 7f51ac9e9640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 0x7f51940b60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51a80ff7d0 tx=0x7f5198005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.420+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f51a4014030 con 0x7f51a81035e0 2024-07-24T21:46:31.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.638+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f51a80fec40 con 0x7f51940b3c20 2024-07-24T21:46:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.645+0000 7f519e7fc640 1 -- 172.21.15.60:0/3643577331 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f51a80fec40 con 0x7f51940b3c20 2024-07-24T21:46:31.646 INFO:teuthology.orchestra.run.smithi060.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-24T00:00:00", "created": "2024-07-24T21:46:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 msgr2=0x7f51940b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 0x7f51940b60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51a80ff7d0 tx=0x7f5198005f90 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 msgr2=0x7f51a8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c890 tx=0x7f51a400cd50 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 shutdown_connections 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f51940b3c20 0x7f51940b60e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51a810ca30 0x7f51a819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51a8103f00 0x7f51a81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 --2- 172.21.15.60:0/3643577331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a81035e0 0x7f51a8198b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 >> 172.21.15.60:0/3643577331 conn(0x7f51a80fd030 msgr2=0x7f51a810ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 shutdown_connections 2024-07-24T21:46:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.647+0000 7f51af44e640 1 -- 172.21.15.60:0/3643577331 wait complete. 2024-07-24T21:46:31.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-24T21:46:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:31 smithi123 ceph-mon[37143]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:31 smithi123 ceph-mon[37143]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 -- 172.21.15.60:0/1302753566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb23810ca10 msgr2=0x7fb23810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 --2- 172.21.15.60:0/1302753566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb23810ca10 0x7fb23810ee90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb230009ca0 tx=0x7fb23002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 -- 172.21.15.60:0/1302753566 shutdown_connections 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 --2- 172.21.15.60:0/1302753566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb23810ca10 0x7fb23810ee90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 --2- 172.21.15.60:0/1302753566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 0x7fb23810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.764+0000 7fb240567640 1 --2- 172.21.15.60:0/1302753566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2381035c0 0x7fb2381039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- 172.21.15.60:0/1302753566 >> 172.21.15.60:0/1302753566 conn(0x7fb2380fcfb0 msgr2=0x7fb2380ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- 172.21.15.60:0/1302753566 shutdown_connections 2024-07-24T21:46:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- 172.21.15.60:0/1302753566 wait complete. 2024-07-24T21:46:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 Processor -- start 2024-07-24T21:46:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- start start 2024-07-24T21:46:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 0x7fb238199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23810ca10 0x7fb23819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb238112690 con 0x7fb2381035c0 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb238112510 con 0x7fb238103ee0 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.765+0000 7fb240567640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb238112810 con 0x7fb23810ca10 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60994/0 (socket says 172.21.15.60:60994) 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 -- 172.21.15.60:0/429779930 learned_addr learned my addr 172.21.15.60:0/429779930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23db02640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 0x7fb238199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23eb04640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23810ca10 0x7fb23819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 -- 172.21.15.60:0/429779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23810ca10 msgr2=0x7fb23819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23810ca10 0x7fb23819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 -- 172.21.15.60:0/429779930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 msgr2=0x7fb238199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 0x7fb238199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 -- 172.21.15.60:0/429779930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23819ec90 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb23e303640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb23400d770 tx=0x7fb23400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234017070 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2340054d0 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23819ef80 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234004c30 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.766+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a5640 con 0x7fb2381035c0 2024-07-24T21:46:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.767+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2381039a0 con 0x7fb2381035c0 2024-07-24T21:46:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.769+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fb234004dd0 con 0x7fb2381035c0 2024-07-24T21:46:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.769+0000 7fb22f7fe640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 0x7fb2240b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.769+0000 7fb23db02640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 0x7fb2240b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.770+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb2340e3ca0 con 0x7fb2381035c0 2024-07-24T21:46:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.770+0000 7fb23db02640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 0x7fb2240b6160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb228005db0 tx=0x7fb228005d40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.772+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb2340ab250 con 0x7fb2381035c0 2024-07-24T21:46:31.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:31 smithi060 ceph-mon[33000]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:31.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:31 smithi060 ceph-mon[33000]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:31 smithi086 ceph-mon[37128]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-24T21:46:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:31 smithi086 ceph-mon[37128]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:31.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:31.998+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb238062f80 con 0x7fb2240b3ca0 2024-07-24T21:46:32.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.005+0000 7fb22f7fe640 1 -- 172.21.15.60:0/429779930 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb238062f80 con 0x7fb2240b3ca0 2024-07-24T21:46:32.007 INFO:teuthology.orchestra.run.smithi060.stdout:/ 1m 6m3h 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 msgr2=0x7fb2240b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 0x7fb2240b6160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb228005db0 tx=0x7fb228005d40 comp rx=0 tx=0).stop 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 msgr2=0x7fb238198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb23400d770 tx=0x7fb23400dc30 comp rx=0 tx=0).stop 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 shutdown_connections 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb2240b3ca0 0x7fb2240b6160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23810ca10 0x7fb23819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb238103ee0 0x7fb238199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 --2- 172.21.15.60:0/429779930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2381035c0 0x7fb238198bf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 >> 172.21.15.60:0/429779930 conn(0x7fb2380fcfb0 msgr2=0x7fb2380fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 shutdown_connections 2024-07-24T21:46:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.008+0000 7fb240567640 1 -- 172.21.15.60:0/429779930 wait complete. 2024-07-24T21:46:32.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-07-24T21:46:32.047 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-07-24T21:46:32.058 INFO:teuthology.run_tasks:Running task background_exec... 2024-07-24T21:46:32.104 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T21:46:32.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-07-24T21:46:32.106 INFO:teuthology.run_tasks:Running task quiescer... 2024-07-24T21:46:32.141 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-24T21:46:32.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:46:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 -- 172.21.15.60:0/3584496202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 msgr2=0x7f7bdc10e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/3584496202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc10e8c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009900 tx=0x7f7bc802a480 comp rx=0 tx=0).stop 2024-07-24T21:46:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 -- 172.21.15.60:0/3584496202 shutdown_connections 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/3584496202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc10ee00 0x7f7bdc1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/3584496202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc10e8c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/3584496202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc101250 0x7f7bdc101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 -- 172.21.15.60:0/3584496202 >> 172.21.15.60:0/3584496202 conn(0x7f7bdc0fd0c0 msgr2=0x7f7bdc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 -- 172.21.15.60:0/3584496202 shutdown_connections 2024-07-24T21:46:32.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.306+0000 7f7be2bd9640 1 -- 172.21.15.60:0/3584496202 wait complete. 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 Processor -- start 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 -- start start 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc101250 0x7f7bdc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc10ee00 0x7f7bdc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc114940 con 0x7f7bdc101b70 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1147c0 con 0x7f7bdc101250 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be2bd9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc114ac0 con 0x7f7bdc10ee00 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32788/0 (socket says 172.21.15.60:32788) 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2379137571 learned_addr learned my addr 172.21.15.60:0/2379137571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be0975640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc101250 0x7f7bdc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.307+0000 7f7be1176640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc10ee00 0x7f7bdc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2379137571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc10ee00 msgr2=0x7f7bdc1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc10ee00 0x7f7bdc1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2379137571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc101250 msgr2=0x7f7bdc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc101250 0x7f7bdc19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2379137571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc1a0f40 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005940 tx=0x7f7bc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bd9ffb640 1 -- 172.21.15.60:0/2379137571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc803f030 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bd9ffb640 1 -- 172.21.15.60:0/2379137571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc8003cb0 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc1a11d0 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7bd9ffb640 1 -- 172.21.15.60:0/2379137571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8035070 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.308+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1a78f0 con 0x7f7bdc101b70 2024-07-24T21:46:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.311+0000 7f7be4f8a740 1 Processor -- start 2024-07-24T21:46:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 -- start start 2024-07-24T21:46:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7bd9ffb640 1 -- 172.21.15.60:0/2379137571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f7bc8042030 con 0x7f7bdc101b70 2024-07-24T21:46:32.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 0x56218eec2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec2850 0x56218eece2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56218ee399f0 con 0x56218eece7e0 2024-07-24T21:46:32.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56218ee28ef0 con 0x56218eec2850 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56218ee2cf30 con 0x56218eec1f30 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32802/0 (socket says 172.21.15.60:32802) 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2376210367 learned_addr learned my addr 172.21.15.60:0/2376210367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be1176640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 0x56218eec2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.312+0000 7f7be0975640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec2850 0x56218eece2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2376210367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 msgr2=0x56218eec2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 0x56218eec2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2376210367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec2850 msgr2=0x56218eece2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec2850 0x56218eece2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 -- 172.21.15.60:0/2376210367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218eb89930 con 0x56218eece7e0 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7be1176640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 0x56218eec2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:46:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bd9ffb640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 0x7f7bcc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7be1176640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 0x7f7bcc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bd9ffb640 1 -- 172.21.15.60:0/2379137571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7bc81045c0 con 0x7f7bdc101b70 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7bc810a730 tx=0x7f7bc810a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5f0fcbae9376e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bc6ffd640 1 -- 172.21.15.60:0/2376210367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8049070 con 0x56218eece7e0 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bc6ffd640 1 -- 172.21.15.60:0/2376210367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc80caf00 con 0x56218eece7e0 2024-07-24T21:46:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.313+0000 7f7bc6ffd640 1 -- 172.21.15.60:0/2376210367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80c72b0 con 0x56218eece7e0 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 -- 172.21.15.60:0/2376210367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 msgr2=0x56218eed0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be1176640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 0x7f7bcc0b6160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009fd0 tx=0x7f7bd40097e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7bc810a730 tx=0x7f7bc810a820 comp rx=0 tx=0).stop 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 -- 172.21.15.60:0/2376210367 shutdown_connections 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eece7e0 0x56218eed0bd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec2850 0x56218eece2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/2376210367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eec1f30 0x56218eec2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 -- 172.21.15.60:0/2376210367 >> 172.21.15.60:0/2376210367 conn(0x56218eebfd60 msgr2=0x56218eec0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 -- 172.21.15.60:0/2376210367 shutdown_connections 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.314+0000 7f7be4f8a740 1 -- 172.21.15.60:0/2376210367 wait complete. 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 -- start start 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 Processor -- start 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eecc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56218ec8c890 con 0x56218eec2850 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56218edcb390 con 0x56218eec1f30 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56218eed1220 con 0x56218eece7e0 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.315+0000 7f7be4f8a740 10 client.0 _ll_register_callbacks cb 0x56218edd5500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7be1176640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eecc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32806/0 (socket says 172.21.15.60:32806) 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3111709200 learned_addr learned my addr 172.21.15.60:0/3111709200 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7be0975640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 msgr2=0x56218eecc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eecc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3111709200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 msgr2=0x56218eecb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3111709200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218edc5d60 con 0x56218eec2850 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7be0975640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80c7470 tx=0x7f7bc802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bbb801596f8ee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8049040 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7bc80c7680 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.316+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc80caf00 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8005d50 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 10 client.15408 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56218eedd090 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc80caf00 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7bc810ac80 con 0x56218eec2850 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 20 client.15408 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218eede3e0 0x56218eee07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 4 client.15408 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218ee3b940 con 0x56218eede3e0 2024-07-24T21:46:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.317+0000 7f7be0975640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218eede3e0 0x56218eee07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.318+0000 7f7be0975640 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218eede3e0 0x56218eee07d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275500465db691e0 server_cookie=6a6ccb65ef89b2e5 in_seq=0 out_seq=0 2024-07-24T21:46:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.318+0000 7f7bbbfff640 10 client.15408 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3111709200 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56218ee3b940 con 0x56218eede3e0 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7bbbfff640 10 client.15408 handle_command_reply: tid=0 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 client.15408 shutdown 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 msgr2=0x56218eecbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80c7470 tx=0x7f7bc802a480 comp rx=0 tx=0).stop 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 shutdown_connections 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218eede3e0 0x56218eee07d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eecc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eecbc50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3111709200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 >> 172.21.15.60:0/3111709200 conn(0x56218eebfd60 msgr2=0x56218eec5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7bbbfff640 0 client.15408 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 shutdown_connections 2024-07-24T21:46:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3111709200 wait complete. 2024-07-24T21:46:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.319+0000 7f7be4f8a740 20 client.15408 trim_cache size 0 max 16384 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 Processor -- start 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 -- start start 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eebed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1b2c00 con 0x56218eec2850 2024-07-24T21:46:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1b04d0 con 0x56218eec1f30 2024-07-24T21:46:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1ae320 con 0x56218eece7e0 2024-07-24T21:46:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32808/0 (socket says 172.21.15.60:32808) 2024-07-24T21:46:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7bdbfff640 1 -- 172.21.15.60:0/139102357 learned_addr learned my addr 172.21.15.60:0/139102357 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be0975640 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.338+0000 7f7be1176640 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eebed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 -- 172.21.15.60:0/139102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 msgr2=0x56218eebed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eebed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 -- 172.21.15.60:0/139102357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 msgr2=0x56218eecd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 -- 172.21.15.60:0/139102357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218eec9c10 con 0x56218eec2850 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8003fd0 tx=0x7f7bc810a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c2b38e982d9edd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bc67fc640 1 -- 172.21.15.60:0/139102357 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8049070 con 0x56218eec2850 2024-07-24T21:46:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bc67fc640 1 -- 172.21.15.60:0/139102357 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc80caf00 con 0x56218eec2850 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.339+0000 7f7bc67fc640 1 -- 172.21.15.60:0/139102357 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80cab60 con 0x56218eec2850 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- 172.21.15.60:0/139102357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 msgr2=0x56218eec6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8003fd0 tx=0x7f7bc810a820 comp rx=0 tx=0).stop 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- 172.21.15.60:0/139102357 shutdown_connections 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eebed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eec6080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/139102357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eecd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- 172.21.15.60:0/139102357 >> 172.21.15.60:0/139102357 conn(0x56218eebfd60 msgr2=0x56218eebe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- 172.21.15.60:0/139102357 shutdown_connections 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- 172.21.15.60:0/139102357 wait complete. 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.340+0000 7f7be4f8a740 1 -- start start 2024-07-24T21:46:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 Processor -- start 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eed23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eed2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1ae990 con 0x56218eec2850 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1afe00 con 0x56218eec1f30 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1af060 con 0x56218eece7e0 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be4f8a740 10 client.0 _ll_register_callbacks cb 0x7f7bc810a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32822/0 (socket says 172.21.15.60:32822) 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3127175029 learned_addr learned my addr 172.21.15.60:0/3127175029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be0975640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eed23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.341+0000 7f7be1176640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eed2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 msgr2=0x56218eed2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eed2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3127175029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 msgr2=0x56218eed23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eed23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 -- 172.21.15.60:0/3127175029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218ef48020 con 0x56218eec2850 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bdbfff640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80c7470 tx=0x7f7bc80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dff8a702fec332 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8049070 con 0x56218eec2850 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7bc803bc70 con 0x56218eec2850 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc80caf00 con 0x56218eec2850 2024-07-24T21:46:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc80cad40 con 0x56218eec2850 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7be4f8a740 10 client.15414 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56218ef48020 con 0x56218eec2850 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.342+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc80caf00 con 0x56218eec2850 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7bc80d4010 con 0x56218eec2850 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 10 client.15414 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 20 client.15414 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218ef49560 0x56218ef4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 4 client.15414 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56218ee373e0 con 0x56218ef49560 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7be0975640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218ef49560 0x56218ef4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.343+0000 7f7bbbfff640 10 client.15414 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.344+0000 7f7be0975640 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218ef49560 0x56218ef4b950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfb22836341a3e4 server_cookie=55abcfb62ef5ccbb in_seq=0 out_seq=0 2024-07-24T21:46:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7bbbfff640 1 -- 172.21.15.60:0/3127175029 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56218ee373e0 con 0x56218ef49560 2024-07-24T21:46:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7bbbfff640 10 client.15414 handle_command_reply: tid=0 2024-07-24T21:46:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 client.15414 shutdown 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 msgr2=0x56218eed2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80c7470 tx=0x7f7bc80045a0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 shutdown_connections 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56218ef49560 0x56218ef4b950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56218eece7e0 0x56218eed2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7bbbfff640 0 client.15414 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56218eec2850 0x56218eed2920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 --2- 172.21.15.60:0/3127175029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56218eec1f30 0x56218eed23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.345+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 >> 172.21.15.60:0/3127175029 conn(0x56218eebfd60 msgr2=0x56218eebe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 shutdown_connections 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be4f8a740 1 -- 172.21.15.60:0/3127175029 wait complete. 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be4f8a740 20 client.15414 trim_cache size 0 max 16384 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "e6459d77-fff2-4472-a69b-382adb661e1c", 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:46:32.351 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 msgr2=0x7f7bcc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 0x7f7bcc0b6160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009fd0 tx=0x7f7bd40097e0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 msgr2=0x7f7bdc19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.346+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005940 tx=0x7f7bc80042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 shutdown_connections 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7bcc0b3ca0 0x7f7bcc0b6160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bdc10ee00 0x7f7bdc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bdc101b70 0x7f7bdc19b3d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 --2- 172.21.15.60:0/2379137571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bdc101250 0x7f7bdc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 >> 172.21.15.60:0/2379137571 conn(0x7f7bdc0fd0c0 msgr2=0x7f7bdc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 shutdown_connections 2024-07-24T21:46:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:32.347+0000 7f7be2bd9640 1 -- 172.21.15.60:0/2379137571 wait complete. 2024-07-24T21:46:32.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:32 smithi060 ceph-mon[33000]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:32.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:32 smithi060 ceph-mon[33000]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:32.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:32 smithi123 ceph-mon[37143]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:32.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:32 smithi123 ceph-mon[37143]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:32.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:32 smithi086 ceph-mon[37128]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:32.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:32 smithi086 ceph-mon[37128]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:33.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:46:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:46:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:46:33.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:33 smithi060 ceph-mon[33000]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-07-24T21:46:33.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:33 smithi123 ceph-mon[37143]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-07-24T21:46:33.760 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:46:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:33 smithi086 ceph-mon[37128]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-07-24T21:46:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 -- 172.21.15.60:0/126976271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7801081a0 msgr2=0x7ff7801085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 --2- 172.21.15.60:0/126976271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7801081a0 0x7ff7801085e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402a410 comp rx=0 tx=0).stop 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 -- 172.21.15.60:0/126976271 shutdown_connections 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 --2- 172.21.15.60:0/126976271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff780108b20 0x7ff78010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 --2- 172.21.15.60:0/126976271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7801081a0 0x7ff7801085e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 --2- 172.21.15.60:0/126976271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780102790 0x7ff780102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 -- 172.21.15.60:0/126976271 >> 172.21.15.60:0/126976271 conn(0x7ff7800fe200 msgr2=0x7ff780100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 -- 172.21.15.60:0/126976271 shutdown_connections 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.250+0000 7ff785186640 1 -- 172.21.15.60:0/126976271 wait complete. 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 Processor -- start 2024-07-24T21:46:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 -- start start 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7801081a0 0x7ff78019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 0x7ff7801a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff780112750 con 0x7ff780102790 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7801125d0 con 0x7ff7801081a0 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff785186640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7801128d0 con 0x7ff780108b20 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff77ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff77ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32832/0 (socket says 172.21.15.60:32832) 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff77ed9d640 1 -- 172.21.15.60:0/1248985827 learned_addr learned my addr 172.21.15.60:0/1248985827 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.251+0000 7ff77e59c640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7801081a0 0x7ff78019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77f59e640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 0x7ff7801a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 -- 172.21.15.60:0/1248985827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 msgr2=0x7ff7801a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 0x7ff7801a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 -- 172.21.15.60:0/1248985827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7801081a0 msgr2=0x7ff78019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7801081a0 0x7ff78019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 -- 172.21.15.60:0/1248985827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7801a0f50 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77f59e640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 0x7ff7801a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff77ed9d640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff77000e9f0 tx=0x7ff77000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77000cd80 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7801a1240 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7700054d0 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770010690 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.252+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7801a7900 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.253+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff780102b70 con 0x7ff780102790 2024-07-24T21:46:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.255+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7ff770004e90 con 0x7ff780102790 2024-07-24T21:46:35.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.255+0000 7ff767fff640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 0x7ff7680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:35.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.255+0000 7ff77e59c640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 0x7ff7680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:35.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.255+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff7700e43f0 con 0x7ff780102790 2024-07-24T21:46:35.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.256+0000 7ff77e59c640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 0x7ff7680b6170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff774031440 tx=0x7ff774005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:35.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.260+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff7700ab760 con 0x7ff780102790 2024-07-24T21:46:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.452+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff780062f80 con 0x7ff7680b3cb0 2024-07-24T21:46:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.456+0000 7ff767fff640 1 -- 172.21.15.60:0/1248985827 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258876 (secure 0 0 0) 0x7ff780062f80 con 0x7ff7680b3cb0 2024-07-24T21:46:35.460 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 msgr2=0x7ff7680b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 0x7ff7680b6170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff774031440 tx=0x7ff774005b50 comp rx=0 tx=0).stop 2024-07-24T21:46:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 msgr2=0x7ff78019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff77000e9f0 tx=0x7ff77000edf0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 shutdown_connections 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7680b3cb0 0x7ff7680b6170 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff780108b20 0x7ff7801a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7801081a0 0x7ff78019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 --2- 172.21.15.60:0/1248985827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff780102790 0x7ff78019aea0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 >> 172.21.15.60:0/1248985827 conn(0x7ff7800fe200 msgr2=0x7ff780105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.459+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 shutdown_connections 2024-07-24T21:46:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:35.460+0000 7ff785186640 1 -- 172.21.15.60:0/1248985827 wait complete. 2024-07-24T21:46:35.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:35 smithi060 ceph-mon[33000]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:35 smithi123 ceph-mon[37143]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:35.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:35 smithi086 ceph-mon[37128]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 -- 172.21.15.60:0/2072560860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 msgr2=0x7ff64810e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 --2- 172.21.15.60:0/2072560860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff64810e8e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009900 tx=0x7ff63c02a480 comp rx=0 tx=0).stop 2024-07-24T21:46:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 -- 172.21.15.60:0/2072560860 shutdown_connections 2024-07-24T21:46:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 --2- 172.21.15.60:0/2072560860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64810ee20 0x7ff648111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 --2- 172.21.15.60:0/2072560860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff64810e8e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 --2- 172.21.15.60:0/2072560860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff648101270 0x7ff648101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 -- 172.21.15.60:0/2072560860 >> 172.21.15.60:0/2072560860 conn(0x7ff6480fd0c0 msgr2=0x7ff6480ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 -- 172.21.15.60:0/2072560860 shutdown_connections 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.393+0000 7ff6506d5640 1 -- 172.21.15.60:0/2072560860 wait complete. 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 Processor -- start 2024-07-24T21:46:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 -- start start 2024-07-24T21:46:36.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff648101270 0x7ff6481035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 0x7ff648104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff648114930 con 0x7ff648101b90 2024-07-24T21:46:36.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481147b0 con 0x7ff64810ee20 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff6506d5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff648114ab0 con 0x7ff648101270 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff64dc70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff64dc70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49922/0 (socket says 172.21.15.60:49922) 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.394+0000 7ff64dc70640 1 -- 172.21.15.60:0/3936470529 learned_addr learned my addr 172.21.15.60:0/3936470529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64ec72640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 0x7ff648104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64e471640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff648101270 0x7ff6481035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 -- 172.21.15.60:0/3936470529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff648101270 msgr2=0x7ff6481035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff648101270 0x7ff6481035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 -- 172.21.15.60:0/3936470529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 msgr2=0x7ff648104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 0x7ff648104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 -- 172.21.15.60:0/3936470529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6481ab4b0 con 0x7ff648101b90 2024-07-24T21:46:36.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64ec72640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 0x7ff648104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff64dc70640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005940 tx=0x7ff63c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff6477fe640 1 -- 172.21.15.60:0/3936470529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c03f030 con 0x7ff648101b90 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff6477fe640 1 -- 172.21.15.60:0/3936470529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff63c003aa0 con 0x7ff648101b90 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff6477fe640 1 -- 172.21.15.60:0/3936470529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c035070 con 0x7ff648101b90 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6481ab740 con 0x7ff648101b90 2024-07-24T21:46:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.395+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6481abc70 con 0x7ff648101b90 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff6477fe640 1 -- 172.21.15.60:0/3936470529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7ff63c042030 con 0x7ff648101b90 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 Processor -- start 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 -- start start 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6ead990 con 0x55b8b6f366b0 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6e9ce90 con 0x55b8b6f36fd0 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff652a86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6ea0ed0 con 0x55b8b6f42f60 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64ec72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64ec72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49924/0 (socket says 172.21.15.60:49924) 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64ec72640 1 -- 172.21.15.60:0/3851218246 learned_addr learned my addr 172.21.15.60:0/3851218246 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64e471640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64dc70640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.398+0000 7ff64ec72640 1 -- 172.21.15.60:0/3851218246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 msgr2=0x55b8b6f45350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64ec72640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f45350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64ec72640 1 -- 172.21.15.60:0/3851218246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 msgr2=0x55b8b6f42a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64ec72640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f42a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64ec72640 1 -- 172.21.15.60:0/3851218246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b6e26300 con 0x55b8b6f366b0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff6477fe640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 0x7ff6300b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64e471640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64dc70640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64ec72640 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff640009c20 tx=0x7ff6400117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc573275104544f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff64e471640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 0x7ff6300b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff6477fe640 1 -- 172.21.15.60:0/3936470529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff63c1042b0 con 0x7ff648101b90 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff644ff9640 1 -- 172.21.15.60:0/3851218246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640012450 con 0x55b8b6f366b0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff644ff9640 1 -- 172.21.15.60:0/3851218246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff640012aa0 con 0x55b8b6f366b0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.399+0000 7ff644ff9640 1 -- 172.21.15.60:0/3851218246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64000fa30 con 0x55b8b6f366b0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- 172.21.15.60:0/3851218246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 msgr2=0x55b8b6f36a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff64e471640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 0x7ff6300b61f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55b8b6f353c0 tx=0x7ff638009980 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff640009c20 tx=0x7ff6400117b0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- 172.21.15.60:0/3851218246 shutdown_connections 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 --2- 172.21.15.60:0/3851218246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f36a90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- 172.21.15.60:0/3851218246 >> 172.21.15.60:0/3851218246 conn(0x55b8b6f344e0 msgr2=0x55b8b6f35680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- 172.21.15.60:0/3851218246 shutdown_connections 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- 172.21.15.60:0/3851218246 wait complete. 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 -- start start 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.400+0000 7ff652a86740 1 Processor -- start 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6d00700 con 0x55b8b6f366b0 2024-07-24T21:46:36.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6e3f5d0 con 0x55b8b6f36fd0 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6f45be0 con 0x55b8b6f42f60 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff652a86740 10 client.0 _ll_register_callbacks cb 0x55b8b6e5f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49938/0 (socket says 172.21.15.60:49938) 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 -- 172.21.15.60:0/597303044 learned_addr learned my addr 172.21.15.60:0/597303044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64ec72640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64dc70640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 -- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 msgr2=0x55b8b6f40580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f40580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 -- 172.21.15.60:0/597303044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 msgr2=0x55b8b6f3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 -- 172.21.15.60:0/597303044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b6e32660 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff64e471640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff638011c40 tx=0x7ff638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1546bea95a0744d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.401+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638023020 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff63801f430 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff640012aa0 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff63802e050 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 10 client.15432 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b8b6bfd650 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff640012aa0 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff63801e690 con 0x55b8b6f366b0 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 10 client.15432 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:36.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 20 client.15432 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6f529d0 0x55b8b6f54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 4 client.15432 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8b6eaf8e0 con 0x55b8b6f529d0 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.402+0000 7ff64dc70640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6f529d0 0x55b8b6f54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.403+0000 7ff62dffb640 10 client.15432 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.403+0000 7ff64dc70640 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6f529d0 0x55b8b6f54dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21033f0d343485eb server_cookie=3bd25480ec0b30f3 in_seq=0 out_seq=0 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.403+0000 7ff62dffb640 1 -- 172.21.15.60:0/597303044 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8b6eaf8e0 con 0x55b8b6f529d0 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.403+0000 7ff62dffb640 10 client.15432 handle_command_reply: tid=0 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 client.15432 shutdown 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 msgr2=0x55b8b6f429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff638011c40 tx=0x7ff638004a40 comp rx=0 tx=0).stop 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 shutdown_connections 2024-07-24T21:46:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6f529d0 0x55b8b6f54dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f42f60 0x55b8b6f40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff62dffb640 0 client.15432 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 --2- 172.21.15.60:0/597303044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f429a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 >> 172.21.15.60:0/597303044 conn(0x55b8b6f344e0 msgr2=0x55b8b6f3a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 shutdown_connections 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 1 -- 172.21.15.60:0/597303044 wait complete. 2024-07-24T21:46:36.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.404+0000 7ff652a86740 20 client.15432 trim_cache size 0 max 16384 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.422+0000 7ff652a86740 1 Processor -- start 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.422+0000 7ff652a86740 1 -- start start 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b7020 con 0x55b8b6f42f60 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b48f0 con 0x55b8b6f36fd0 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff652a86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b2740 con 0x55b8b6f366b0 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49942/0 (socket says 172.21.15.60:49942) 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 -- 172.21.15.60:0/2644985815 learned_addr learned my addr 172.21.15.60:0/2644985815 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:36.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64e471640 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64dc70640 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 -- 172.21.15.60:0/2644985815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f366b0 msgr2=0x55b8b6f4efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 -- 172.21.15.60:0/2644985815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 msgr2=0x55b8b6f39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f39f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 -- 172.21.15.60:0/2644985815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b6f3e2f0 con 0x55b8b6f42f60 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.423+0000 7ff64ec72640 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff640009b70 tx=0x7ff64000f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2107ee469cae35 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff62ffff640 1 -- 172.21.15.60:0/2644985815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64002a020 con 0x55b8b6f42f60 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff62ffff640 1 -- 172.21.15.60:0/2644985815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff640009e10 con 0x55b8b6f42f60 2024-07-24T21:46:36.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff62ffff640 1 -- 172.21.15.60:0/2644985815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640004660 con 0x55b8b6f42f60 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 -- 172.21.15.60:0/2644985815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 msgr2=0x55b8b6f5a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff640009b70 tx=0x7ff64000f4f0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 -- 172.21.15.60:0/2644985815 shutdown_connections 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f42f60 0x55b8b6f5a6b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 --2- 172.21.15.60:0/2644985815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 -- 172.21.15.60:0/2644985815 >> 172.21.15.60:0/2644985815 conn(0x55b8b6f344e0 msgr2=0x55b8b6f397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 -- 172.21.15.60:0/2644985815 shutdown_connections 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.424+0000 7ff652a86740 1 -- 172.21.15.60:0/2644985815 wait complete. 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 -- start start 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 Processor -- start 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f42f60 0x55b8b6f44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b2db0 con 0x55b8b6f366b0 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b4220 con 0x55b8b6f42f60 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6481b3480 con 0x55b8b6f36fd0 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff652a86740 10 client.0 _ll_register_callbacks cb 0x7ff638009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff64e471640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff64e471640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49956/0 (socket says 172.21.15.60:49956) 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff64e471640 1 -- 172.21.15.60:0/977795418 learned_addr learned my addr 172.21.15.60:0/977795418 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff64ec72640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f42f60 0x55b8b6f44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.425+0000 7ff64dc70640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 -- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f36fd0 msgr2=0x55b8b6f43ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f43ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 -- 172.21.15.60:0/977795418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f42f60 msgr2=0x55b8b6f44470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f42f60 0x55b8b6f44470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 -- 172.21.15.60:0/977795418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b6f44ca0 con 0x55b8b6f366b0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff64e471640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff63801eca0 tx=0x7ff63801ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be0a2c3c23a1960 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638023040 con 0x55b8b6f366b0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff63801e390 con 0x55b8b6f366b0 2024-07-24T21:46:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff640009e10 con 0x55b8b6f366b0 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff63801e590 con 0x55b8b6f366b0 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff652a86740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b8b6fbd740 con 0x55b8b6f366b0 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.426+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff640009e10 con 0x55b8b6f366b0 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff63801fb50 con 0x55b8b6f366b0 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 20 client.15438 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6fbdf20 0x55b8b6fc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 4 client.15438 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8b6eab380 con 0x55b8b6fbdf20 2024-07-24T21:46:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff64dc70640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6fbdf20 0x55b8b6fc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff64dc70640 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6fbdf20 0x55b8b6fc0340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d611a95a6ac7afbb server_cookie=1c32911006d97d33 in_seq=0 out_seq=0 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.427+0000 7ff62dffb640 10 client.15438 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff62dffb640 1 -- 172.21.15.60:0/977795418 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b8b6eab380 con 0x55b8b6fbdf20 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff62dffb640 10 client.15438 handle_command_reply: tid=0 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 client.15438 shutdown 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 msgr2=0x55b8b6f4b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff63801eca0 tx=0x7ff63801ef50 comp rx=0 tx=0).stop 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 shutdown_connections 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b8b6fbdf20 0x55b8b6fc0340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b6f42f60 0x55b8b6f44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b8b6f36fd0 0x55b8b6f43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 --2- 172.21.15.60:0/977795418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8b6f366b0 0x55b8b6f4b550 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 >> 172.21.15.60:0/977795418 conn(0x55b8b6f344e0 msgr2=0x55b8b6f33550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff62dffb640 0 client.15438 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 shutdown_connections 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 1 -- 172.21.15.60:0/977795418 wait complete. 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.428+0000 7ff652a86740 20 client.15438 trim_cache size 0 max 16384 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stdout: "e6459d77-fff2-4472-a69b-382adb661e1c": { 2024-07-24T21:46:36.435 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "e6459d77-fff2-4472-a69b-382adb661e1c", 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 msgr2=0x7ff6300b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 0x7ff6300b61f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55b8b6f353c0 tx=0x7ff638009980 comp rx=0 tx=0).stop 2024-07-24T21:46:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 msgr2=0x7ff648103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005940 tx=0x7ff63c004270 comp rx=0 tx=0).stop 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 shutdown_connections 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6300b3d30 0x7ff6300b61f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64810ee20 0x7ff648104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff648101b90 0x7ff648103ae0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 --2- 172.21.15.60:0/3936470529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff648101270 0x7ff6481035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 >> 172.21.15.60:0/3936470529 conn(0x7ff6480fd0c0 msgr2=0x7ff648110210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 shutdown_connections 2024-07-24T21:46:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:36.429+0000 7ff6506d5640 1 -- 172.21.15.60:0/3936470529 wait complete. 2024-07-24T21:46:36.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:36 smithi123 ceph-mon[37143]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:36 smithi086 ceph-mon[37128]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:36.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:36 smithi060 ceph-mon[33000]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:37.167 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-07-24T21:46:34.087134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1447428,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1541,"num_read_kb":1003,"num_write":524,"num_write_kb":4363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":886,"ondisk_log_size":886,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405436,"kb_used_data":14624,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536388,"statfs":{"total":1151940427776,"available":1151525261312,"internally_reserved":0,"allocated":14974976,"data_stored":8927356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":405,"num_read_kb":335,"num_write":226,"num_write_kb":2089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003044"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":224,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:32.644904+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:32.644904+0000","last_peered":"2024-07-24T21:46:32.644904+0000","last_clean":"2024-07-24T21:46:32.644904+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:32.644904+0000","last_undegraded":"2024-07-24T21:46:32.644904+0000","last_fullsized":"2024-07-24T21:46:32.644904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":216,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:29.640055+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:29.640055+0000","last_peered":"2024-07-24T21:46:29.640055+0000","last_clean":"2024-07-24T21:46:29.640055+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:29.640055+0000","last_undegraded":"2024-07-24T21:46:29.640055+0000","last_fullsized":"2024-07-24T21:46:29.640055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":441,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:30.641533+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:30.641533+0000","last_peered":"2024-07-24T21:46:30.641533+0000","last_clean":"2024-07-24T21:46:30.641533+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:30.641533+0000","last_undegraded":"2024-07-24T21:46:30.641533+0000","last_fullsized":"2024-07-24T21:46:30.641533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":375,"num_read_kb":252,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":224,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:30.671671+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:30.671671+0000","last_peered":"2024-07-24T21:46:30.671671+0000","last_clean":"2024-07-24T21:46:30.671671+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:30.671671+0000","last_undegraded":"2024-07-24T21:46:30.671671+0000","last_fullsized":"2024-07-24T21:46:30.671671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'60","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:31.643813+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:46:31.643813+0000","last_peered":"2024-07-24T21:46:31.643813+0000","last_clean":"2024-07-24T21:46:31.643813+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:46:31.643813+0000","last_undegraded":"2024-07-24T21:46:31.643813+0000","last_fullsized":"2024-07-24T21:46:31.643813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":219,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:30.670946+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:30.670946+0000","last_peered":"2024-07-24T21:46:30.670946+0000","last_clean":"2024-07-24T21:46:30.670946+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:30.670946+0000","last_undegraded":"2024-07-24T21:46:30.670946+0000","last_fullsized":"2024-07-24T21:46:30.670946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":183,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'97","reported_seq":218,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:32.003849+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:46:32.003849+0000","last_peered":"2024-07-24T21:46:32.003849+0000","last_clean":"2024-07-24T21:46:32.003849+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:46:32.003849+0000","last_undegraded":"2024-07-24T21:46:32.003849+0000","last_fullsized":"2024-07-24T21:46:32.003849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":178,"num_write_kb":1221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:17.695016+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:45:17.695016+0000","last_peered":"2024-07-24T21:45:17.695016+0000","last_clean":"2024-07-24T21:45:17.695016+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:45:17.695016+0000","last_undegraded":"2024-07-24T21:45:17.695016+0000","last_fullsized":"2024-07-24T21:45:17.695016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":856826,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1460,"num_read_kb":934,"num_write":399,"num_write_kb":2211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1617920,"data_stored":1390270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29032,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716120,"statfs":{"total":95995035648,"available":95965306880,"internally_reserved":0,"allocated":2064384,"data_stored":1532503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1376256,"data_stored":874474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":481156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528398,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1986560,"data_stored":1466177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117084,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40460,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704692,"statfs":{"total":95995035648,"available":95953604608,"internally_reserved":0,"allocated":987136,"data_stored":480958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378720,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":546132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705770,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":461578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:37.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:46:37.584 INFO:tasks.cephfs.filesystem:scrub status for tag:e6459d77-fff2-4472-a69b-382adb661e1c - {'path': '/', 'tag': 'e6459d77-fff2-4472-a69b-382adb661e1c', 'options': 'recursive,force'} 2024-07-24T21:46:37.672 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:37.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:37 smithi123 ceph-mon[37143]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-24T21:46:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:37 smithi086 ceph-mon[37128]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-24T21:46:37.795 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:37 smithi060 ceph-mon[33000]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-24T21:46:39.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:38 smithi060 ceph-mon[33000]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:39.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:38 smithi123 ceph-mon[37143]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:39.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 -- 172.21.15.60:0/3885917413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 msgr2=0x7f1ecc102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:39.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3885917413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc102b70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009900 tx=0x7f1ebc02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 -- 172.21.15.60:0/3885917413 shutdown_connections 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3885917413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 0x7f1ecc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3885917413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3885917413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc102b70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 -- 172.21.15.60:0/3885917413 >> 172.21.15.60:0/3885917413 conn(0x7f1ecc0fe200 msgr2=0x7f1ecc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 -- 172.21.15.60:0/3885917413 shutdown_connections 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.275+0000 7f1ed2243640 1 -- 172.21.15.60:0/3885917413 wait complete. 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 Processor -- start 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 -- start start 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 0x7f1ecc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc112720 con 0x7f1ecc102790 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1125a0 con 0x7f1ecc1081a0 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ed2243640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1128a0 con 0x7f1ecc108b20 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49972/0 (socket says 172.21.15.60:49972) 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 -- 172.21.15.60:0/3974420391 learned_addr learned my addr 172.21.15.60:0/3974420391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ec3fff640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecbfff640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 0x7f1ecc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 -- 172.21.15.60:0/3974420391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 msgr2=0x7f1ecc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 0x7f1ecc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 -- 172.21.15.60:0/3974420391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 msgr2=0x7f1ecc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ecb7fe640 1 -- 172.21.15.60:0/3974420391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc19ee60 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.276+0000 7f1ec3fff640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ecb7fe640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005940 tx=0x7f1ebc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc03f030 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc19f0f0 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc003cb0 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc035070 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.277+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a5810 con 0x7f1ecc102790 2024-07-24T21:46:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.278+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc103a40 con 0x7f1ecc102790 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.279+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f1ebc042030 con 0x7f1ecc102790 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.280+0000 7f1ec97fa640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 0x7f1eb80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.280+0000 7f1ec3fff640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 0x7f1eb80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.280+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1ebc104b90 con 0x7f1ecc102790 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.281+0000 7f1ec3fff640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 0x7f1eb80b6170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40058c0 tx=0x7f1eb4005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.283+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1ebc0cbf60 con 0x7f1ecc102790 2024-07-24T21:46:39.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:38 smithi086 ceph-mon[37128]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.474+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ecc062f80 con 0x7f1eb80b3cb0 2024-07-24T21:46:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.478+0000 7f1ec97fa640 1 -- 172.21.15.60:0/3974420391 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7f1ecc062f80 con 0x7f1eb80b3cb0 2024-07-24T21:46:39.482 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:39.483 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 msgr2=0x7f1eb80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 0x7f1eb80b6170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40058c0 tx=0x7f1eb4005850 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 msgr2=0x7f1ecc198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005940 tx=0x7f1ebc0042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 shutdown_connections 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1eb80b3cb0 0x7f1eb80b6170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ecc108b20 0x7f1ecc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ecc1081a0 0x7f1ecc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 --2- 172.21.15.60:0/3974420391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ecc102790 0x7f1ecc198db0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.481+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 >> 172.21.15.60:0/3974420391 conn(0x7f1ecc0fe200 msgr2=0x7f1ecc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.482+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 shutdown_connections 2024-07-24T21:46:39.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:39.482+0000 7f1ed2243640 1 -- 172.21.15.60:0/3974420391 wait complete. 2024-07-24T21:46:40.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:39 smithi060 ceph-mon[33000]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:40.178 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-07-24T21:46:38.088170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1447428,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1567,"num_read_kb":1019,"num_write":524,"num_write_kb":4363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":886,"ondisk_log_size":886,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":406712,"kb_used_data":15780,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124535112,"statfs":{"total":1151940427776,"available":1151523954688,"internally_reserved":0,"allocated":16158720,"data_stored":10107564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":421,"num_read_kb":346,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003142"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":224,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:32.644904+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:32.644904+0000","last_peered":"2024-07-24T21:46:32.644904+0000","last_clean":"2024-07-24T21:46:32.644904+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:32.644904+0000","last_undegraded":"2024-07-24T21:46:32.644904+0000","last_fullsized":"2024-07-24T21:46:32.644904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":221,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:34.646792+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:34.646792+0000","last_peered":"2024-07-24T21:46:34.646792+0000","last_clean":"2024-07-24T21:46:34.646792+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:34.646792+0000","last_undegraded":"2024-07-24T21:46:34.646792+0000","last_fullsized":"2024-07-24T21:46:34.646792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":187,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":452,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:35.647992+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:35.647992+0000","last_peered":"2024-07-24T21:46:35.647992+0000","last_clean":"2024-07-24T21:46:35.647992+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:35.647992+0000","last_undegraded":"2024-07-24T21:46:35.647992+0000","last_fullsized":"2024-07-24T21:46:35.647992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":386,"num_read_kb":258,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":229,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:35.678570+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:35.678570+0000","last_peered":"2024-07-24T21:46:35.678570+0000","last_clean":"2024-07-24T21:46:35.678570+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:35.678570+0000","last_undegraded":"2024-07-24T21:46:35.678570+0000","last_fullsized":"2024-07-24T21:46:35.678570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'60","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:31.643813+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:46:31.643813+0000","last_peered":"2024-07-24T21:46:31.643813+0000","last_clean":"2024-07-24T21:46:31.643813+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:46:31.643813+0000","last_undegraded":"2024-07-24T21:46:31.643813+0000","last_fullsized":"2024-07-24T21:46:31.643813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":224,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:35.678136+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:35.678136+0000","last_peered":"2024-07-24T21:46:35.678136+0000","last_clean":"2024-07-24T21:46:35.678136+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:35.678136+0000","last_undegraded":"2024-07-24T21:46:35.678136+0000","last_fullsized":"2024-07-24T21:46:35.678136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'97","reported_seq":218,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:32.003849+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:46:32.003849+0000","last_peered":"2024-07-24T21:46:32.003849+0000","last_clean":"2024-07-24T21:46:32.003849+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:46:32.003849+0000","last_undegraded":"2024-07-24T21:46:32.003849+0000","last_fullsized":"2024-07-24T21:46:32.003849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":178,"num_write_kb":1221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:17.695016+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:45:17.695016+0000","last_peered":"2024-07-24T21:45:17.695016+0000","last_clean":"2024-07-24T21:45:17.695016+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:45:17.695016+0000","last_undegraded":"2024-07-24T21:45:17.695016+0000","last_fullsized":"2024-07-24T21:45:17.695016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":856826,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1486,"num_read_kb":950,"num_write":399,"num_write_kb":2211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2801664,"data_stored":2570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29092,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716060,"statfs":{"total":95995035648,"available":95965245440,"internally_reserved":0,"allocated":2064384,"data_stored":1532503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1376256,"data_stored":874474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103238,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36676,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708476,"statfs":{"total":95995035648,"available":95957479424,"internally_reserved":0,"allocated":1372160,"data_stored":874372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528399,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1986560,"data_stored":1466177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117085,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703860,"statfs":{"total":95995035648,"available":95952752640,"internally_reserved":0,"allocated":1777664,"data_stored":1267950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378721,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":546132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705771,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":461578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:40.182 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-24T21:46:40.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-24T21:46:40.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:39 smithi123 ceph-mon[37143]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:40.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:39 smithi086 ceph-mon[37128]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:40.450 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:41.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:40 smithi123 ceph-mon[37143]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:41.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:40 smithi060 ceph-mon[33000]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:41.294 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:40 smithi086 ceph-mon[37128]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 -- 172.21.15.60:0/2083643289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c103570 msgr2=0x7fb62c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 --2- 172.21.15.60:0/2083643289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c103570 0x7fb62c103950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb62802a410 comp rx=0 tx=0).stop 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 -- 172.21.15.60:0/2083643289 shutdown_connections 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 --2- 172.21.15.60:0/2083643289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c10cb20 0x7fb62c10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 --2- 172.21.15.60:0/2083643289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103f70 0x7fb62c10c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 --2- 172.21.15.60:0/2083643289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c103570 0x7fb62c103950 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 -- 172.21.15.60:0/2083643289 >> 172.21.15.60:0/2083643289 conn(0x7fb62c0fd0e0 msgr2=0x7fb62c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.046+0000 7fb6346c0640 1 -- 172.21.15.60:0/2083643289 shutdown_connections 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 -- 172.21.15.60:0/2083643289 wait complete. 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 Processor -- start 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 -- start start 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103570 0x7fb62c1013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 0x7fb62c101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1127a0 con 0x7fb62c10cb20 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c112620 con 0x7fb62c103570 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.047+0000 7fb6346c0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c112920 con 0x7fb62c103f70 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49978/0 (socket says 172.21.15.60:49978) 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 -- 172.21.15.60:0/1238296852 learned_addr learned my addr 172.21.15.60:0/1238296852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:42.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb631c5b640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 0x7fb62c101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb63245c640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103570 0x7fb62c1013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 -- 172.21.15.60:0/1238296852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 msgr2=0x7fb62c101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 0x7fb62c101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 -- 172.21.15.60:0/1238296852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103570 msgr2=0x7fb62c1013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103570 0x7fb62c1013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 -- 172.21.15.60:0/1238296852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c1a7010 con 0x7fb62c10cb20 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb631c5b640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 0x7fb62c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb632c5d640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9d0 tx=0x7fb62400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62400cd80 con 0x7fb62c10cb20 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c1a7300 con 0x7fb62c10cb20 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb624005020 con 0x7fb62c10cb20 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240116d0 con 0x7fb62c10cb20 2024-07-24T21:46:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.048+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a7840 con 0x7fb62c10cb20 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.049+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c103950 con 0x7fb62c10cb20 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.051+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7fb6240051d0 con 0x7fb62c10cb20 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.052+0000 7fb6237fe640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 0x7fb6180b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.052+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb6240e44e0 con 0x7fb62c10cb20 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.052+0000 7fb63245c640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 0x7fb6180b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.053+0000 7fb63245c640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 0x7fb6180b6170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb628000c00 tx=0x7fb628031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.055+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb624016030 con 0x7fb62c10cb20 2024-07-24T21:46:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.247+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb62c062f80 con 0x7fb6180b3cb0 2024-07-24T21:46:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.251+0000 7fb6237fe640 1 -- 172.21.15.60:0/1238296852 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7fb62c062f80 con 0x7fb6180b3cb0 2024-07-24T21:46:42.255 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:42.255 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 msgr2=0x7fb6180b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 0x7fb6180b6170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb628000c00 tx=0x7fb628031400 comp rx=0 tx=0).stop 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 msgr2=0x7fb62c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9d0 tx=0x7fb62400ee90 comp rx=0 tx=0).stop 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 shutdown_connections 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb6180b3cb0 0x7fb6180b6170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb62c10cb20 0x7fb62c101e70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c103f70 0x7fb62c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 --2- 172.21.15.60:0/1238296852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62c103570 0x7fb62c1013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 >> 172.21.15.60:0/1238296852 conn(0x7fb62c0fd0e0 msgr2=0x7fb62c1044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 shutdown_connections 2024-07-24T21:46:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:42.254+0000 7fb6346c0640 1 -- 172.21.15.60:0/1238296852 wait complete. 2024-07-24T21:46:42.944 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-07-24T21:46:42.089240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1447428,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1592,"num_read_kb":1029,"num_write":524,"num_write_kb":4363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951862,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":886,"ondisk_log_size":886,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":406712,"kb_used_data":15780,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124535112,"statfs":{"total":1151940427776,"available":1151523954688,"internally_reserved":0,"allocated":16158720,"data_stored":10107564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":416,"num_read_kb":341,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069651+0000","last_change":"2024-07-24T21:43:12.283038+0000","last_active":"2024-07-24T21:45:06.069651+0000","last_peered":"2024-07-24T21:45:06.069651+0000","last_clean":"2024-07-24T21:45:06.069651+0000","last_became_active":"2024-07-24T21:43:12.282734+0000","last_became_peered":"2024-07-24T21:43:12.282734+0000","last_unstale":"2024-07-24T21:45:06.069651+0000","last_undegraded":"2024-07-24T21:45:06.069651+0000","last_fullsized":"2024-07-24T21:45:06.069651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:11:54.386780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073066+0000","last_change":"2024-07-24T21:43:12.281161+0000","last_active":"2024-07-24T21:45:06.073066+0000","last_peered":"2024-07-24T21:45:06.073066+0000","last_clean":"2024-07-24T21:45:06.073066+0000","last_became_active":"2024-07-24T21:43:12.280830+0000","last_became_peered":"2024-07-24T21:43:12.280830+0000","last_unstale":"2024-07-24T21:45:06.073066+0000","last_undegraded":"2024-07-24T21:45:06.073066+0000","last_fullsized":"2024-07-24T21:45:06.073066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:41:07.526680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073052+0000","last_change":"2024-07-24T21:43:12.277660+0000","last_active":"2024-07-24T21:45:06.073052+0000","last_peered":"2024-07-24T21:45:06.073052+0000","last_clean":"2024-07-24T21:45:06.073052+0000","last_became_active":"2024-07-24T21:43:12.277355+0000","last_became_peered":"2024-07-24T21:43:12.277355+0000","last_unstale":"2024-07-24T21:45:06.073052+0000","last_undegraded":"2024-07-24T21:45:06.073052+0000","last_fullsized":"2024-07-24T21:45:06.073052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:21.036959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072144+0000","last_change":"2024-07-24T21:43:12.281905+0000","last_active":"2024-07-24T21:45:06.072144+0000","last_peered":"2024-07-24T21:45:06.072144+0000","last_clean":"2024-07-24T21:45:06.072144+0000","last_became_active":"2024-07-24T21:43:12.281490+0000","last_became_peered":"2024-07-24T21:43:12.281490+0000","last_unstale":"2024-07-24T21:45:06.072144+0000","last_undegraded":"2024-07-24T21:45:06.072144+0000","last_fullsized":"2024-07-24T21:45:06.072144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:39:04.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068353+0000","last_change":"2024-07-24T21:43:12.267146+0000","last_active":"2024-07-24T21:45:06.068353+0000","last_peered":"2024-07-24T21:45:06.068353+0000","last_clean":"2024-07-24T21:45:06.068353+0000","last_became_active":"2024-07-24T21:43:12.266764+0000","last_became_peered":"2024-07-24T21:43:12.266764+0000","last_unstale":"2024-07-24T21:45:06.068353+0000","last_undegraded":"2024-07-24T21:45:06.068353+0000","last_fullsized":"2024-07-24T21:45:06.068353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:27:50.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056305+0000","last_change":"2024-07-24T21:43:37.409407+0000","last_active":"2024-07-24T21:45:06.056305+0000","last_peered":"2024-07-24T21:45:06.056305+0000","last_clean":"2024-07-24T21:45:06.056305+0000","last_became_active":"2024-07-24T21:43:37.408983+0000","last_became_peered":"2024-07-24T21:43:37.408983+0000","last_unstale":"2024-07-24T21:45:06.056305+0000","last_undegraded":"2024-07-24T21:45:06.056305+0000","last_fullsized":"2024-07-24T21:45:06.056305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:51:54.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066088+0000","last_change":"2024-07-24T21:43:12.277931+0000","last_active":"2024-07-24T21:45:06.066088+0000","last_peered":"2024-07-24T21:45:06.066088+0000","last_clean":"2024-07-24T21:45:06.066088+0000","last_became_active":"2024-07-24T21:43:12.277609+0000","last_became_peered":"2024-07-24T21:43:12.277609+0000","last_unstale":"2024-07-24T21:45:06.066088+0000","last_undegraded":"2024-07-24T21:45:06.066088+0000","last_fullsized":"2024-07-24T21:45:06.066088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:16:15.745165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072505+0000","last_change":"2024-07-24T21:43:12.280420+0000","last_active":"2024-07-24T21:45:06.072505+0000","last_peered":"2024-07-24T21:45:06.072505+0000","last_clean":"2024-07-24T21:45:06.072505+0000","last_became_active":"2024-07-24T21:43:12.280128+0000","last_became_peered":"2024-07-24T21:43:12.280128+0000","last_unstale":"2024-07-24T21:45:06.072505+0000","last_undegraded":"2024-07-24T21:45:06.072505+0000","last_fullsized":"2024-07-24T21:45:06.072505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:07:02.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073904+0000","last_change":"2024-07-24T21:43:12.279232+0000","last_active":"2024-07-24T21:45:06.073904+0000","last_peered":"2024-07-24T21:45:06.073904+0000","last_clean":"2024-07-24T21:45:06.073904+0000","last_became_active":"2024-07-24T21:43:12.278902+0000","last_became_peered":"2024-07-24T21:43:12.278902+0000","last_unstale":"2024-07-24T21:45:06.073904+0000","last_undegraded":"2024-07-24T21:45:06.073904+0000","last_fullsized":"2024-07-24T21:45:06.073904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:11:39.915865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.059499+0000","last_change":"2024-07-24T21:43:37.409356+0000","last_active":"2024-07-24T21:45:06.059499+0000","last_peered":"2024-07-24T21:45:06.059499+0000","last_clean":"2024-07-24T21:45:06.059499+0000","last_became_active":"2024-07-24T21:43:37.408858+0000","last_became_peered":"2024-07-24T21:43:37.408858+0000","last_unstale":"2024-07-24T21:45:06.059499+0000","last_undegraded":"2024-07-24T21:45:06.059499+0000","last_fullsized":"2024-07-24T21:45:06.059499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:14:18.627685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074694+0000","last_change":"2024-07-24T21:43:12.270620+0000","last_active":"2024-07-24T21:45:06.074694+0000","last_peered":"2024-07-24T21:45:06.074694+0000","last_clean":"2024-07-24T21:45:06.074694+0000","last_became_active":"2024-07-24T21:43:12.270259+0000","last_became_peered":"2024-07-24T21:43:12.270259+0000","last_unstale":"2024-07-24T21:45:06.074694+0000","last_undegraded":"2024-07-24T21:45:06.074694+0000","last_fullsized":"2024-07-24T21:45:06.074694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:15:52.533405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071788+0000","last_change":"2024-07-24T21:43:12.286967+0000","last_active":"2024-07-24T21:45:06.071788+0000","last_peered":"2024-07-24T21:45:06.071788+0000","last_clean":"2024-07-24T21:45:06.071788+0000","last_became_active":"2024-07-24T21:43:12.286590+0000","last_became_peered":"2024-07-24T21:43:12.286590+0000","last_unstale":"2024-07-24T21:45:06.071788+0000","last_undegraded":"2024-07-24T21:45:06.071788+0000","last_fullsized":"2024-07-24T21:45:06.071788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:04:58.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061719+0000","last_change":"2024-07-24T21:43:12.281459+0000","last_active":"2024-07-24T21:45:06.061719+0000","last_peered":"2024-07-24T21:45:06.061719+0000","last_clean":"2024-07-24T21:45:06.061719+0000","last_became_active":"2024-07-24T21:43:12.281085+0000","last_became_peered":"2024-07-24T21:43:12.281085+0000","last_unstale":"2024-07-24T21:45:06.061719+0000","last_undegraded":"2024-07-24T21:45:06.061719+0000","last_fullsized":"2024-07-24T21:45:06.061719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:56:08.080719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071751+0000","last_change":"2024-07-24T21:43:12.280473+0000","last_active":"2024-07-24T21:45:06.071751+0000","last_peered":"2024-07-24T21:45:06.071751+0000","last_clean":"2024-07-24T21:45:06.071751+0000","last_became_active":"2024-07-24T21:43:12.280180+0000","last_became_peered":"2024-07-24T21:43:12.280180+0000","last_unstale":"2024-07-24T21:45:06.071751+0000","last_undegraded":"2024-07-24T21:45:06.071751+0000","last_fullsized":"2024-07-24T21:45:06.071751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:48:20.931657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067975+0000","last_change":"2024-07-24T21:43:12.286976+0000","last_active":"2024-07-24T21:45:06.067975+0000","last_peered":"2024-07-24T21:45:06.067975+0000","last_clean":"2024-07-24T21:45:06.067975+0000","last_became_active":"2024-07-24T21:43:12.286392+0000","last_became_peered":"2024-07-24T21:43:12.286392+0000","last_unstale":"2024-07-24T21:45:06.067975+0000","last_undegraded":"2024-07-24T21:45:06.067975+0000","last_fullsized":"2024-07-24T21:45:06.067975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:33:10.560083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074449+0000","last_change":"2024-07-24T21:43:12.275373+0000","last_active":"2024-07-24T21:45:06.074449+0000","last_peered":"2024-07-24T21:45:06.074449+0000","last_clean":"2024-07-24T21:45:06.074449+0000","last_became_active":"2024-07-24T21:43:12.275046+0000","last_became_peered":"2024-07-24T21:43:12.275046+0000","last_unstale":"2024-07-24T21:45:06.074449+0000","last_undegraded":"2024-07-24T21:45:06.074449+0000","last_fullsized":"2024-07-24T21:45:06.074449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:26:18.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070676+0000","last_change":"2024-07-24T21:43:12.281947+0000","last_active":"2024-07-24T21:45:06.070676+0000","last_peered":"2024-07-24T21:45:06.070676+0000","last_clean":"2024-07-24T21:45:06.070676+0000","last_became_active":"2024-07-24T21:43:12.281627+0000","last_became_peered":"2024-07-24T21:43:12.281627+0000","last_unstale":"2024-07-24T21:45:06.070676+0000","last_undegraded":"2024-07-24T21:45:06.070676+0000","last_fullsized":"2024-07-24T21:45:06.070676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:07:38.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.056918+0000","last_change":"2024-07-24T21:43:12.283460+0000","last_active":"2024-07-24T21:45:06.056918+0000","last_peered":"2024-07-24T21:45:06.056918+0000","last_clean":"2024-07-24T21:45:06.056918+0000","last_became_active":"2024-07-24T21:43:12.283140+0000","last_became_peered":"2024-07-24T21:43:12.283140+0000","last_unstale":"2024-07-24T21:45:06.056918+0000","last_undegraded":"2024-07-24T21:45:06.056918+0000","last_fullsized":"2024-07-24T21:45:06.056918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:58:11.674351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069221+0000","last_change":"2024-07-24T21:43:12.277158+0000","last_active":"2024-07-24T21:45:06.069221+0000","last_peered":"2024-07-24T21:45:06.069221+0000","last_clean":"2024-07-24T21:45:06.069221+0000","last_became_active":"2024-07-24T21:43:12.276796+0000","last_became_peered":"2024-07-24T21:43:12.276796+0000","last_unstale":"2024-07-24T21:45:06.069221+0000","last_undegraded":"2024-07-24T21:45:06.069221+0000","last_fullsized":"2024-07-24T21:45:06.069221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:28:58.163247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.057279+0000","last_change":"2024-07-24T21:43:12.287041+0000","last_active":"2024-07-24T21:45:06.057279+0000","last_peered":"2024-07-24T21:45:06.057279+0000","last_clean":"2024-07-24T21:45:06.057279+0000","last_became_active":"2024-07-24T21:43:12.286409+0000","last_became_peered":"2024-07-24T21:43:12.286409+0000","last_unstale":"2024-07-24T21:45:06.057279+0000","last_undegraded":"2024-07-24T21:45:06.057279+0000","last_fullsized":"2024-07-24T21:45:06.057279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:04.304335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066554+0000","last_change":"2024-07-24T21:43:12.271720+0000","last_active":"2024-07-24T21:45:06.066554+0000","last_peered":"2024-07-24T21:45:06.066554+0000","last_clean":"2024-07-24T21:45:06.066554+0000","last_became_active":"2024-07-24T21:43:12.271198+0000","last_became_peered":"2024-07-24T21:43:12.271198+0000","last_unstale":"2024-07-24T21:45:06.066554+0000","last_undegraded":"2024-07-24T21:45:06.066554+0000","last_fullsized":"2024-07-24T21:45:06.066554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:22:46.710796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074440+0000","last_change":"2024-07-24T21:43:12.269872+0000","last_active":"2024-07-24T21:45:06.074440+0000","last_peered":"2024-07-24T21:45:06.074440+0000","last_clean":"2024-07-24T21:45:06.074440+0000","last_became_active":"2024-07-24T21:43:12.269514+0000","last_became_peered":"2024-07-24T21:43:12.269514+0000","last_unstale":"2024-07-24T21:45:06.074440+0000","last_undegraded":"2024-07-24T21:45:06.074440+0000","last_fullsized":"2024-07-24T21:45:06.074440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:15:03.869917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070981+0000","last_change":"2024-07-24T21:43:12.275518+0000","last_active":"2024-07-24T21:45:06.070981+0000","last_peered":"2024-07-24T21:45:06.070981+0000","last_clean":"2024-07-24T21:45:06.070981+0000","last_became_active":"2024-07-24T21:43:12.275230+0000","last_became_peered":"2024-07-24T21:43:12.275230+0000","last_unstale":"2024-07-24T21:45:06.070981+0000","last_undegraded":"2024-07-24T21:45:06.070981+0000","last_fullsized":"2024-07-24T21:45:06.070981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:34:34.107348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067719+0000","last_change":"2024-07-24T21:43:37.409363+0000","last_active":"2024-07-24T21:45:06.067719+0000","last_peered":"2024-07-24T21:45:06.067719+0000","last_clean":"2024-07-24T21:45:06.067719+0000","last_became_active":"2024-07-24T21:43:37.408840+0000","last_became_peered":"2024-07-24T21:43:37.408840+0000","last_unstale":"2024-07-24T21:45:06.067719+0000","last_undegraded":"2024-07-24T21:45:06.067719+0000","last_fullsized":"2024-07-24T21:45:06.067719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:00:37.673774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069220+0000","last_change":"2024-07-24T21:43:12.284655+0000","last_active":"2024-07-24T21:45:06.069220+0000","last_peered":"2024-07-24T21:45:06.069220+0000","last_clean":"2024-07-24T21:45:06.069220+0000","last_became_active":"2024-07-24T21:43:12.283973+0000","last_became_peered":"2024-07-24T21:43:12.283973+0000","last_unstale":"2024-07-24T21:45:06.069220+0000","last_undegraded":"2024-07-24T21:45:06.069220+0000","last_fullsized":"2024-07-24T21:45:06.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:22:33.840544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073330+0000","last_change":"2024-07-24T21:43:12.277990+0000","last_active":"2024-07-24T21:45:06.073330+0000","last_peered":"2024-07-24T21:45:06.073330+0000","last_clean":"2024-07-24T21:45:06.073330+0000","last_became_active":"2024-07-24T21:43:12.277329+0000","last_became_peered":"2024-07-24T21:43:12.277329+0000","last_unstale":"2024-07-24T21:45:06.073330+0000","last_undegraded":"2024-07-24T21:45:06.073330+0000","last_fullsized":"2024-07-24T21:45:06.073330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:45:39.291378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.055135+0000","last_change":"2024-07-24T21:43:12.277762+0000","last_active":"2024-07-24T21:45:06.055135+0000","last_peered":"2024-07-24T21:45:06.055135+0000","last_clean":"2024-07-24T21:45:06.055135+0000","last_became_active":"2024-07-24T21:43:12.277385+0000","last_became_peered":"2024-07-24T21:43:12.277385+0000","last_unstale":"2024-07-24T21:45:06.055135+0000","last_undegraded":"2024-07-24T21:45:06.055135+0000","last_fullsized":"2024-07-24T21:45:06.055135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:22:17.763901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066066+0000","last_change":"2024-07-24T21:43:12.284243+0000","last_active":"2024-07-24T21:45:06.066066+0000","last_peered":"2024-07-24T21:45:06.066066+0000","last_clean":"2024-07-24T21:45:06.066066+0000","last_became_active":"2024-07-24T21:43:12.283891+0000","last_became_peered":"2024-07-24T21:43:12.283891+0000","last_unstale":"2024-07-24T21:45:06.066066+0000","last_undegraded":"2024-07-24T21:45:06.066066+0000","last_fullsized":"2024-07-24T21:45:06.066066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:59:53.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.068475+0000","last_change":"2024-07-24T21:43:12.286796+0000","last_active":"2024-07-24T21:45:06.068475+0000","last_peered":"2024-07-24T21:45:06.068475+0000","last_clean":"2024-07-24T21:45:06.068475+0000","last_became_active":"2024-07-24T21:43:12.286339+0000","last_became_peered":"2024-07-24T21:43:12.286339+0000","last_unstale":"2024-07-24T21:45:06.068475+0000","last_undegraded":"2024-07-24T21:45:06.068475+0000","last_fullsized":"2024-07-24T21:45:06.068475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:39:26.189922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066801+0000","last_change":"2024-07-24T21:43:12.278328+0000","last_active":"2024-07-24T21:45:06.066801+0000","last_peered":"2024-07-24T21:45:06.066801+0000","last_clean":"2024-07-24T21:45:06.066801+0000","last_became_active":"2024-07-24T21:43:12.277906+0000","last_became_peered":"2024-07-24T21:43:12.277906+0000","last_unstale":"2024-07-24T21:45:06.066801+0000","last_undegraded":"2024-07-24T21:45:06.066801+0000","last_fullsized":"2024-07-24T21:45:06.066801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:00:01.904641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064648+0000","last_change":"2024-07-24T21:43:37.875306+0000","last_active":"2024-07-24T21:45:06.064648+0000","last_peered":"2024-07-24T21:45:06.064648+0000","last_clean":"2024-07-24T21:45:06.064648+0000","last_became_active":"2024-07-24T21:43:37.874863+0000","last_became_peered":"2024-07-24T21:43:37.874863+0000","last_unstale":"2024-07-24T21:45:06.064648+0000","last_undegraded":"2024-07-24T21:45:06.064648+0000","last_fullsized":"2024-07-24T21:45:06.064648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:09:25.727755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066012+0000","last_change":"2024-07-24T21:43:12.278548+0000","last_active":"2024-07-24T21:45:06.066012+0000","last_peered":"2024-07-24T21:45:06.066012+0000","last_clean":"2024-07-24T21:45:06.066012+0000","last_became_active":"2024-07-24T21:43:12.278261+0000","last_became_peered":"2024-07-24T21:43:12.278261+0000","last_unstale":"2024-07-24T21:45:06.066012+0000","last_undegraded":"2024-07-24T21:45:06.066012+0000","last_fullsized":"2024-07-24T21:45:06.066012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:41:53.366720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070566+0000","last_change":"2024-07-24T21:43:12.274739+0000","last_active":"2024-07-24T21:45:06.070566+0000","last_peered":"2024-07-24T21:45:06.070566+0000","last_clean":"2024-07-24T21:45:06.070566+0000","last_became_active":"2024-07-24T21:43:12.274423+0000","last_became_peered":"2024-07-24T21:43:12.274423+0000","last_unstale":"2024-07-24T21:45:06.070566+0000","last_undegraded":"2024-07-24T21:45:06.070566+0000","last_fullsized":"2024-07-24T21:45:06.070566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:55:58.935089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":229,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:37.651373+0000","last_change":"2024-07-24T21:43:09.256236+0000","last_active":"2024-07-24T21:46:37.651373+0000","last_peered":"2024-07-24T21:46:37.651373+0000","last_clean":"2024-07-24T21:46:37.651373+0000","last_became_active":"2024-07-24T21:43:09.255921+0000","last_became_peered":"2024-07-24T21:43:09.255921+0000","last_unstale":"2024-07-24T21:46:37.651373+0000","last_undegraded":"2024-07-24T21:46:37.651373+0000","last_fullsized":"2024-07-24T21:46:37.651373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:28:37.328677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070396+0000","last_change":"2024-07-24T21:43:12.279460+0000","last_active":"2024-07-24T21:45:06.070396+0000","last_peered":"2024-07-24T21:45:06.070396+0000","last_clean":"2024-07-24T21:45:06.070396+0000","last_became_active":"2024-07-24T21:43:12.279140+0000","last_became_peered":"2024-07-24T21:43:12.279140+0000","last_unstale":"2024-07-24T21:45:06.070396+0000","last_undegraded":"2024-07-24T21:45:06.070396+0000","last_fullsized":"2024-07-24T21:45:06.070396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:24.029627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.974167+0000","last_change":"2024-07-24T21:43:09.251659+0000","last_active":"2024-07-24T21:44:37.974167+0000","last_peered":"2024-07-24T21:44:37.974167+0000","last_clean":"2024-07-24T21:44:37.974167+0000","last_became_active":"2024-07-24T21:43:09.251133+0000","last_became_peered":"2024-07-24T21:43:09.251133+0000","last_unstale":"2024-07-24T21:44:37.974167+0000","last_undegraded":"2024-07-24T21:44:37.974167+0000","last_fullsized":"2024-07-24T21:44:37.974167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:47:26.679435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.739546+0000","last_change":"2024-07-24T21:43:09.249656+0000","last_active":"2024-07-24T21:44:37.739546+0000","last_peered":"2024-07-24T21:44:37.739546+0000","last_clean":"2024-07-24T21:44:37.739546+0000","last_became_active":"2024-07-24T21:43:09.249352+0000","last_became_peered":"2024-07-24T21:43:09.249352+0000","last_unstale":"2024-07-24T21:44:37.739546+0000","last_undegraded":"2024-07-24T21:44:37.739546+0000","last_fullsized":"2024-07-24T21:44:37.739546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:27:34.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070647+0000","last_change":"2024-07-24T21:43:12.275043+0000","last_active":"2024-07-24T21:45:06.070647+0000","last_peered":"2024-07-24T21:45:06.070647+0000","last_clean":"2024-07-24T21:45:06.070647+0000","last_became_active":"2024-07-24T21:43:12.274662+0000","last_became_peered":"2024-07-24T21:43:12.274662+0000","last_unstale":"2024-07-24T21:45:06.070647+0000","last_undegraded":"2024-07-24T21:45:06.070647+0000","last_fullsized":"2024-07-24T21:45:06.070647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:33:54.677259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081339+0000","last_change":"2024-07-24T21:43:09.252595+0000","last_active":"2024-07-24T21:44:38.081339+0000","last_peered":"2024-07-24T21:44:38.081339+0000","last_clean":"2024-07-24T21:44:38.081339+0000","last_became_active":"2024-07-24T21:43:09.252271+0000","last_became_peered":"2024-07-24T21:43:09.252271+0000","last_unstale":"2024-07-24T21:44:38.081339+0000","last_undegraded":"2024-07-24T21:44:38.081339+0000","last_fullsized":"2024-07-24T21:44:38.081339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:08:43.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053892+0000","last_change":"2024-07-24T21:43:12.279387+0000","last_active":"2024-07-24T21:45:06.053892+0000","last_peered":"2024-07-24T21:45:06.053892+0000","last_clean":"2024-07-24T21:45:06.053892+0000","last_became_active":"2024-07-24T21:43:12.279073+0000","last_became_peered":"2024-07-24T21:43:12.279073+0000","last_unstale":"2024-07-24T21:45:06.053892+0000","last_undegraded":"2024-07-24T21:45:06.053892+0000","last_fullsized":"2024-07-24T21:45:06.053892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:55:41.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":226,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:39.653696+0000","last_change":"2024-07-24T21:43:09.252023+0000","last_active":"2024-07-24T21:46:39.653696+0000","last_peered":"2024-07-24T21:46:39.653696+0000","last_clean":"2024-07-24T21:46:39.653696+0000","last_became_active":"2024-07-24T21:43:09.251707+0000","last_became_peered":"2024-07-24T21:43:09.251707+0000","last_unstale":"2024-07-24T21:46:39.653696+0000","last_undegraded":"2024-07-24T21:46:39.653696+0000","last_fullsized":"2024-07-24T21:46:39.653696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:34:45.357766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074072+0000","last_change":"2024-07-24T21:43:12.281071+0000","last_active":"2024-07-24T21:45:06.074072+0000","last_peered":"2024-07-24T21:45:06.074072+0000","last_clean":"2024-07-24T21:45:06.074072+0000","last_became_active":"2024-07-24T21:43:12.280380+0000","last_became_peered":"2024-07-24T21:43:12.280380+0000","last_unstale":"2024-07-24T21:45:06.074072+0000","last_undegraded":"2024-07-24T21:45:06.074072+0000","last_fullsized":"2024-07-24T21:45:06.074072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:12:18.825622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741575+0000","last_change":"2024-07-24T21:43:09.255848+0000","last_active":"2024-07-24T21:44:37.741575+0000","last_peered":"2024-07-24T21:44:37.741575+0000","last_clean":"2024-07-24T21:44:37.741575+0000","last_became_active":"2024-07-24T21:43:09.255456+0000","last_became_peered":"2024-07-24T21:43:09.255456+0000","last_unstale":"2024-07-24T21:44:37.741575+0000","last_undegraded":"2024-07-24T21:44:37.741575+0000","last_fullsized":"2024-07-24T21:44:37.741575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:59:14.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.062953+0000","last_change":"2024-07-24T21:43:37.404913+0000","last_active":"2024-07-24T21:45:06.062953+0000","last_peered":"2024-07-24T21:45:06.062953+0000","last_clean":"2024-07-24T21:45:06.062953+0000","last_became_active":"2024-07-24T21:43:37.404381+0000","last_became_peered":"2024-07-24T21:43:37.404381+0000","last_unstale":"2024-07-24T21:45:06.062953+0000","last_undegraded":"2024-07-24T21:45:06.062953+0000","last_fullsized":"2024-07-24T21:45:06.062953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:32:42.151301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081454+0000","last_change":"2024-07-24T21:43:09.377240+0000","last_active":"2024-07-24T21:44:38.081454+0000","last_peered":"2024-07-24T21:44:38.081454+0000","last_clean":"2024-07-24T21:44:38.081454+0000","last_became_active":"2024-07-24T21:43:09.376758+0000","last_became_peered":"2024-07-24T21:43:09.376758+0000","last_unstale":"2024-07-24T21:44:38.081454+0000","last_undegraded":"2024-07-24T21:44:38.081454+0000","last_fullsized":"2024-07-24T21:44:38.081454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:46:22.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074312+0000","last_change":"2024-07-24T21:43:12.281790+0000","last_active":"2024-07-24T21:45:06.074312+0000","last_peered":"2024-07-24T21:45:06.074312+0000","last_clean":"2024-07-24T21:45:06.074312+0000","last_became_active":"2024-07-24T21:43:12.281477+0000","last_became_peered":"2024-07-24T21:43:12.281477+0000","last_unstale":"2024-07-24T21:45:06.074312+0000","last_undegraded":"2024-07-24T21:45:06.074312+0000","last_fullsized":"2024-07-24T21:45:06.074312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:33:56.245609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":462,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:40.654977+0000","last_change":"2024-07-24T21:43:09.254497+0000","last_active":"2024-07-24T21:46:40.654977+0000","last_peered":"2024-07-24T21:46:40.654977+0000","last_clean":"2024-07-24T21:46:40.654977+0000","last_became_active":"2024-07-24T21:43:09.254136+0000","last_became_peered":"2024-07-24T21:43:09.254136+0000","last_unstale":"2024-07-24T21:46:40.654977+0000","last_undegraded":"2024-07-24T21:46:40.654977+0000","last_fullsized":"2024-07-24T21:46:40.654977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:26.494035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65692,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":396,"num_read_kb":263,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072277+0000","last_change":"2024-07-24T21:43:12.273762+0000","last_active":"2024-07-24T21:45:06.072277+0000","last_peered":"2024-07-24T21:45:06.072277+0000","last_clean":"2024-07-24T21:45:06.072277+0000","last_became_active":"2024-07-24T21:43:12.273453+0000","last_became_peered":"2024-07-24T21:43:12.273453+0000","last_unstale":"2024-07-24T21:45:06.072277+0000","last_undegraded":"2024-07-24T21:45:06.072277+0000","last_fullsized":"2024-07-24T21:45:06.072277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:54:58.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742460+0000","last_change":"2024-07-24T21:43:09.254597+0000","last_active":"2024-07-24T21:44:37.742460+0000","last_peered":"2024-07-24T21:44:37.742460+0000","last_clean":"2024-07-24T21:44:37.742460+0000","last_became_active":"2024-07-24T21:43:09.254234+0000","last_became_peered":"2024-07-24T21:43:09.254234+0000","last_unstale":"2024-07-24T21:44:37.742460+0000","last_undegraded":"2024-07-24T21:44:37.742460+0000","last_fullsized":"2024-07-24T21:44:37.742460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:49:38.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063874+0000","last_change":"2024-07-24T21:43:37.409388+0000","last_active":"2024-07-24T21:45:06.063874+0000","last_peered":"2024-07-24T21:45:06.063874+0000","last_clean":"2024-07-24T21:45:06.063874+0000","last_became_active":"2024-07-24T21:43:37.408990+0000","last_became_peered":"2024-07-24T21:43:37.408990+0000","last_unstale":"2024-07-24T21:45:06.063874+0000","last_undegraded":"2024-07-24T21:45:06.063874+0000","last_fullsized":"2024-07-24T21:45:06.063874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:26:12.927908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081182+0000","last_change":"2024-07-24T21:43:09.255531+0000","last_active":"2024-07-24T21:44:38.081182+0000","last_peered":"2024-07-24T21:44:38.081182+0000","last_clean":"2024-07-24T21:44:38.081182+0000","last_became_active":"2024-07-24T21:43:09.254937+0000","last_became_peered":"2024-07-24T21:43:09.254937+0000","last_unstale":"2024-07-24T21:44:38.081182+0000","last_undegraded":"2024-07-24T21:44:38.081182+0000","last_fullsized":"2024-07-24T21:44:38.081182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:45:20.294420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066623+0000","last_change":"2024-07-24T21:43:12.282033+0000","last_active":"2024-07-24T21:45:06.066623+0000","last_peered":"2024-07-24T21:45:06.066623+0000","last_clean":"2024-07-24T21:45:06.066623+0000","last_became_active":"2024-07-24T21:43:12.281720+0000","last_became_peered":"2024-07-24T21:43:12.281720+0000","last_unstale":"2024-07-24T21:45:06.066623+0000","last_undegraded":"2024-07-24T21:45:06.066623+0000","last_fullsized":"2024-07-24T21:45:06.066623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:05:33.532858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":234,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:40.685868+0000","last_change":"2024-07-24T21:43:09.252238+0000","last_active":"2024-07-24T21:46:40.685868+0000","last_peered":"2024-07-24T21:46:40.685868+0000","last_clean":"2024-07-24T21:46:40.685868+0000","last_became_active":"2024-07-24T21:43:09.251879+0000","last_became_peered":"2024-07-24T21:43:09.251879+0000","last_unstale":"2024-07-24T21:46:40.685868+0000","last_undegraded":"2024-07-24T21:46:40.685868+0000","last_fullsized":"2024-07-24T21:46:40.685868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:54:03.031806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.066351+0000","last_change":"2024-07-24T21:43:12.280153+0000","last_active":"2024-07-24T21:45:06.066351+0000","last_peered":"2024-07-24T21:45:06.066351+0000","last_clean":"2024-07-24T21:45:06.066351+0000","last_became_active":"2024-07-24T21:43:12.279732+0000","last_became_peered":"2024-07-24T21:43:12.279732+0000","last_unstale":"2024-07-24T21:45:06.066351+0000","last_undegraded":"2024-07-24T21:45:06.066351+0000","last_fullsized":"2024-07-24T21:45:06.066351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:26:43.525720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.740735+0000","last_change":"2024-07-24T21:43:09.376741+0000","last_active":"2024-07-24T21:44:37.740735+0000","last_peered":"2024-07-24T21:44:37.740735+0000","last_clean":"2024-07-24T21:44:37.740735+0000","last_became_active":"2024-07-24T21:43:09.376292+0000","last_became_peered":"2024-07-24T21:43:09.376292+0000","last_unstale":"2024-07-24T21:44:37.740735+0000","last_undegraded":"2024-07-24T21:44:37.740735+0000","last_fullsized":"2024-07-24T21:44:37.740735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T04:07:40.365867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067366+0000","last_change":"2024-07-24T21:43:12.280454+0000","last_active":"2024-07-24T21:45:06.067366+0000","last_peered":"2024-07-24T21:45:06.067366+0000","last_clean":"2024-07-24T21:45:06.067366+0000","last_became_active":"2024-07-24T21:43:12.279934+0000","last_became_peered":"2024-07-24T21:43:12.279934+0000","last_unstale":"2024-07-24T21:45:06.067366+0000","last_undegraded":"2024-07-24T21:45:06.067366+0000","last_fullsized":"2024-07-24T21:45:06.067366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:48:25.138518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741224+0000","last_change":"2024-07-24T21:43:09.251544+0000","last_active":"2024-07-24T21:44:37.741224+0000","last_peered":"2024-07-24T21:44:37.741224+0000","last_clean":"2024-07-24T21:44:37.741224+0000","last_became_active":"2024-07-24T21:43:09.251121+0000","last_became_peered":"2024-07-24T21:43:09.251121+0000","last_unstale":"2024-07-24T21:44:37.741224+0000","last_undegraded":"2024-07-24T21:44:37.741224+0000","last_fullsized":"2024-07-24T21:44:37.741224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:03:09.034149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.053332+0000","last_change":"2024-07-24T21:43:12.286671+0000","last_active":"2024-07-24T21:45:06.053332+0000","last_peered":"2024-07-24T21:45:06.053332+0000","last_clean":"2024-07-24T21:45:06.053332+0000","last_became_active":"2024-07-24T21:43:12.286304+0000","last_became_peered":"2024-07-24T21:43:12.286304+0000","last_unstale":"2024-07-24T21:45:06.053332+0000","last_undegraded":"2024-07-24T21:45:06.053332+0000","last_fullsized":"2024-07-24T21:45:06.053332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:15:30.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.895178+0000","last_change":"2024-07-24T21:44:37.894744+0000","last_active":"2024-07-24T21:44:37.895178+0000","last_peered":"2024-07-24T21:44:37.895178+0000","last_clean":"2024-07-24T21:44:37.895178+0000","last_became_active":"2024-07-24T21:44:37.892338+0000","last_became_peered":"2024-07-24T21:44:37.892338+0000","last_unstale":"2024-07-24T21:44:37.895178+0000","last_undegraded":"2024-07-24T21:44:37.895178+0000","last_fullsized":"2024-07-24T21:44:37.895178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:12:56.611126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.074416+0000","last_change":"2024-07-24T21:43:12.275365+0000","last_active":"2024-07-24T21:45:06.074416+0000","last_peered":"2024-07-24T21:45:06.074416+0000","last_clean":"2024-07-24T21:45:06.074416+0000","last_became_active":"2024-07-24T21:43:12.275073+0000","last_became_peered":"2024-07-24T21:43:12.275073+0000","last_unstale":"2024-07-24T21:45:06.074416+0000","last_undegraded":"2024-07-24T21:45:06.074416+0000","last_fullsized":"2024-07-24T21:45:06.074416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:56:10.194934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'60","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:31.643813+0000","last_change":"2024-07-24T21:43:09.249347+0000","last_active":"2024-07-24T21:46:31.643813+0000","last_peered":"2024-07-24T21:46:31.643813+0000","last_clean":"2024-07-24T21:46:31.643813+0000","last_became_active":"2024-07-24T21:43:09.248942+0000","last_became_peered":"2024-07-24T21:43:09.248942+0000","last_unstale":"2024-07-24T21:46:31.643813+0000","last_undegraded":"2024-07-24T21:46:31.643813+0000","last_fullsized":"2024-07-24T21:46:31.643813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T03:24:09.557778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.067179+0000","last_change":"2024-07-24T21:43:12.278002+0000","last_active":"2024-07-24T21:45:06.067179+0000","last_peered":"2024-07-24T21:45:06.067179+0000","last_clean":"2024-07-24T21:45:06.067179+0000","last_became_active":"2024-07-24T21:43:12.277670+0000","last_became_peered":"2024-07-24T21:43:12.277670+0000","last_unstale":"2024-07-24T21:45:06.067179+0000","last_undegraded":"2024-07-24T21:45:06.067179+0000","last_fullsized":"2024-07-24T21:45:06.067179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:59:42.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742642+0000","last_change":"2024-07-24T21:43:09.666747+0000","last_active":"2024-07-24T21:44:37.742642+0000","last_peered":"2024-07-24T21:44:37.742642+0000","last_clean":"2024-07-24T21:44:37.742642+0000","last_became_active":"2024-07-24T21:43:09.666311+0000","last_became_peered":"2024-07-24T21:43:09.666311+0000","last_unstale":"2024-07-24T21:44:37.742642+0000","last_undegraded":"2024-07-24T21:44:37.742642+0000","last_fullsized":"2024-07-24T21:44:37.742642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:39:04.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073467+0000","last_change":"2024-07-24T21:43:12.276166+0000","last_active":"2024-07-24T21:45:06.073467+0000","last_peered":"2024-07-24T21:45:06.073467+0000","last_clean":"2024-07-24T21:45:06.073467+0000","last_became_active":"2024-07-24T21:43:12.275878+0000","last_became_peered":"2024-07-24T21:43:12.275878+0000","last_unstale":"2024-07-24T21:45:06.073467+0000","last_undegraded":"2024-07-24T21:45:06.073467+0000","last_fullsized":"2024-07-24T21:45:06.073467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:42:30.959238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741348+0000","last_change":"2024-07-24T21:41:52.253751+0000","last_active":"2024-07-24T21:44:37.741348+0000","last_peered":"2024-07-24T21:44:37.741348+0000","last_clean":"2024-07-24T21:44:37.741348+0000","last_became_active":"2024-07-24T21:41:52.248095+0000","last_became_peered":"2024-07-24T21:41:52.248095+0000","last_unstale":"2024-07-24T21:44:37.741348+0000","last_undegraded":"2024-07-24T21:44:37.741348+0000","last_fullsized":"2024-07-24T21:44:37.741348+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:38:46.282064+0000","last_clean_scrub_stamp":"2024-07-24T21:38:46.282064+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:46.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080855+0000","last_change":"2024-07-24T21:43:09.253404+0000","last_active":"2024-07-24T21:44:38.080855+0000","last_peered":"2024-07-24T21:44:38.080855+0000","last_clean":"2024-07-24T21:44:38.080855+0000","last_became_active":"2024-07-24T21:43:09.253090+0000","last_became_peered":"2024-07-24T21:43:09.253090+0000","last_unstale":"2024-07-24T21:44:38.080855+0000","last_undegraded":"2024-07-24T21:44:38.080855+0000","last_fullsized":"2024-07-24T21:44:38.080855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:58:58.877744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072330+0000","last_change":"2024-07-24T21:43:12.274097+0000","last_active":"2024-07-24T21:45:06.072330+0000","last_peered":"2024-07-24T21:45:06.072330+0000","last_clean":"2024-07-24T21:45:06.072330+0000","last_became_active":"2024-07-24T21:43:12.273654+0000","last_became_peered":"2024-07-24T21:43:12.273654+0000","last_unstale":"2024-07-24T21:45:06.072330+0000","last_undegraded":"2024-07-24T21:45:06.072330+0000","last_fullsized":"2024-07-24T21:45:06.072330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:05:58.909321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741293+0000","last_change":"2024-07-24T21:43:09.250392+0000","last_active":"2024-07-24T21:44:37.741293+0000","last_peered":"2024-07-24T21:44:37.741293+0000","last_clean":"2024-07-24T21:44:37.741293+0000","last_became_active":"2024-07-24T21:43:09.249916+0000","last_became_peered":"2024-07-24T21:43:09.249916+0000","last_unstale":"2024-07-24T21:44:37.741293+0000","last_undegraded":"2024-07-24T21:44:37.741293+0000","last_fullsized":"2024-07-24T21:44:37.741293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:06:45.771881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.064005+0000","last_change":"2024-07-24T21:43:12.280066+0000","last_active":"2024-07-24T21:45:06.064005+0000","last_peered":"2024-07-24T21:45:06.064005+0000","last_clean":"2024-07-24T21:45:06.064005+0000","last_became_active":"2024-07-24T21:43:12.279704+0000","last_became_peered":"2024-07-24T21:43:12.279704+0000","last_unstale":"2024-07-24T21:45:06.064005+0000","last_undegraded":"2024-07-24T21:45:06.064005+0000","last_fullsized":"2024-07-24T21:45:06.064005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:11:30.797867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":224,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:35.678136+0000","last_change":"2024-07-24T21:43:09.375718+0000","last_active":"2024-07-24T21:46:35.678136+0000","last_peered":"2024-07-24T21:46:35.678136+0000","last_clean":"2024-07-24T21:46:35.678136+0000","last_became_active":"2024-07-24T21:43:09.375358+0000","last_became_peered":"2024-07-24T21:43:09.375358+0000","last_unstale":"2024-07-24T21:46:35.678136+0000","last_undegraded":"2024-07-24T21:46:35.678136+0000","last_fullsized":"2024-07-24T21:46:35.678136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:21:36.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071878+0000","last_change":"2024-07-24T21:43:37.405658+0000","last_active":"2024-07-24T21:45:06.071878+0000","last_peered":"2024-07-24T21:45:06.071878+0000","last_clean":"2024-07-24T21:45:06.071878+0000","last_became_active":"2024-07-24T21:43:37.405170+0000","last_became_peered":"2024-07-24T21:43:37.405170+0000","last_unstale":"2024-07-24T21:45:06.071878+0000","last_undegraded":"2024-07-24T21:45:06.071878+0000","last_fullsized":"2024-07-24T21:45:06.071878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:28.823856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741433+0000","last_change":"2024-07-24T21:43:09.249089+0000","last_active":"2024-07-24T21:44:37.741433+0000","last_peered":"2024-07-24T21:44:37.741433+0000","last_clean":"2024-07-24T21:44:37.741433+0000","last_became_active":"2024-07-24T21:43:09.248303+0000","last_became_peered":"2024-07-24T21:43:09.248303+0000","last_unstale":"2024-07-24T21:44:37.741433+0000","last_undegraded":"2024-07-24T21:44:37.741433+0000","last_fullsized":"2024-07-24T21:44:37.741433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:10:48.051840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069211+0000","last_change":"2024-07-24T21:43:12.281939+0000","last_active":"2024-07-24T21:45:06.069211+0000","last_peered":"2024-07-24T21:45:06.069211+0000","last_clean":"2024-07-24T21:45:06.069211+0000","last_became_active":"2024-07-24T21:43:12.281600+0000","last_became_peered":"2024-07-24T21:43:12.281600+0000","last_unstale":"2024-07-24T21:45:06.069211+0000","last_undegraded":"2024-07-24T21:45:06.069211+0000","last_fullsized":"2024-07-24T21:45:06.069211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:47:18.718785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.080740+0000","last_change":"2024-07-24T21:43:09.376693+0000","last_active":"2024-07-24T21:44:38.080740+0000","last_peered":"2024-07-24T21:44:38.080740+0000","last_clean":"2024-07-24T21:44:38.080740+0000","last_became_active":"2024-07-24T21:43:09.376303+0000","last_became_peered":"2024-07-24T21:43:09.376303+0000","last_unstale":"2024-07-24T21:44:38.080740+0000","last_undegraded":"2024-07-24T21:44:38.080740+0000","last_fullsized":"2024-07-24T21:44:38.080740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:05:43.333091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.052002+0000","last_change":"2024-07-24T21:43:12.273209+0000","last_active":"2024-07-24T21:45:06.052002+0000","last_peered":"2024-07-24T21:45:06.052002+0000","last_clean":"2024-07-24T21:45:06.052002+0000","last_became_active":"2024-07-24T21:43:12.272905+0000","last_became_peered":"2024-07-24T21:43:12.272905+0000","last_unstale":"2024-07-24T21:45:06.052002+0000","last_undegraded":"2024-07-24T21:45:06.052002+0000","last_fullsized":"2024-07-24T21:45:06.052002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:23:47.259284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.743068+0000","last_change":"2024-07-24T21:43:37.411899+0000","last_active":"2024-07-24T21:44:37.743068+0000","last_peered":"2024-07-24T21:44:37.743068+0000","last_clean":"2024-07-24T21:44:37.743068+0000","last_became_active":"2024-07-24T21:43:37.408469+0000","last_became_peered":"2024-07-24T21:43:37.408469+0000","last_unstale":"2024-07-24T21:44:37.743068+0000","last_undegraded":"2024-07-24T21:44:37.743068+0000","last_fullsized":"2024-07-24T21:44:37.743068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T21:58:31.232605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.061123+0000","last_change":"2024-07-24T21:43:12.273138+0000","last_active":"2024-07-24T21:45:06.061123+0000","last_peered":"2024-07-24T21:45:06.061123+0000","last_clean":"2024-07-24T21:45:06.061123+0000","last_became_active":"2024-07-24T21:43:12.272800+0000","last_became_peered":"2024-07-24T21:43:12.272800+0000","last_unstale":"2024-07-24T21:45:06.061123+0000","last_undegraded":"2024-07-24T21:45:06.061123+0000","last_fullsized":"2024-07-24T21:45:06.061123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:27:17.681374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'97","reported_seq":218,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:46:32.003849+0000","last_change":"2024-07-24T21:43:09.248961+0000","last_active":"2024-07-24T21:46:32.003849+0000","last_peered":"2024-07-24T21:46:32.003849+0000","last_clean":"2024-07-24T21:46:32.003849+0000","last_became_active":"2024-07-24T21:43:09.248640+0000","last_became_peered":"2024-07-24T21:43:09.248640+0000","last_unstale":"2024-07-24T21:46:32.003849+0000","last_undegraded":"2024-07-24T21:46:32.003849+0000","last_fullsized":"2024-07-24T21:46:32.003849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:40:17.293270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":178,"num_write_kb":1221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071163+0000","last_change":"2024-07-24T21:43:12.269853+0000","last_active":"2024-07-24T21:45:06.071163+0000","last_peered":"2024-07-24T21:45:06.071163+0000","last_clean":"2024-07-24T21:45:06.071163+0000","last_became_active":"2024-07-24T21:43:12.269530+0000","last_became_peered":"2024-07-24T21:43:12.269530+0000","last_unstale":"2024-07-24T21:45:06.071163+0000","last_undegraded":"2024-07-24T21:45:06.071163+0000","last_fullsized":"2024-07-24T21:45:06.071163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T23:47:37.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:38.081018+0000","last_change":"2024-07-24T21:43:09.666654+0000","last_active":"2024-07-24T21:44:38.081018+0000","last_peered":"2024-07-24T21:44:38.081018+0000","last_clean":"2024-07-24T21:44:38.081018+0000","last_became_active":"2024-07-24T21:43:09.666343+0000","last_became_peered":"2024-07-24T21:43:09.666343+0000","last_unstale":"2024-07-24T21:44:38.081018+0000","last_undegraded":"2024-07-24T21:44:38.081018+0000","last_fullsized":"2024-07-24T21:44:38.081018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T06:08:38.197656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.069854+0000","last_change":"2024-07-24T21:43:12.286649+0000","last_active":"2024-07-24T21:45:06.069854+0000","last_peered":"2024-07-24T21:45:06.069854+0000","last_clean":"2024-07-24T21:45:06.069854+0000","last_became_active":"2024-07-24T21:43:12.286306+0000","last_became_peered":"2024-07-24T21:43:12.286306+0000","last_unstale":"2024-07-24T21:45:06.069854+0000","last_undegraded":"2024-07-24T21:45:06.069854+0000","last_fullsized":"2024-07-24T21:45:06.069854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T09:40:40.301761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.891131+0000","last_change":"2024-07-24T21:43:09.375612+0000","last_active":"2024-07-24T21:44:37.891131+0000","last_peered":"2024-07-24T21:44:37.891131+0000","last_clean":"2024-07-24T21:44:37.891131+0000","last_became_active":"2024-07-24T21:43:09.375283+0000","last_became_peered":"2024-07-24T21:43:09.375283+0000","last_unstale":"2024-07-24T21:44:37.891131+0000","last_undegraded":"2024-07-24T21:44:37.891131+0000","last_fullsized":"2024-07-24T21:44:37.891131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-25T22:46:01.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.063506+0000","last_change":"2024-07-24T21:43:12.279968+0000","last_active":"2024-07-24T21:45:06.063506+0000","last_peered":"2024-07-24T21:45:06.063506+0000","last_clean":"2024-07-24T21:45:06.063506+0000","last_became_active":"2024-07-24T21:43:12.279614+0000","last_became_peered":"2024-07-24T21:43:12.279614+0000","last_unstale":"2024-07-24T21:45:06.063506+0000","last_undegraded":"2024-07-24T21:45:06.063506+0000","last_fullsized":"2024-07-24T21:45:06.063506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:40:17.920512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738063+0000","last_change":"2024-07-24T21:43:09.248804+0000","last_active":"2024-07-24T21:44:37.738063+0000","last_peered":"2024-07-24T21:44:37.738063+0000","last_clean":"2024-07-24T21:44:37.738063+0000","last_became_active":"2024-07-24T21:43:09.248497+0000","last_became_peered":"2024-07-24T21:43:09.248497+0000","last_unstale":"2024-07-24T21:44:37.738063+0000","last_undegraded":"2024-07-24T21:44:37.738063+0000","last_fullsized":"2024-07-24T21:44:37.738063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:31:28.769581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070964+0000","last_change":"2024-07-24T21:43:12.282842+0000","last_active":"2024-07-24T21:45:06.070964+0000","last_peered":"2024-07-24T21:45:06.070964+0000","last_clean":"2024-07-24T21:45:06.070964+0000","last_became_active":"2024-07-24T21:43:12.282525+0000","last_became_peered":"2024-07-24T21:43:12.282525+0000","last_unstale":"2024-07-24T21:45:06.070964+0000","last_undegraded":"2024-07-24T21:45:06.070964+0000","last_fullsized":"2024-07-24T21:45:06.070964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:57:21.927040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.742494+0000","last_change":"2024-07-24T21:43:09.258683+0000","last_active":"2024-07-24T21:44:37.742494+0000","last_peered":"2024-07-24T21:44:37.742494+0000","last_clean":"2024-07-24T21:44:37.742494+0000","last_became_active":"2024-07-24T21:43:09.258112+0000","last_became_peered":"2024-07-24T21:43:09.258112+0000","last_unstale":"2024-07-24T21:44:37.742494+0000","last_undegraded":"2024-07-24T21:44:37.742494+0000","last_fullsized":"2024-07-24T21:44:37.742494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:32:26.430260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.070667+0000","last_change":"2024-07-24T21:43:12.281155+0000","last_active":"2024-07-24T21:45:06.070667+0000","last_peered":"2024-07-24T21:45:06.070667+0000","last_clean":"2024-07-24T21:45:06.070667+0000","last_became_active":"2024-07-24T21:43:12.280746+0000","last_became_peered":"2024-07-24T21:43:12.280746+0000","last_unstale":"2024-07-24T21:45:06.070667+0000","last_undegraded":"2024-07-24T21:45:06.070667+0000","last_fullsized":"2024-07-24T21:45:06.070667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:06:21.305587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.749272+0000","last_change":"2024-07-24T21:44:37.748750+0000","last_active":"2024-07-24T21:44:37.749272+0000","last_peered":"2024-07-24T21:44:37.749272+0000","last_clean":"2024-07-24T21:44:37.749272+0000","last_became_active":"2024-07-24T21:44:37.744684+0000","last_became_peered":"2024-07-24T21:44:37.744684+0000","last_unstale":"2024-07-24T21:44:37.749272+0000","last_undegraded":"2024-07-24T21:44:37.749272+0000","last_fullsized":"2024-07-24T21:44:37.749272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T05:47:47.463381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073536+0000","last_change":"2024-07-24T21:43:12.274114+0000","last_active":"2024-07-24T21:45:06.073536+0000","last_peered":"2024-07-24T21:45:06.073536+0000","last_clean":"2024-07-24T21:45:06.073536+0000","last_became_active":"2024-07-24T21:43:12.273805+0000","last_became_peered":"2024-07-24T21:43:12.273805+0000","last_unstale":"2024-07-24T21:45:06.073536+0000","last_undegraded":"2024-07-24T21:45:06.073536+0000","last_fullsized":"2024-07-24T21:45:06.073536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:21:11.449291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.071915+0000","last_change":"2024-07-24T21:43:12.279773+0000","last_active":"2024-07-24T21:45:06.071915+0000","last_peered":"2024-07-24T21:45:06.071915+0000","last_clean":"2024-07-24T21:45:06.071915+0000","last_became_active":"2024-07-24T21:43:12.279099+0000","last_became_peered":"2024-07-24T21:43:12.279099+0000","last_unstale":"2024-07-24T21:45:06.071915+0000","last_undegraded":"2024-07-24T21:45:06.071915+0000","last_fullsized":"2024-07-24T21:45:06.071915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:06:50.496428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741164+0000","last_change":"2024-07-24T21:43:09.254078+0000","last_active":"2024-07-24T21:44:37.741164+0000","last_peered":"2024-07-24T21:44:37.741164+0000","last_clean":"2024-07-24T21:44:37.741164+0000","last_became_active":"2024-07-24T21:43:09.253718+0000","last_became_peered":"2024-07-24T21:43:09.253718+0000","last_unstale":"2024-07-24T21:44:37.741164+0000","last_undegraded":"2024-07-24T21:44:37.741164+0000","last_fullsized":"2024-07-24T21:44:37.741164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T08:10:56.151964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073850+0000","last_change":"2024-07-24T21:43:12.280979+0000","last_active":"2024-07-24T21:45:06.073850+0000","last_peered":"2024-07-24T21:45:06.073850+0000","last_clean":"2024-07-24T21:45:06.073850+0000","last_became_active":"2024-07-24T21:43:12.280040+0000","last_became_peered":"2024-07-24T21:43:12.280040+0000","last_unstale":"2024-07-24T21:45:06.073850+0000","last_undegraded":"2024-07-24T21:45:06.073850+0000","last_fullsized":"2024-07-24T21:45:06.073850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:08:16.995186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.741034+0000","last_change":"2024-07-24T21:43:37.879932+0000","last_active":"2024-07-24T21:44:37.741034+0000","last_peered":"2024-07-24T21:44:37.741034+0000","last_clean":"2024-07-24T21:44:37.741034+0000","last_became_active":"2024-07-24T21:43:37.875824+0000","last_became_peered":"2024-07-24T21:43:37.875824+0000","last_unstale":"2024-07-24T21:44:37.741034+0000","last_undegraded":"2024-07-24T21:44:37.741034+0000","last_fullsized":"2024-07-24T21:44:37.741034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T00:40:30.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.073156+0000","last_change":"2024-07-24T21:43:12.273974+0000","last_active":"2024-07-24T21:45:06.073156+0000","last_peered":"2024-07-24T21:45:06.073156+0000","last_clean":"2024-07-24T21:45:06.073156+0000","last_became_active":"2024-07-24T21:43:12.273610+0000","last_became_peered":"2024-07-24T21:43:12.273610+0000","last_unstale":"2024-07-24T21:45:06.073156+0000","last_undegraded":"2024-07-24T21:45:06.073156+0000","last_fullsized":"2024-07-24T21:45:06.073156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T01:11:17.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:44:37.738499+0000","last_change":"2024-07-24T21:43:09.375847+0000","last_active":"2024-07-24T21:44:37.738499+0000","last_peered":"2024-07-24T21:44:37.738499+0000","last_clean":"2024-07-24T21:44:37.738499+0000","last_became_active":"2024-07-24T21:43:09.375494+0000","last_became_peered":"2024-07-24T21:43:09.375494+0000","last_unstale":"2024-07-24T21:44:37.738499+0000","last_undegraded":"2024-07-24T21:44:37.738499+0000","last_fullsized":"2024-07-24T21:44:37.738499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T02:00:55.265794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:06.072178+0000","last_change":"2024-07-24T21:43:12.274816+0000","last_active":"2024-07-24T21:45:06.072178+0000","last_peered":"2024-07-24T21:45:06.072178+0000","last_clean":"2024-07-24T21:45:06.072178+0000","last_became_active":"2024-07-24T21:43:12.274251+0000","last_became_peered":"2024-07-24T21:43:12.274251+0000","last_unstale":"2024-07-24T21:45:06.072178+0000","last_undegraded":"2024-07-24T21:45:06.072178+0000","last_fullsized":"2024-07-24T21:45:06.072178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:11.253809+0000","last_clean_scrub_stamp":"2024-07-24T21:43:11.253809+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:04:56.300679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-07-24T21:45:17.695016+0000","last_change":"2024-07-24T21:43:09.250406+0000","last_active":"2024-07-24T21:45:17.695016+0000","last_peered":"2024-07-24T21:45:17.695016+0000","last_clean":"2024-07-24T21:45:17.695016+0000","last_became_active":"2024-07-24T21:43:09.249969+0000","last_became_peered":"2024-07-24T21:43:09.249969+0000","last_unstale":"2024-07-24T21:45:17.695016+0000","last_undegraded":"2024-07-24T21:45:17.695016+0000","last_fullsized":"2024-07-24T21:45:17.695016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-24T21:43:08.231576+0000","last_clean_scrub_stamp":"2024-07-24T21:43:08.231576+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-26T07:01:22.462651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":856826,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1511,"num_read_kb":960,"num_write":399,"num_write_kb":2211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2801664,"data_stored":2570478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":481044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29092,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716060,"statfs":{"total":95995035648,"available":95965245440,"internally_reserved":0,"allocated":2064384,"data_stored":1532503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1376256,"data_stored":874474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103239,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36676,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708476,"statfs":{"total":95995035648,"available":95957479424,"internally_reserved":0,"allocated":1372160,"data_stored":874372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1576960,"data_stored":1073249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528399,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":480647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724629,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1986560,"data_stored":1466177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":481616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117086,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703860,"statfs":{"total":95995035648,"available":95952752640,"internally_reserved":0,"allocated":1777664,"data_stored":1267950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":546132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705772,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704688,"statfs":{"total":95995035648,"available":95953600512,"internally_reserved":0,"allocated":991232,"data_stored":481918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":787276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":395252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":461578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-24T21:46:42.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:46:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:43 smithi060 ceph-mon[33000]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:43 smithi060 ceph-mon[33000]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:43 smithi123 ceph-mon[37143]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:43.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:43 smithi123 ceph-mon[37143]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:43.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:43 smithi086 ceph-mon[37128]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-24T21:46:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:43 smithi086 ceph-mon[37128]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 -- 172.21.15.60:0/2802655188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b4108b20 msgr2=0x7fb3b410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 --2- 172.21.15.60:0/2802655188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b4108b20 0x7fb3b410f320 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ca0 tx=0x7fb3ac02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 -- 172.21.15.60:0/2802655188 shutdown_connections 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 --2- 172.21.15.60:0/2802655188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b4108b20 0x7fb3b410f320 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 --2- 172.21.15.60:0/2802655188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b41081a0 0x7fb3b41085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 --2- 172.21.15.60:0/2802655188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4102790 0x7fb3b4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 -- 172.21.15.60:0/2802655188 >> 172.21.15.60:0/2802655188 conn(0x7fb3b40fe200 msgr2=0x7fb3b4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 -- 172.21.15.60:0/2802655188 shutdown_connections 2024-07-24T21:46:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.775+0000 7fb3ba809640 1 -- 172.21.15.60:0/2802655188 wait complete. 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 Processor -- start 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 -- start start 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 0x7fb3b4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 0x7fb3b4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112790 con 0x7fb3b41081a0 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112610 con 0x7fb3b4102790 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3ba809640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112910 con 0x7fb3b4108b20 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50004/0 (socket says 172.21.15.60:50004) 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3b37fe640 1 -- 172.21.15.60:0/1574192148 learned_addr learned my addr 172.21.15.60:0/1574192148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3b8da6640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 0x7fb3b4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.776+0000 7fb3b3fff640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 0x7fb3b4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 -- 172.21.15.60:0/1574192148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 msgr2=0x7fb3b4075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 0x7fb3b4075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 -- 172.21.15.60:0/1574192148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 msgr2=0x7fb3b4074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 0x7fb3b4074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 -- 172.21.15.60:0/1574192148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b41a6fa0 con 0x7fb3b41081a0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b8da6640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 0x7fb3b4075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b3fff640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 0x7fb3b4074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b37fe640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c920 tx=0x7fb3a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8007c70 con 0x7fb3b41081a0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a8007e10 con 0x7fb3b41081a0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b41a7290 con 0x7fb3b41081a0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8018820 con 0x7fb3b41081a0 2024-07-24T21:46:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.777+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41a77d0 con 0x7fb3b41081a0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.778+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b4102b70 con 0x7fb3b41081a0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.779+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7fb3a800f450 con 0x7fb3b41081a0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.780+0000 7fb3b17fa640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 0x7fb39c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.780+0000 7fb3b3fff640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 0x7fb39c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.780+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb3a80e43a0 con 0x7fb3b41081a0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.781+0000 7fb3b3fff640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 0x7fb39c0b6170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40058c0 tx=0x7fb3a4005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:44.783+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fb3a8014030 con 0x7fb3b41081a0 2024-07-24T21:46:45.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.015+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b4076190 con 0x7fb3b41081a0 2024-07-24T21:46:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.016+0000 7fb3b17fa640 1 -- 172.21.15.60:0/1574192148 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fb3a80ab800 con 0x7fb3b41081a0 2024-07-24T21:46:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:45.017 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:46:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:46:45.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 msgr2=0x7fb39c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 0x7fb39c0b6170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40058c0 tx=0x7fb3a4005850 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 msgr2=0x7fb3b40753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c920 tx=0x7fb3a800cde0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 shutdown_connections 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb39c0b3cb0 0x7fb39c0b6170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3b4108b20 0x7fb3b4075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b41081a0 0x7fb3b40753d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 --2- 172.21.15.60:0/1574192148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b4102790 0x7fb3b4074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.019+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 >> 172.21.15.60:0/1574192148 conn(0x7fb3b40fe200 msgr2=0x7fb3b41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.020+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 shutdown_connections 2024-07-24T21:46:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:45.020+0000 7fb3ba809640 1 -- 172.21.15.60:0/1574192148 wait complete. 2024-07-24T21:46:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.240+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/61303567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf41035d0 msgr2=0x7ffbf41039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.240+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/61303567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf41035d0 0x7ffbf41039b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009900 tx=0x7ffbe402a410 comp rx=0 tx=0).stop 2024-07-24T21:46:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/61303567 shutdown_connections 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/61303567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf410c9f0 0x7ffbf410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/61303567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 0x7ffbf410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/61303567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf41035d0 0x7ffbf41039b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/61303567 >> 172.21.15.60:0/61303567 conn(0x7ffbf40782e0 msgr2=0x7ffbf40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/61303567 shutdown_connections 2024-07-24T21:46:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/61303567 wait complete. 2024-07-24T21:46:45.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:45 smithi123 ceph-mon[37143]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:45.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:45 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:46:45.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:45 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1574192148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 Processor -- start 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.241+0000 7ffbf8fc0640 1 -- start start 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf41035d0 0x7ffbf4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 0x7ffbf4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4112660 con 0x7ffbf410c9f0 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41124e0 con 0x7ffbf4103ef0 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf8fc0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41127e0 con 0x7ffbf41035d0 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf41035d0 0x7ffbf4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50014/0 (socket says 172.21.15.60:50014) 2024-07-24T21:46:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/513670982 learned_addr learned my addr 172.21.15.60:0/513670982 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 0x7ffbf4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/513670982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf41035d0 msgr2=0x7ffbf4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf41035d0 0x7ffbf4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/513670982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 msgr2=0x7ffbf4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 0x7ffbf4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/513670982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf419eca0 con 0x7ffbf410c9f0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.242+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00eaa0 tx=0x7ffbec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.243+0000 7ffbe37fe640 1 -- 172.21.15.60:0/513670982 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec00cdb0 con 0x7ffbf410c9f0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.243+0000 7ffbe37fe640 1 -- 172.21.15.60:0/513670982 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbec005020 con 0x7ffbf410c9f0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.243+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf419ef90 con 0x7ffbf410c9f0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.243+0000 7ffbe37fe640 1 -- 172.21.15.60:0/513670982 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0116d0 con 0x7ffbf410c9f0 2024-07-24T21:46:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.243+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf41a5650 con 0x7ffbf410c9f0 2024-07-24T21:46:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.245+0000 7ffbfbdf9740 1 Processor -- start 2024-07-24T21:46:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbe37fe640 1 -- 172.21.15.60:0/513670982 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7ffbec0051d0 con 0x7ffbf410c9f0 2024-07-24T21:46:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 -- start start 2024-07-24T21:46:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fd8c0 0x55b2df7fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 0x55b2df80e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b2df773280 con 0x55b2df80e580 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2df762780 con 0x55b2df7fd8c0 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.246+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2df7667c0 con 0x55b2df7fe1e0 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 0x55b2df80e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50024/0 (socket says 172.21.15.60:50024) 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf1d9b640 1 -- 172.21.15.60:0/1513178247 learned_addr learned my addr 172.21.15.60:0/1513178247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fd8c0 0x55b2df7fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 -- 172.21.15.60:0/1513178247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fd8c0 msgr2=0x55b2df7fdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fd8c0 0x55b2df7fdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 -- 172.21.15.60:0/1513178247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 msgr2=0x55b2df810970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 -- 172.21.15.60:0/1513178247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df4c6790 con 0x55b2df7fe1e0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbe37fe640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 0x7ffbdc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf259c640 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 0x55b2df80e040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009230 tx=0x7ffbe4032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599197bfe4be7ff9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 0x7ffbdc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbe37fe640 1 -- 172.21.15.60:0/513670982 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbec0e40c0 con 0x7ffbf410c9f0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.247+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/1513178247 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4040070 con 0x55b2df7fe1e0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/1513178247 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe4004030 con 0x55b2df7fe1e0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/1513178247 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4035040 con 0x55b2df7fe1e0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1513178247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 msgr2=0x55b2df80e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 0x7ffbdc0b6170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009580 tx=0x7ffbe80096c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 0x55b2df80e040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009230 tx=0x7ffbe4032b90 comp rx=0 tx=0).stop 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1513178247 shutdown_connections 2024-07-24T21:46:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fe1e0 0x55b2df80e040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1513178247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fd8c0 0x55b2df7fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1513178247 >> 172.21.15.60:0/1513178247 conn(0x55b2df7fb6f0 msgr2=0x55b2df7fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1513178247 shutdown_connections 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.248+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1513178247 wait complete. 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 -- start start 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 Processor -- start 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df80e580 0x55b2df80b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b2df69cab0 con 0x55b2df7fd8c0 2024-07-24T21:46:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2df6ea9e0 con 0x55b2df7fe1e0 2024-07-24T21:46:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2df811200 con 0x55b2df80e580 2024-07-24T21:46:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbfbdf9740 10 client.0 _ll_register_callbacks cb 0x55b2df472a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbf259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbf259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50040/0 (socket says 172.21.15.60:50040) 2024-07-24T21:46:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbf259c640 1 -- 172.21.15.60:0/610907688 learned_addr learned my addr 172.21.15.60:0/610907688 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df80e580 0x55b2df80b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.249+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 -- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df80e580 msgr2=0x55b2df80b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df80e580 0x55b2df80b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 -- 172.21.15.60:0/610907688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 msgr2=0x55b2df80b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 -- 172.21.15.60:0/610907688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df53f620 con 0x55b2df7fd8c0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbf259c640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009650 tx=0x7ffbe4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2268aafb831a33 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4035070 con 0x55b2df7fd8c0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbe40316a0 con 0x55b2df7fd8c0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2df711ce0 con 0x55b2df7fd8c0 2024-07-24T21:46:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffbe4055050 con 0x55b2df7fd8c0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbfbdf9740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2df81c950 con 0x55b2df7fd8c0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b2df711ce0 con 0x55b2df7fd8c0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.250+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ffbe4004c00 con 0x55b2df7fd8c0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 20 client.15468 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df81dca0 0x55b2df820090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 4 client.15468 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2df7751d0 con 0x55b2df81dca0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df81dca0 0x55b2df820090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df81dca0 0x55b2df820090 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb2b293f30ccca0 server_cookie=586df02551abfca5 in_seq=0 out_seq=0 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.251+0000 7ffbd9ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/610907688 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2df7751d0 con 0x55b2df81dca0 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbd9ffb640 10 client.15468 handle_command_reply: tid=0 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 client.15468 shutdown 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 msgr2=0x55b2df80dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009650 tx=0x7ffbe4009260 comp rx=0 tx=0).stop 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 shutdown_connections 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df81dca0 0x55b2df820090 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df80e580 0x55b2df80b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbd9ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/610907688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df7fd8c0 0x55b2df80dc90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 >> 172.21.15.60:0/610907688 conn(0x55b2df7fb6f0 msgr2=0x55b2df801290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 shutdown_connections 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/610907688 wait complete. 2024-07-24T21:46:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.252+0000 7ffbfbdf9740 20 client.15468 trim_cache size 0 max 16384 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.268+0000 7ffbfbdf9740 1 Processor -- start 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 -- start start 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df80b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41ae720 con 0x55b2df80e580 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41abff0 con 0x55b2df7fe1e0 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41a9e40 con 0x55b2df7fd8c0 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50056/0 (socket says 172.21.15.60:50056) 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/1432495112 learned_addr learned my addr 172.21.15.60:0/1432495112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf259c640 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df80b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/1432495112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 msgr2=0x55b2df80b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df80b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/1432495112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 msgr2=0x55b2df80b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/1432495112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df80bbc0 con 0x55b2df80e580 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.269+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00efd0 tx=0x7ffbec0ab550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a36151de0af33 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbdbfff640 1 -- 172.21.15.60:0/1432495112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec01f040 con 0x55b2df80e580 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbdbfff640 1 -- 172.21.15.60:0/1432495112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbec0b3e40 con 0x55b2df80e580 2024-07-24T21:46:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbdbfff640 1 -- 172.21.15.60:0/1432495112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0a6960 con 0x55b2df80e580 2024-07-24T21:46:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1432495112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 msgr2=0x55b2df810230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00efd0 tx=0x7ffbec0ab550 comp rx=0 tx=0).stop 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1432495112 shutdown_connections 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df810230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df80b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/1432495112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df80b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.270+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1432495112 >> 172.21.15.60:0/1432495112 conn(0x55b2df7fb6f0 msgr2=0x55b2df80a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1432495112 shutdown_connections 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/1432495112 wait complete. 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- start start 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 Processor -- start 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df8256f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df825c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aa4b0 con 0x55b2df80e580 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41ab920 con 0x55b2df7fe1e0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aab80 con 0x55b2df7fd8c0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbfbdf9740 10 client.0 _ll_register_callbacks cb 0x7ffbe8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbf2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50072/0 (socket says 172.21.15.60:50072) 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/144086919 learned_addr learned my addr 172.21.15.60:0/144086919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbf1d9b640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df825c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.271+0000 7ffbf259c640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df8256f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 msgr2=0x55b2df8256f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df8256f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/144086919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 msgr2=0x55b2df825c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df825c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 -- 172.21.15.60:0/144086919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df8169d0 con 0x55b2df80e580 2024-07-24T21:46:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf259c640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df8256f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbf2d9d640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0aaa90 tx=0x7ffbec0aac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d433725faa695842 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec01f070 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ffbec0c5040 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbec0b3e40 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffbec0a7380 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbfbdf9740 10 client.15474 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2df7ff470 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.272+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbec0b3e40 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ffbec08c080 con 0x55b2df80e580 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 10 client.15474 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 20 client.15474 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df888760 0x55b2df88ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 4 client.15474 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2df770c70 con 0x55b2df888760 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbf259c640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df888760 0x55b2df88ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbd9ffb640 10 client.15474 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.273+0000 7ffbf259c640 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df888760 0x55b2df88ab80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d166dce6aac93569 server_cookie=d198f429db2b79a0 in_seq=0 out_seq=0 2024-07-24T21:46:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbd9ffb640 1 -- 172.21.15.60:0/144086919 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2df770c70 con 0x55b2df888760 2024-07-24T21:46:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbd9ffb640 10 client.15474 handle_command_reply: tid=0 2024-07-24T21:46:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 client.15474 shutdown 2024-07-24T21:46:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 msgr2=0x55b2df816490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0aaa90 tx=0x7ffbec0aac10 comp rx=0 tx=0).stop 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 shutdown_connections 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2df888760 0x55b2df88ab80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2df80e580 0x55b2df816490 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2df7fe1e0 0x55b2df825c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 --2- 172.21.15.60:0/144086919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df7fd8c0 0x55b2df8256f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 >> 172.21.15.60:0/144086919 conn(0x55b2df7fb6f0 msgr2=0x55b2df7f9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbd9ffb640 0 client.15474 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 shutdown_connections 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 1 -- 172.21.15.60:0/144086919 wait complete. 2024-07-24T21:46:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.285+0000 7ffbfbdf9740 20 client.15474 trim_cache size 0 max 16384 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 msgr2=0x7ffbdc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 0x7ffbdc0b6170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009580 tx=0x7ffbe80096c0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 msgr2=0x7ffbf419e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00eaa0 tx=0x7ffbec00ef60 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 shutdown_connections 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffbdc0b3cb0 0x7ffbdc0b6170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbf410c9f0 0x7ffbf419e4d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbf4103ef0 0x7ffbf4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 --2- 172.21.15.60:0/513670982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbf41035d0 0x7ffbf4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 >> 172.21.15.60:0/513670982 conn(0x7ffbf40782e0 msgr2=0x7ffbf40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 shutdown_connections 2024-07-24T21:46:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:45.286+0000 7ffbf8fc0640 1 -- 172.21.15.60:0/513670982 wait complete. 2024-07-24T21:46:45.307 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:45 smithi060 ceph-mon[33000]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:45.307 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:45 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:46:45.307 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1574192148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:45.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:45 smithi086 ceph-mon[37128]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-24T21:46:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:45 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:46:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:45 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1574192148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:45.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:46:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:47.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:47 smithi060 ceph-mon[33000]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-24T21:46:47.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:47 smithi123 ceph-mon[37143]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-24T21:46:47.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:47 smithi086 ceph-mon[37128]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.644+0000 7f6200aec640 1 -- 172.21.15.60:0/3971286237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc102790 msgr2=0x7f61fc102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.644+0000 7f6200aec640 1 --2- 172.21.15.60:0/3971286237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc102790 0x7f61fc102b70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009900 tx=0x7f61ec02a410 comp rx=0 tx=0).stop 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 -- 172.21.15.60:0/3971286237 shutdown_connections 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 --2- 172.21.15.60:0/3971286237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc108b20 0x7f61fc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 --2- 172.21.15.60:0/3971286237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc1081a0 0x7f61fc1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 --2- 172.21.15.60:0/3971286237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc102790 0x7f61fc102b70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 -- 172.21.15.60:0/3971286237 >> 172.21.15.60:0/3971286237 conn(0x7f61fc0fe200 msgr2=0x7f61fc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 -- 172.21.15.60:0/3971286237 shutdown_connections 2024-07-24T21:46:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 -- 172.21.15.60:0/3971286237 wait complete. 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 Processor -- start 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.645+0000 7f6200aec640 1 -- start start 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 0x7f61fc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc1081a0 0x7f61fc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc112720 con 0x7f61fc108b20 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc1125a0 con 0x7f61fc102790 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f6200aec640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc1128a0 con 0x7f61fc1081a0 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43318/0 (socket says 172.21.15.60:43318) 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 -- 172.21.15.60:0/2273351058 learned_addr learned my addr 172.21.15.60:0/2273351058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61f9d9b640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc1081a0 0x7f61fc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fa59c640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 0x7f61fc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 -- 172.21.15.60:0/2273351058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc1081a0 msgr2=0x7f61fc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc1081a0 0x7f61fc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 -- 172.21.15.60:0/2273351058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 msgr2=0x7f61fc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 0x7f61fc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 -- 172.21.15.60:0/2273351058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc19ee60 con 0x7f61fc108b20 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fa59c640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 0x7f61fc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.646+0000 7f61fad9d640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fd0 tx=0x7f61f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.647+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4017070 con 0x7f61fc108b20 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.647+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61f4003e20 con 0x7f61fc108b20 2024-07-24T21:46:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.647+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4010460 con 0x7f61fc108b20 2024-07-24T21:46:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.647+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc19f150 con 0x7f61fc108b20 2024-07-24T21:46:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.647+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc1a5810 con 0x7f61fc108b20 2024-07-24T21:46:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.648+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61fc103a40 con 0x7f61fc108b20 2024-07-24T21:46:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.649+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f61f4003970 con 0x7f61fc108b20 2024-07-24T21:46:47.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.650+0000 7f61eb7fe640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 0x7f61e40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:47.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.650+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f61f40e3d30 con 0x7f61fc108b20 2024-07-24T21:46:47.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.650+0000 7f61fa59c640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 0x7f61e40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:47.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.651+0000 7f61fa59c640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 0x7f61e40b6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f61ec031440 tx=0x7f61ec005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:47.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.654+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f61f40ab0a0 con 0x7f61fc108b20 2024-07-24T21:46:47.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.887+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61fc10a900 con 0x7f61fc108b20 2024-07-24T21:46:47.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.888+0000 7f61eb7fe640 1 -- 172.21.15.60:0/2273351058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f61f40aa9a0 con 0x7f61fc108b20 2024-07-24T21:46:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:47.889 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:46:47.889 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 msgr2=0x7f61e40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 0x7f61e40b6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f61ec031440 tx=0x7f61ec005a20 comp rx=0 tx=0).stop 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 msgr2=0x7f61fc19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fd0 tx=0x7f61f400b260 comp rx=0 tx=0).stop 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 shutdown_connections 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61e40b3cb0 0x7f61e40b6170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61fc108b20 0x7f61fc19e690 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61fc1081a0 0x7f61fc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 --2- 172.21.15.60:0/2273351058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61fc102790 0x7f61fc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 >> 172.21.15.60:0/2273351058 conn(0x7f61fc0fe200 msgr2=0x7f61fc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 shutdown_connections 2024-07-24T21:46:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:47.890+0000 7f6200aec640 1 -- 172.21.15.60:0/2273351058 wait complete. 2024-07-24T21:46:48.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:48 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2273351058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:48.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:48 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2273351058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:48.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T21:46:48.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:48 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2273351058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:49.177 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:49 smithi060 ceph-mon[33000]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-24T21:46:49.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:49 smithi123 ceph-mon[37143]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-24T21:46:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:49 smithi086 ceph-mon[37128]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-24T21:46:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 -- 172.21.15.60:0/2426234397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 msgr2=0x7fed8c1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 --2- 172.21.15.60:0/2426234397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c1085e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed80009900 tx=0x7fed8002a410 comp rx=0 tx=0).stop 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 -- 172.21.15.60:0/2426234397 shutdown_connections 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 --2- 172.21.15.60:0/2426234397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 0x7fed8c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 --2- 172.21.15.60:0/2426234397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c1085e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 --2- 172.21.15.60:0/2426234397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 0x7fed8c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 -- 172.21.15.60:0/2426234397 >> 172.21.15.60:0/2426234397 conn(0x7fed8c0fe200 msgr2=0x7fed8c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 -- 172.21.15.60:0/2426234397 shutdown_connections 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.423+0000 7fed9135e640 1 -- 172.21.15.60:0/2426234397 wait complete. 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 Processor -- start 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 -- start start 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 0x7fed8c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 0x7fed8c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c112730 con 0x7fed8c1081a0 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1125b0 con 0x7fed8c108b20 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed9135e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1128b0 con 0x7fed8c102790 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.424+0000 7fed8a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43340/0 (socket says 172.21.15.60:43340) 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 -- 172.21.15.60:0/2015490188 learned_addr learned my addr 172.21.15.60:0/2015490188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8b7fe640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 0x7fed8c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8affd640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 0x7fed8c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 -- 172.21.15.60:0/2015490188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 msgr2=0x7fed8c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 0x7fed8c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 -- 172.21.15.60:0/2015490188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 msgr2=0x7fed8c19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 0x7fed8c19e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 -- 172.21.15.60:0/2015490188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c19ec90 con 0x7fed8c1081a0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed8a7fc640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fed80005940 tx=0x7fed800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8003f030 con 0x7fed8c1081a0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed80003cb0 con 0x7fed8c1081a0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c19ef20 con 0x7fed8c1081a0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80035070 con 0x7fed8c1081a0 2024-07-24T21:46:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.425+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1a5640 con 0x7fed8c1081a0 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.426+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c102b70 con 0x7fed8c1081a0 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.428+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7fed80042030 con 0x7fed8c1081a0 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.428+0000 7fed73fff640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 0x7fed740b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.429+0000 7fed8affd640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 0x7fed740b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.429+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fed80104990 con 0x7fed8c1081a0 2024-07-24T21:46:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.429+0000 7fed8affd640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 0x7fed740b6170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbe0 tx=0x7fed7c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.431+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fed800cb710 con 0x7fed8c1081a0 2024-07-24T21:46:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.627+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed8c10a900 con 0x7fed8c1081a0 2024-07-24T21:46:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.628+0000 7fed73fff640 1 -- 172.21.15.60:0/2015490188 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23071 (secure 0 0 0) 0x7fed800cfc60 con 0x7fed8c1081a0 2024-07-24T21:46:50.630 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:50.630 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T21:44:37.734936+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 msgr2=0x7fed740b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 0x7fed740b6170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbe0 tx=0x7fed7c00b390 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 msgr2=0x7fed8c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fed80005940 tx=0x7fed800042f0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 shutdown_connections 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fed740b3cb0 0x7fed740b6170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed8c108b20 0x7fed8c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed8c1081a0 0x7fed8c199120 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 --2- 172.21.15.60:0/2015490188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed8c102790 0x7fed8c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.630+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 >> 172.21.15.60:0/2015490188 conn(0x7fed8c0fe200 msgr2=0x7fed8c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.631+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 shutdown_connections 2024-07-24T21:46:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:50.631+0000 7fed9135e640 1 -- 172.21.15.60:0/2015490188 wait complete. 2024-07-24T21:46:51.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:51 smithi060 ceph-mon[33000]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:46:51.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:51 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2015490188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:51.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-07-24T21:46:51.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:51 smithi123 ceph-mon[37143]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:46:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:51 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2015490188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:51 smithi086 ceph-mon[37128]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:46:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:51 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2015490188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T21:46:51.596 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:53.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:53 smithi060 ceph-mon[33000]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:53.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:53 smithi123 ceph-mon[37143]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:53 smithi086 ceph-mon[37128]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:53.975 INFO:teuthology.orchestra.run.smithi060.stdout:{"run_dir":"/var/run/ceph"} 2024-07-24T21:46:53.976 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-07-24T21:46:53.976 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1368 2024-07-24T21:46:53.976 DEBUG:tasks.quiescer:Yielding 2024-07-24T21:46:53.977 INFO:teuthology.run_tasks:Running task workunit... 2024-07-24T21:46:53.991 INFO:tasks.workunit:Pulling workunits from ref 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-24T21:46:53.991 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-24T21:46:53.991 INFO:tasks.workunit:timeout=3h 2024-07-24T21:46:53.991 INFO:tasks.workunit:cleanup=True 2024-07-24T21:46:53.992 DEBUG:teuthology.orchestra.run.smithi060:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:53.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:46:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-24T21:46:54.010 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-07-24 21:43:43.815617692 +0000 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-07-24 21:43:43.815617692 +0000 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-07-24 21:46:17.856406074 +0000 2024-07-24T21:46:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-07-24 21:43:43.815617692 +0000 2024-07-24T21:46:54.011 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-24T21:46:54.011 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-24T21:46:54.086 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-24T21:46:54.143 INFO:tasks.workunit.client.0.smithi060.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-24T21:46:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:55.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:55 smithi060 ceph-mon[33000]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:55 smithi123 ceph-mon[37143]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:55 smithi086 ceph-mon[37128]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-24T21:46:55.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 -- 172.21.15.60:0/342478138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df8103670 msgr2=0x7f3df8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 --2- 172.21.15.60:0/342478138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df8103670 0x7f3df8103a50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009ca0 tx=0x7f3de802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 -- 172.21.15.60:0/342478138 shutdown_connections 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 --2- 172.21.15.60:0/342478138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df810cad0 0x7f3df810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 --2- 172.21.15.60:0/342478138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 0x7f3df810c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 --2- 172.21.15.60:0/342478138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df8103670 0x7f3df8103a50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 -- 172.21.15.60:0/342478138 >> 172.21.15.60:0/342478138 conn(0x7f3df80fd0f0 msgr2=0x7f3df80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 -- 172.21.15.60:0/342478138 shutdown_connections 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.886+0000 7f3dfff32640 1 -- 172.21.15.60:0/342478138 wait complete. 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 Processor -- start 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 -- start start 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df8103670 0x7f3df8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 0x7f3df81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8112730 con 0x7f3df810cad0 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3df81125b0 con 0x7f3df8103f90 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfff32640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3df81128b0 con 0x7f3df8103670 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51396/0 (socket says 172.21.15.60:51396) 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 -- 172.21.15.60:0/589727193 learned_addr learned my addr 172.21.15.60:0/589727193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfdcce640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df8103670 0x7f3df8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfd4cd640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 0x7f3df81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 -- 172.21.15.60:0/589727193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df8103670 msgr2=0x7f3df8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df8103670 0x7f3df8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 -- 172.21.15.60:0/589727193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 msgr2=0x7f3df81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 0x7f3df81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 -- 172.21.15.60:0/589727193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df819ed50 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.887+0000 7f3dfe4cf640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3df000c970 tx=0x7f3df000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.888+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0007bb0 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.888+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df0007d50 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.888+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df819f040 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.888+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0004e60 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.888+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df81a5700 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.889+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df8103a50 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.891+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f3df0005000 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.892+0000 7f3df6ffd640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 0x7f3de00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.892+0000 7f3dfdcce640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 0x7f3de00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.892+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3df00e3f30 con 0x7f3df810cad0 2024-07-24T21:46:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.892+0000 7f3dfdcce640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 0x7f3de00b6170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009430 tx=0x7f3de8009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:55.895+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3df0014030 con 0x7f3df810cad0 2024-07-24T21:46:56.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.131+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3df8199dd0 con 0x7f3df810cad0 2024-07-24T21:46:56.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.132+0000 7f3df6ffd640 1 -- 172.21.15.60:0/589727193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f3df00ab2a0 con 0x7f3df810cad0 2024-07-24T21:46:56.134 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:46:56.134 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:46:56.134 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 msgr2=0x7f3de00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 0x7f3de00b6170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009430 tx=0x7f3de8009360 comp rx=0 tx=0).stop 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 msgr2=0x7f3df819e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3df000c970 tx=0x7f3df000ce30 comp rx=0 tx=0).stop 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 shutdown_connections 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3de00b3cb0 0x7f3de00b6170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df810cad0 0x7f3df819e580 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:56.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3df8103f90 0x7f3df81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:56.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 --2- 172.21.15.60:0/589727193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df8103670 0x7f3df8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:56.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 >> 172.21.15.60:0/589727193 conn(0x7f3df80fd0f0 msgr2=0x7f3df80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:56.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 shutdown_connections 2024-07-24T21:46:56.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:56.135+0000 7f3dfff32640 1 -- 172.21.15.60:0/589727193 wait complete. 2024-07-24T21:46:56.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/589727193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:56.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:56 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/589727193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:56 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/589727193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:46:56.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-24T21:43:14.079489+0000', 'modified': '2024-07-24T21:46:18.571687+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24610, 'mds_1': 24605, 'mds_2': 24598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24560': {'gid': 24560, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/1999114136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1999114136}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1999114136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/549557535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 549557535}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 549557535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/3784109187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3784109187}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3784109187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/2950411419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2950411419}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2950411419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.86:6839/1688764220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 1688764220}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 1688764220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.123:6839/3718566677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 3718566677}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 3718566677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24610, 'qdb_cluster': [24598, 24605, 24610]} 2024-07-24T21:46:56.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-24T21:46:56.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.c status 2024-07-24T21:46:57.150 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:46:57.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:57 smithi060 ceph-mon[33000]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:46:57.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:57 smithi123 ceph-mon[37143]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:46:57.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:57 smithi086 ceph-mon[37128]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- 172.21.15.60:0/2456296230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5564106fd0 msgr2=0x7f5564107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 --2- 172.21.15.60:0/2456296230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5564106fd0 0x7f5564107410 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f555402a410 comp rx=0 tx=0).stop 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- 172.21.15.60:0/2456296230 shutdown_connections 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 --2- 172.21.15.60:0/2456296230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 0x7f556410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 --2- 172.21.15.60:0/2456296230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5564106fd0 0x7f5564107410 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 --2- 172.21.15.60:0/2456296230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55641015d0 0x7f55641019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- 172.21.15.60:0/2456296230 >> 172.21.15.60:0/2456296230 conn(0x7f55640fd030 msgr2=0x7f55640ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- 172.21.15.60:0/2456296230 shutdown_connections 2024-07-24T21:46:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- 172.21.15.60:0/2456296230 wait complete. 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 Processor -- start 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.380+0000 7f556c5ec640 1 -- start start 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5564106fd0 0x7f55640fda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 0x7f55640fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5564111570 con 0x7f55641015d0 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55641113f0 con 0x7f5564107950 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55641116f0 con 0x7f5564106fd0 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51418/0 (socket says 172.21.15.60:51418) 2024-07-24T21:46:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 -- 172.21.15.60:0/1641503861 learned_addr learned my addr 172.21.15.60:0/1641503861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f5569b87640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5564106fd0 0x7f55640fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556ab89640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 0x7f55640fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 -- 172.21.15.60:0/1641503861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5564106fd0 msgr2=0x7f55640fda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5564106fd0 0x7f55640fda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 -- 172.21.15.60:0/1641503861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 msgr2=0x7f55640fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 0x7f55640fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 -- 172.21.15.60:0/1641503861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641a7050 con 0x7f55641015d0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556a388640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f556000d950 tx=0x7f556000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f555b7fe640 1 -- 172.21.15.60:0/1641503861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560017030 con 0x7f55641015d0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f555b7fe640 1 -- 172.21.15.60:0/1641503861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560004020 con 0x7f55641015d0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55641a7340 con 0x7f55641015d0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.381+0000 7f555b7fe640 1 -- 172.21.15.60:0/1641503861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600047a0 con 0x7f55641015d0 2024-07-24T21:46:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.382+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a78b0 con 0x7f55641015d0 2024-07-24T21:46:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.384+0000 7f555b7fe640 1 -- 172.21.15.60:0/1641503861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f5560009a30 con 0x7f55641015d0 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.384+0000 7f556f417740 1 Processor -- start 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.384+0000 7f556f417740 1 -- start start 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55951bc12280 con 0x55951bca9170 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55951bc01780 con 0x55951bc9d1e0 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556f417740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55951bc057c0 con 0x55951bc9c8c0 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51420/0 (socket says 172.21.15.60:51420) 2024-07-24T21:46:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 -- 172.21.15.60:0/766243978 learned_addr learned my addr 172.21.15.60:0/766243978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556ab89640 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556a388640 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f555b7fe640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 0x7f55500b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 -- 172.21.15.60:0/766243978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 msgr2=0x55951bc9cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 -- 172.21.15.60:0/766243978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 msgr2=0x55951bca8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 -- 172.21.15.60:0/766243978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951b965790 con 0x55951bca9170 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f556ab89640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 0x7f55500b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f555b7fe640 1 -- 172.21.15.60:0/1641503861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f55600e3a10 con 0x7f55641015d0 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.385+0000 7f5569b87640 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f5554032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ccaf678aef0f9e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f5558bfe640 1 -- 172.21.15.60:0/766243978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554040070 con 0x55951bca9170 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556ab89640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 0x7f55500b61f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f555c005fd0 tx=0x7f555c004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f5558bfe640 1 -- 172.21.15.60:0/766243978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5554004030 con 0x55951bca9170 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f5558bfe640 1 -- 172.21.15.60:0/766243978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554035040 con 0x55951bca9170 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 -- 172.21.15.60:0/766243978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 msgr2=0x55951bcab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f5554032bd0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 -- 172.21.15.60:0/766243978 shutdown_connections 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcab560 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 --2- 172.21.15.60:0/766243978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.386+0000 7f556f417740 1 -- 172.21.15.60:0/766243978 >> 172.21.15.60:0/766243978 conn(0x55951bc9a6f0 msgr2=0x55951bc9b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- 172.21.15.60:0/766243978 shutdown_connections 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- 172.21.15.60:0/766243978 wait complete. 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- start start 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 Processor -- start 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bca8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bca9170 0x55951bca62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55951bb3bab0 con 0x55951bc9d1e0 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55951bb899e0 con 0x55951bca9170 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55951bcabdf0 con 0x55951bc9c8c0 2024-07-24T21:46:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556f417740 10 client.0 _ll_register_callbacks cb 0x55951b911a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f5569b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f5569b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51424/0 (socket says 172.21.15.60:51424) 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f5569b87640 1 -- 172.21.15.60:0/688570726 learned_addr learned my addr 172.21.15.60:0/688570726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.387+0000 7f556a388640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bca8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f556ab89640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bca9170 0x55951bca62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 -- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 msgr2=0x55951bca8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bca8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 -- 172.21.15.60:0/688570726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bca9170 msgr2=0x55951bca62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bca9170 0x55951bca62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 -- 172.21.15.60:0/688570726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951b9de620 con 0x55951bc9d1e0 2024-07-24T21:46:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f5569b87640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5554005940 tx=0x7f555402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c698f5942de514fb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f554dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554035070 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f55540316a0 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55951bbb0ce0 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5554055050 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f556f417740 10 client.15504 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55951bcb7550 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.388+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55951bbb0ce0 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5554003b60 con 0x55951bc9d1e0 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 20 client.15504 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bcb88a0 0x55951bcbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 4 client.15504 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951bc141d0 con 0x55951bcb88a0 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.389+0000 7f556a388640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bcb88a0 0x55951bcbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556a388640 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bcb88a0 0x55951bcbac90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e97658770b862c2 server_cookie=baaf0aac0911e666 in_seq=0 out_seq=0 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f554dffb640 10 client.15504 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f554dffb640 1 -- 172.21.15.60:0/688570726 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55951bc141d0 con 0x55951bcb88a0 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f554dffb640 10 client.15504 handle_command_reply: tid=0 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 client.15504 shutdown 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 msgr2=0x55951bca5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5554005940 tx=0x7f555402a410 comp rx=0 tx=0).stop 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 shutdown_connections 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bcb88a0 0x55951bcbac90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bca9170 0x55951bca62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9d1e0 0x55951bca5d70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f554dffb640 0 client.15504 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 --2- 172.21.15.60:0/688570726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bca8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.390+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 >> 172.21.15.60:0/688570726 conn(0x55951bc9a6f0 msgr2=0x55951bca0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.391+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 shutdown_connections 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.391+0000 7f556f417740 1 -- 172.21.15.60:0/688570726 wait complete. 2024-07-24T21:46:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.391+0000 7f556f417740 20 client.15504 trim_cache size 0 max 16384 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 Processor -- start 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 -- start start 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b2b20 con 0x55951bca9170 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b03f0 con 0x55951bc9d1e0 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556f417740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55641ae240 con 0x55951bc9c8c0 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51436/0 (socket says 172.21.15.60:51436) 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 -- 172.21.15.60:0/2981517436 learned_addr learned my addr 172.21.15.60:0/2981517436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556a388640 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 -- 172.21.15.60:0/2981517436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 msgr2=0x55951bc9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 -- 172.21.15.60:0/2981517436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 msgr2=0x55951bca54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556ab89640 1 -- 172.21.15.60:0/2981517436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951bca5a70 con 0x55951bca9170 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.406+0000 7f556a388640 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556ab89640 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f555c004790 tx=0x7f555c0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81e9f83247dd4b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f554ffff640 1 -- 172.21.15.60:0/2981517436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c018070 con 0x55951bca9170 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f554ffff640 1 -- 172.21.15.60:0/2981517436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555c025030 con 0x55951bca9170 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f554ffff640 1 -- 172.21.15.60:0/2981517436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c021840 con 0x55951bca9170 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 -- 172.21.15.60:0/2981517436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 msgr2=0x55951bcc0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f555c004790 tx=0x7f555c0099c0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 -- 172.21.15.60:0/2981517436 shutdown_connections 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bca9170 0x55951bcc0290 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bca54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 --2- 172.21.15.60:0/2981517436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bc9c8c0 0x55951bc9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 -- 172.21.15.60:0/2981517436 >> 172.21.15.60:0/2981517436 conn(0x55951bc9a6f0 msgr2=0x55951bca7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 -- 172.21.15.60:0/2981517436 shutdown_connections 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.407+0000 7f556f417740 1 -- 172.21.15.60:0/2981517436 wait complete. 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 -- start start 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 Processor -- start 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bcaec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 0x55951bd21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f55641ae8b0 con 0x55951bc9c8c0 2024-07-24T21:46:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55641afd20 con 0x55951bc9d1e0 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55641aef80 con 0x55951bca9170 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556f417740 10 client.0 _ll_register_callbacks cb 0x7f5554032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51452/0 (socket says 172.21.15.60:51452) 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 -- 172.21.15.60:0/1271467932 learned_addr learned my addr 172.21.15.60:0/1271467932 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f5569b87640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bcaec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 -- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 msgr2=0x55951bd21260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556ab89640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 0x55951bd21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 0x55951bd21260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 -- 172.21.15.60:0/1271467932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 msgr2=0x55951bcaec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bcaec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556a388640 1 -- 172.21.15.60:0/1271467932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951bd21950 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.408+0000 7f556ab89640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 0x55951bd21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556a388640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55600aa610 tx=0x7f55600aa100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec242c8e45454f67 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600aec80 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f55600a6c70 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555c025030 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f55600b3730 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 10 client.15510 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55951bd22cc0 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555c025030 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f55600e1490 con 0x55951bc9c8c0 2024-07-24T21:46:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 20 client.15510 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bd23390 0x55951bd257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 4 client.15510 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.409+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55951bc0fc70 con 0x55951bd23390 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.410+0000 7f5569b87640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bd23390 0x55951bd257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.410+0000 7f554dffb640 10 client.15510 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.410+0000 7f5569b87640 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bd23390 0x55951bd257b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6319adbf4956a41b server_cookie=ca6d798788d8d7cc in_seq=0 out_seq=0 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f554dffb640 1 -- 172.21.15.60:0/1271467932 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55951bc0fc70 con 0x55951bd23390 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f554dffb640 10 client.15510 handle_command_reply: tid=0 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 client.15510 shutdown 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 msgr2=0x55951bcae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55600aa610 tx=0x7f55600aa100 comp rx=0 tx=0).stop 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 shutdown_connections 2024-07-24T21:46:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55951bd23390 0x55951bd257b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55951bca9170 0x55951bd21260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f554dffb640 0 client.15510 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55951bc9d1e0 0x55951bcaec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 --2- 172.21.15.60:0/1271467932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55951bc9c8c0 0x55951bcae740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 >> 172.21.15.60:0/1271467932 conn(0x55951bc9a6f0 msgr2=0x55951bc98f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 shutdown_connections 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.418+0000 7f556f417740 1 -- 172.21.15.60:0/1271467932 wait complete. 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556f417740 20 client.15510 trim_cache size 0 max 16384 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:46:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 msgr2=0x7f55500b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 0x7f55500b61f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f555c005fd0 tx=0x7f555c004ab0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 msgr2=0x7f55640fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f556000d950 tx=0x7f556000de10 comp rx=0 tx=0).stop 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 shutdown_connections 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55500b3d30 0x7f55500b61f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564107950 0x7f55640fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5564106fd0 0x7f55640fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 --2- 172.21.15.60:0/1641503861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55641015d0 0x7f55640fd530 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 >> 172.21.15.60:0/1641503861 conn(0x7f55640fd030 msgr2=0x7f55640fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 shutdown_connections 2024-07-24T21:46:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:46:58.419+0000 7f556c5ec640 1 -- 172.21.15.60:0/1641503861 wait complete. 2024-07-24T21:46:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.643+0000 7f669233a640 1 -- 172.21.15.60:0/254846526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c107a00 msgr2=0x7f668c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.643+0000 7f669233a640 1 --2- 172.21.15.60:0/254846526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c107a00 0x7f668c10e200 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6684009ca0 tx=0x7f668402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 -- 172.21.15.60:0/254846526 shutdown_connections 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 --2- 172.21.15.60:0/254846526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c107a00 0x7f668c10e200 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 --2- 172.21.15.60:0/254846526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107080 0x7f668c1074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 --2- 172.21.15.60:0/254846526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c101670 0x7f668c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 -- 172.21.15.60:0/254846526 >> 172.21.15.60:0/254846526 conn(0x7f668c0fd0e0 msgr2=0x7f668c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 -- 172.21.15.60:0/254846526 shutdown_connections 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 -- 172.21.15.60:0/254846526 wait complete. 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 Processor -- start 2024-07-24T21:46:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.644+0000 7f669233a640 1 -- start start 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c107080 0x7f668c0fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 0x7f668c0fe060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f668c111630 con 0x7f668c101670 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1114b0 con 0x7f668c107a00 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f669233a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1117b0 con 0x7f668c107080 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51460/0 (socket says 172.21.15.60:51460) 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 -- 172.21.15.60:0/1262864308 learned_addr learned my addr 172.21.15.60:0/1262864308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668b7fe640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c107080 0x7f668c0fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f66908d7640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 0x7f668c0fe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 -- 172.21.15.60:0/1262864308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c107080 msgr2=0x7f668c0fdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c107080 0x7f668c0fdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 -- 172.21.15.60:0/1262864308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 msgr2=0x7f668c0fe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 0x7f668c0fe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 -- 172.21.15.60:0/1262864308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c1a6fa0 con 0x7f668c101670 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f66908d7640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 0x7f668c0fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f668bfff640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c890 tx=0x7f667c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.645+0000 7f66897fa640 1 -- 172.21.15.60:0/1262864308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c004220 con 0x7f668c101670 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.646+0000 7f66897fa640 1 -- 172.21.15.60:0/1262864308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667c0043c0 con 0x7f668c101670 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.646+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c1a7290 con 0x7f668c101670 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.646+0000 7f66897fa640 1 -- 172.21.15.60:0/1262864308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c004a00 con 0x7f668c101670 2024-07-24T21:46:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.646+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a77d0 con 0x7f668c101670 2024-07-24T21:46:58.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.648+0000 7f66897fa640 1 -- 172.21.15.60:0/1262864308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f667c0079b0 con 0x7f668c101670 2024-07-24T21:46:58.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.648+0000 7f66946eb740 1 Processor -- start 2024-07-24T21:46:58.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.648+0000 7f66946eb740 1 -- start start 2024-07-24T21:46:58.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.648+0000 7f66946eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 0x55720a66c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66946eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a66ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66946eb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66946eb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55720a5e2720 con 0x55720a678430 2024-07-24T21:46:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66946eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55720a5d1c20 con 0x55720a66c600 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66946eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55720a5d5c60 con 0x55720a66db60 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51466/0 (socket says 172.21.15.60:51466) 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 -- 172.21.15.60:0/3461909698 learned_addr learned my addr 172.21.15.60:0/3461909698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668bfff640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a66ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66908d7640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 0x55720a66c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 -- 172.21.15.60:0/3461909698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 msgr2=0x55720a66c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 0x55720a66c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 -- 172.21.15.60:0/3461909698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 msgr2=0x55720a66ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a66ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 -- 172.21.15.60:0/3461909698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720a55b1d0 con 0x55720a678430 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66908d7640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 0x55720a66c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668bfff640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a66ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66897fa640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 0x7f66740b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66908d7640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 0x7f66740b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f668b7fe640 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f668000aca0 tx=0x7f668000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ce26b71392dab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.649+0000 7f66897fa640 1 -- 172.21.15.60:0/1262864308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f667c0e3c40 con 0x7f668c101670 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66727fc640 1 -- 172.21.15.60:0/3461909698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001d070 con 0x55720a678430 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66727fc640 1 -- 172.21.15.60:0/3461909698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6680018af0 con 0x55720a678430 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66727fc640 1 -- 172.21.15.60:0/3461909698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680010b80 con 0x55720a678430 2024-07-24T21:46:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66946eb740 1 -- 172.21.15.60:0/3461909698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 msgr2=0x55720a67a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66946eb740 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f668000aca0 tx=0x7f668000f790 comp rx=0 tx=0).stop 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.650+0000 7f66908d7640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 0x7f66740b6170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6684009a60 tx=0x7f66840093f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- 172.21.15.60:0/3461909698 shutdown_connections 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a67a820 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a66ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- 172.21.15.60:0/3461909698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66db60 0x55720a66c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- 172.21.15.60:0/3461909698 >> 172.21.15.60:0/3461909698 conn(0x55720a60a6a0 msgr2=0x55720a669ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- 172.21.15.60:0/3461909698 shutdown_connections 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- 172.21.15.60:0/3461909698 wait complete. 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- start start 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 Processor -- start 2024-07-24T21:46:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a677b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66db60 0x55720a675070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55720a4355f0 con 0x55720a678430 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55720a5744a0 con 0x55720a66db60 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55720a67b250 con 0x55720a66c600 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.651+0000 7f66946eb740 10 client.0 _ll_register_callbacks cb 0x55720a5ca0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51480/0 (socket says 172.21.15.60:51480) 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 -- 172.21.15.60:0/930548362 learned_addr learned my addr 172.21.15.60:0/930548362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f668b7fe640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66db60 0x55720a675070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f668bfff640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a677b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 -- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 msgr2=0x55720a677b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a677b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 -- 172.21.15.60:0/930548362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66db60 msgr2=0x55720a675070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66db60 0x55720a675070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 -- 172.21.15.60:0/930548362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720a567530 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66908d7640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6684005fe0 tx=0x7f66840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec360b29ef96434 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f666b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668404e070 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6684004740 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.652+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6680018af0 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f668402f660 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 10 client.15522 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55720a332540 con 0x55720a678430 2024-07-24T21:46:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6680018af0 con 0x55720a678430 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6684043040 con 0x55720a678430 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 10 client.15522 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 20 client.15522 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a688f40 0x55720a68a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 4 client.15522 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720a5e4670 con 0x55720a688f40 2024-07-24T21:46:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.653+0000 7f668bfff640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a688f40 0x55720a68a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.654+0000 7f666b7fe640 10 client.15522 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.654+0000 7f668bfff640 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a688f40 0x55720a68a3b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b688f36ed84f5da server_cookie=d974b3cf13c25661 in_seq=0 out_seq=0 2024-07-24T21:46:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.654+0000 7f666b7fe640 1 -- 172.21.15.60:0/930548362 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55720a5e4670 con 0x55720a688f40 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.654+0000 7f666b7fe640 10 client.15522 handle_command_reply: tid=0 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.654+0000 7f66946eb740 1 client.15522 shutdown 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 msgr2=0x55720a6755b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6684005fe0 tx=0x7f66840095c0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 shutdown_connections 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a688f40 0x55720a68a3b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a678430 0x55720a6755b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66db60 0x55720a675070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f666b7fe640 0 client.15522 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 --2- 172.21.15.60:0/930548362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a677b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 >> 172.21.15.60:0/930548362 conn(0x55720a60a6a0 msgr2=0x55720a66fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 shutdown_connections 2024-07-24T21:46:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 1 -- 172.21.15.60:0/930548362 wait complete. 2024-07-24T21:46:58.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.655+0000 7f66946eb740 20 client.15522 trim_cache size 0 max 16384 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 Processor -- start 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 -- start start 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a66bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a678430 0x55720a669900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55720a6886f0 con 0x55720a66db60 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55720a688dc0 con 0x55720a678430 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f66946eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55720a689490 con 0x55720a66c600 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51488/0 (socket says 172.21.15.60:51488) 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 -- 172.21.15.60:0/710854009 learned_addr learned my addr 172.21.15.60:0/710854009 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 -- 172.21.15.60:0/710854009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 msgr2=0x55720a66bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a66bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 -- 172.21.15.60:0/710854009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a678430 msgr2=0x55720a669900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a678430 0x55720a669900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.674+0000 7f668b7fe640 1 -- 172.21.15.60:0/710854009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720a6696e0 con 0x55720a66db60 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f668b7fe640 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f668000a670 tx=0x7f6680005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7779fed5f6dde99 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f6671ffb640 1 -- 172.21.15.60:0/710854009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680027070 con 0x55720a66db60 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f6671ffb640 1 -- 172.21.15.60:0/710854009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668000a8d0 con 0x55720a66db60 2024-07-24T21:46:58.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f6671ffb640 1 -- 172.21.15.60:0/710854009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001d040 con 0x55720a66db60 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 -- 172.21.15.60:0/710854009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 msgr2=0x55720a664f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f668000a670 tx=0x7f6680005e40 comp rx=0 tx=0).stop 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 -- 172.21.15.60:0/710854009 shutdown_connections 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a678430 0x55720a669900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a664f10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 --2- 172.21.15.60:0/710854009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a66c600 0x55720a66bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 -- 172.21.15.60:0/710854009 >> 172.21.15.60:0/710854009 conn(0x55720a60a6a0 msgr2=0x55720a6738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.675+0000 7f66946eb740 1 -- 172.21.15.60:0/710854009 shutdown_connections 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 -- 172.21.15.60:0/710854009 wait complete. 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 -- start start 2024-07-24T21:46:58.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 Processor -- start 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a692610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a678430 0x55720a68d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55720a68a230 con 0x55720a66db60 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55720a689b60 con 0x55720a66c600 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55720a68a900 con 0x55720a678430 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f66946eb740 10 client.0 _ll_register_callbacks cb 0x7f6684009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f668b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51498/0 (socket says 172.21.15.60:51498) 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.676+0000 7f668b7fe640 1 -- 172.21.15.60:0/2132162232 learned_addr learned my addr 172.21.15.60:0/2132162232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f66908d7640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a678430 0x55720a68d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668bfff640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a692610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 -- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a678430 msgr2=0x55720a68d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a678430 0x55720a68d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 -- 172.21.15.60:0/2132162232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 msgr2=0x55720a692610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a692610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 -- 172.21.15.60:0/2132162232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720a6931f0 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f668b7fe640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f668000a310 tx=0x7f6680010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9701e2939d7f5f93 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f666b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680030070 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6680005040 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55720a5e0110 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f668003c050 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f66946eb740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.677+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55720a6f3b80 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55720a5e0110 con 0x55720a66db60 2024-07-24T21:46:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6680018440 con 0x55720a66db60 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 10 client.15528 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 20 client.15528 populate_metadata read hostname 'smithi060' 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a6f8030 0x55720a6f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 4 client.15528 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "status"}) -- 0x55720a5dd850 con 0x55720a6f8030 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.678+0000 7f668bfff640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a6f8030 0x55720a6f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f668bfff640 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a6f8030 0x55720a6f6d40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f079f6196c33f8ad server_cookie=6d9a700fb5fe0ba6 in_seq=0 out_seq=0 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f666b7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f666b7fe640 1 -- 172.21.15.60:0/2132162232 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55720a5dd850 con 0x55720a6f8030 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f666b7fe640 10 client.15528 handle_command_reply: tid=0 2024-07-24T21:46:58.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 client.15528 shutdown 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 msgr2=0x55720a692b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f668000a310 tx=0x7f6680010540 comp rx=0 tx=0).stop 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 shutdown_connections 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55720a6f8030 0x55720a6f6d40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55720a678430 0x55720a68d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55720a66db60 0x55720a692b50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 --2- 172.21.15.60:0/2132162232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55720a66c600 0x55720a692610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 >> 172.21.15.60:0/2132162232 conn(0x55720a60a6a0 msgr2=0x55720a669900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f666b7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 shutdown_connections 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 1 -- 172.21.15.60:0/2132162232 wait complete. 2024-07-24T21:46:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.679+0000 7f66946eb740 20 client.15528 trim_cache size 0 max 16384 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24598, 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 211.32083831899999, 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 26, 2024-07-24T21:46:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 254.313640956 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 msgr2=0x7f66740b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 0x7f66740b6170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6684009a60 tx=0x7f66840093f0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 msgr2=0x7f668c0fd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c890 tx=0x7f667c00cc90 comp rx=0 tx=0).stop 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 shutdown_connections 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f66740b3cb0 0x7f66740b6170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c107a00 0x7f668c0fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f668c107080 0x7f668c0fdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 --2- 172.21.15.60:0/1262864308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668c101670 0x7f668c0fd5e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 >> 172.21.15.60:0/1262864308 conn(0x7f668c0fd0e0 msgr2=0x7f668c06b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 shutdown_connections 2024-07-24T21:46:58.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:46:58.680+0000 7f669233a640 1 -- 172.21.15.60:0/1262864308 wait complete. 2024-07-24T21:46:59.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:46:59 smithi060 ceph-mon[33000]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-07-24T21:46:59.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.b status 2024-07-24T21:46:59.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:46:59 smithi123 ceph-mon[37143]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-07-24T21:46:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:46:59 smithi086 ceph-mon[37128]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-07-24T21:46:59.774 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:00.571 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:47:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:47:00.296+0000 7f88e21b8640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:47:00.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:47:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:47:00.296+0000 7f88e21b8640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:47:00.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:47:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:47:00.296+0000 7f88e21b8640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:47:00.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:47:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:47:00.296+0000 7f88e21b8640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:47:00.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 21:47:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T21:47:00.296+0000 7f88e21b8640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-24T21:47:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.357+0000 7f9f11975640 1 -- 172.21.15.60:0/2174927874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c1081a0 msgr2=0x7f9f0c1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.357+0000 7f9f11975640 1 --2- 172.21.15.60:0/2174927874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c1085e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef402a410 comp rx=0 tx=0).stop 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 -- 172.21.15.60:0/2174927874 shutdown_connections 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 --2- 172.21.15.60:0/2174927874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c108b20 0x7f9f0c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 --2- 172.21.15.60:0/2174927874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c1085e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 --2- 172.21.15.60:0/2174927874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c102790 0x7f9f0c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 -- 172.21.15.60:0/2174927874 >> 172.21.15.60:0/2174927874 conn(0x7f9f0c0fe200 msgr2=0x7f9f0c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 -- 172.21.15.60:0/2174927874 shutdown_connections 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 -- 172.21.15.60:0/2174927874 wait complete. 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 Processor -- start 2024-07-24T21:47:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.358+0000 7f9f11975640 1 -- start start 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c108b20 0x7f9f0c075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c112820 con 0x7f9f0c102790 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1126a0 con 0x7f9f0c1081a0 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1129a0 con 0x7f9f0c108b20 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51532/0 (socket says 172.21.15.60:51532) 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 -- 172.21.15.60:0/3689189359 learned_addr learned my addr 172.21.15.60:0/3689189359 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c108b20 0x7f9f0c075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f03fff640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 -- 172.21.15.60:0/3689189359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c108b20 msgr2=0x7f9f0c075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c108b20 0x7f9f0c075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 -- 172.21.15.60:0/3689189359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c1081a0 msgr2=0x7f9f0c0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 -- 172.21.15.60:0/3689189359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c1a7190 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f0affd640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b4a0 tx=0x7f9efc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f10973640 1 -- 172.21.15.60:0/3689189359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc014070 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f10973640 1 -- 172.21.15.60:0/3689189359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc010ac0 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f10973640 1 -- 172.21.15.60:0/3689189359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017c30 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.359+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c1a7480 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.360+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c1a7a10 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f10973640 1 -- 172.21.15.60:0/3689189359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 151133+0+0 (secure 0 0 0) 0x7f9efc017070 con 0x7f9f0c102790 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 Processor -- start 2024-07-24T21:47:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 -- start start 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3fb80 0x563022a3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 0x563022a3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5630229b4740 con 0x563022a4a450 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630229a3c40 con 0x563022a3e620 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.362+0000 7f9f13d26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630229a7c80 con 0x563022a3fb80 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51540/0 (socket says 172.21.15.60:51540) 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 -- 172.21.15.60:0/1089635240 learned_addr learned my addr 172.21.15.60:0/1089635240 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3fb80 0x563022a3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f0affd640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 0x563022a3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 -- 172.21.15.60:0/1089635240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3fb80 msgr2=0x563022a3e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3fb80 0x563022a3e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 -- 172.21.15.60:0/1089635240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 msgr2=0x563022a3ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 0x563022a3ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 -- 172.21.15.60:0/1089635240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302292d1f0 con 0x563022a4a450 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f0affd640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 0x563022a3ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f10973640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 0x7f9ef80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f03fff640 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ef402a930 tx=0x7f9ef4031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464fdac1b0078f27 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 0x7f9ef80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f10973640 1 -- 172.21.15.60:0/3689189359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9efc0e3520 con 0x7f9f0c102790 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f01ffb640 1 -- 172.21.15.60:0/1089635240 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef403f070 con 0x563022a4a450 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f01ffb640 1 -- 172.21.15.60:0/1089635240 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef40316a0 con 0x563022a4a450 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.363+0000 7f9f01ffb640 1 -- 172.21.15.60:0/1089635240 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4035070 con 0x563022a4a450 2024-07-24T21:47:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 -- 172.21.15.60:0/1089635240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 msgr2=0x563022a4c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ef402a930 tx=0x7f9ef4031cd0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 0x7f9ef80b6170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c076f50 tx=0x7f9f0400d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 -- 172.21.15.60:0/1089635240 shutdown_connections 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a4c840 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3e620 0x563022a3ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 --2- 172.21.15.60:0/1089635240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3fb80 0x563022a3e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 -- 172.21.15.60:0/1089635240 >> 172.21.15.60:0/1089635240 conn(0x5630229dc6c0 msgr2=0x563022a3bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 -- 172.21.15.60:0/1089635240 shutdown_connections 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.364+0000 7f9f13d26740 1 -- 172.21.15.60:0/1089635240 wait complete. 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 -- start start 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 Processor -- start 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a49b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3fb80 0x563022a47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5630228075f0 con 0x563022a4a450 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630229464c0 con 0x563022a3fb80 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563022a4d210 con 0x563022a3e620 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f13d26740 10 client.0 _ll_register_callbacks cb 0x56302299c0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51544/0 (socket says 172.21.15.60:51544) 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.365+0000 7f9f0b7fe640 1 -- 172.21.15.60:0/2852816902 learned_addr learned my addr 172.21.15.60:0/2852816902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f03fff640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3fb80 0x563022a47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0affd640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a49b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 -- 172.21.15.60:0/2852816902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 msgr2=0x563022a49b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a49b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 -- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3fb80 msgr2=0x563022a47060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3fb80 0x563022a47060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 -- 172.21.15.60:0/2852816902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563022939550 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f04018ea0 tx=0x7f9f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a95079551f6c32b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9eeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04028060 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9f04023670 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef40316a0 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f04023870 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f13d26740 10 client.15546 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.366+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563022704540 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef40316a0 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9f04017b00 con 0x563022a4a450 2024-07-24T21:47:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 10 client.15546 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 20 client.15546 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022a5ae50 0x563022a5c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 4 client.15546 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630229b6690 con 0x563022a5ae50 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.367+0000 7f9f0affd640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022a5ae50 0x563022a5c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9eeaffd640 10 client.15546 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f0affd640 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022a5ae50 0x563022a5c2c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bff5598d89c25f24 server_cookie=22d56eb149573931 in_seq=0 out_seq=0 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2852816902 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630229b6690 con 0x563022a5ae50 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9eeaffd640 10 client.15546 handle_command_reply: tid=0 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 client.15546 shutdown 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 msgr2=0x563022a475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f04018ea0 tx=0x7f9f04004a40 comp rx=0 tx=0).stop 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 shutdown_connections 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022a5ae50 0x563022a5c2c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a4a450 0x563022a475a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a3fb80 0x563022a47060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 --2- 172.21.15.60:0/2852816902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a49b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9eeaffd640 0 client.15546 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.368+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 >> 172.21.15.60:0/2852816902 conn(0x5630229dc6c0 msgr2=0x563022a41ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.369+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 shutdown_connections 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.369+0000 7f9f13d26740 1 -- 172.21.15.60:0/2852816902 wait complete. 2024-07-24T21:47:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.369+0000 7f9f13d26740 20 client.15546 trim_cache size 0 max 16384 2024-07-24T21:47:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.388+0000 7f9f13d26740 1 Processor -- start 2024-07-24T21:47:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.388+0000 7f9f13d26740 1 -- start start 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563022a5a600 con 0x563022a3fb80 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563022a5acd0 con 0x563022a4a450 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f13d26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563022a5b3a0 con 0x563022a3e620 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51552/0 (socket says 172.21.15.60:51552) 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 -- 172.21.15.60:0/4054311450 learned_addr learned my addr 172.21.15.60:0/4054311450 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f0affd640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 -- 172.21.15.60:0/4054311450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 msgr2=0x563022a3de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a3de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 -- 172.21.15.60:0/4054311450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 msgr2=0x563022a3bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a3bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 -- 172.21.15.60:0/4054311450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563022a47a80 con 0x563022a3fb80 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f03fff640 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef40052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778023461d18c70e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f017fa640 1 -- 172.21.15.60:0/4054311450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef403f070 con 0x563022a3fb80 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f017fa640 1 -- 172.21.15.60:0/4054311450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef404b070 con 0x563022a3fb80 2024-07-24T21:47:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.389+0000 7f9f017fa640 1 -- 172.21.15.60:0/4054311450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4035040 con 0x563022a3fb80 2024-07-24T21:47:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 -- 172.21.15.60:0/4054311450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 msgr2=0x563022a47540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef40052d0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 -- 172.21.15.60:0/4054311450 shutdown_connections 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a47540 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 --2- 172.21.15.60:0/4054311450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a3de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 -- 172.21.15.60:0/4054311450 >> 172.21.15.60:0/4054311450 conn(0x5630229dc6c0 msgr2=0x563022a46e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 -- 172.21.15.60:0/4054311450 shutdown_connections 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.390+0000 7f9f13d26740 1 -- 172.21.15.60:0/4054311450 wait complete. 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 -- start start 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 Processor -- start 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a64520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563022a5c140 con 0x563022a3fb80 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563022a5ba70 con 0x563022a4a450 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563022a5c810 con 0x563022a3e620 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51560/0 (socket says 172.21.15.60:51560) 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f13d26740 10 client.0 _ll_register_callbacks cb 0x7f9f04009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f03fff640 1 -- 172.21.15.60:0/358683656 learned_addr learned my addr 172.21.15.60:0/358683656 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f0affd640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a64520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f03fff640 1 -- 172.21.15.60:0/358683656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 msgr2=0x563022a64520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.391+0000 7f9f03fff640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a64520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f03fff640 1 -- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 msgr2=0x563022a5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f03fff640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a5f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f03fff640 1 -- 172.21.15.60:0/358683656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563022a5fb10 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f0b7fe640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f03fff640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1604bb9dc9606de9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9eeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4035040 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9ef40049a0 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef404b070 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ef405f050 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f13d26740 10 client.15552 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563022ac5ea0 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.392+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef404b070 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9ef40038f0 con 0x563022a3fb80 2024-07-24T21:47:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 10 client.15552 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 10 client.15552 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 20 client.15552 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022aca3d0 0x563022ac90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 4 client.15552 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "status"}) -- 0x563022a468c0 con 0x563022aca3d0 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f0affd640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022aca3d0 0x563022ac90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9f0affd640 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022aca3d0 0x563022ac90e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e168413b0187c567 server_cookie=68da7c129ca708a5 in_seq=0 out_seq=0 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.393+0000 7f9eeaffd640 10 client.15552 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9eeaffd640 1 -- 172.21.15.60:0/358683656 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563022a468c0 con 0x563022aca3d0 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9eeaffd640 10 client.15552 handle_command_reply: tid=0 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 client.15552 shutdown 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 msgr2=0x563022a64a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef4000bc0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 shutdown_connections 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563022aca3d0 0x563022ac90e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563022a4a450 0x563022a5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563022a3fb80 0x563022a64a60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 --2- 172.21.15.60:0/358683656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563022a3e620 0x563022a64520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9eeaffd640 0 client.15552 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 >> 172.21.15.60:0/358683656 conn(0x5630229dc6c0 msgr2=0x563022a41f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 shutdown_connections 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 1 -- 172.21.15.60:0/358683656 wait complete. 2024-07-24T21:47:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.394+0000 7f9f13d26740 20 client.15552 trim_cache size 0 max 16384 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24605, 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 215.041732411, 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 26, 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:47:01.403 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-07-24T21:47:01.404 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 242.182131989 2024-07-24T21:47:01.404 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 msgr2=0x7f9ef80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 0x7f9ef80b6170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c076f50 tx=0x7f9f0400d040 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 msgr2=0x7f9f0c074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00b4a0 tx=0x7f9efc00b960 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 shutdown_connections 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ef80b3cb0 0x7f9ef80b6170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f0c108b20 0x7f9f0c075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f0c1081a0 0x7f9f0c0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 --2- 172.21.15.60:0/3689189359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0c102790 0x7f9f0c074e90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 >> 172.21.15.60:0/3689189359 conn(0x7f9f0c0fe200 msgr2=0x7f9f0c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 shutdown_connections 2024-07-24T21:47:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:01.395+0000 7f9f11975640 1 -- 172.21.15.60:0/3689189359 wait complete. 2024-07-24T21:47:01.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:01 smithi123 ceph-mon[37143]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-24T21:47:01.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:01 smithi086 ceph-mon[37128]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-24T21:47:01.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:01 smithi060 ceph-mon[33000]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-24T21:47:02.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.i status 2024-07-24T21:47:02.314 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:03.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:03 smithi123 ceph-mon[37143]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-07-24T21:47:03.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:03 smithi123 ceph-mon[37143]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-07-24T21:47:03.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:03 smithi086 ceph-mon[37128]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-07-24T21:47:03.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:03 smithi086 ceph-mon[37128]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-07-24T21:47:03.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:03 smithi060 ceph-mon[33000]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-07-24T21:47:03.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:03 smithi060 ceph-mon[33000]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 -- 172.21.15.60:0/3471737586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe041068d0 msgr2=0x7efe04106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 --2- 172.21.15.60:0/3471737586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe041068d0 0x7efe04106d20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009ca0 tx=0x7efdfc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 -- 172.21.15.60:0/3471737586 shutdown_connections 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 --2- 172.21.15.60:0/3471737586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe041068d0 0x7efe04106d20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 --2- 172.21.15.60:0/3471737586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe04106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 --2- 172.21.15.60:0/3471737586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04101490 0x7efe04101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 -- 172.21.15.60:0/3471737586 >> 172.21.15.60:0/3471737586 conn(0x7efe040fd100 msgr2=0x7efe040ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.890+0000 7efe0c7da640 1 -- 172.21.15.60:0/3471737586 shutdown_connections 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 -- 172.21.15.60:0/3471737586 wait complete. 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 Processor -- start 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 -- start start 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe0419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 0x7efe041a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efe04118d90 con 0x7efe04101490 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe04118c10 con 0x7efe04101db0 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0c7da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe04118f10 con 0x7efe041068d0 2024-07-24T21:47:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0a576640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0a576640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51568/0 (socket says 172.21.15.60:51568) 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.891+0000 7efe0a576640 1 -- 172.21.15.60:0/2627909230 learned_addr learned my addr 172.21.15.60:0/2627909230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe09d75640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe0419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0ad77640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 0x7efe041a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 -- 172.21.15.60:0/2627909230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 msgr2=0x7efe041a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 0x7efe041a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 -- 172.21.15.60:0/2627909230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 msgr2=0x7efe0419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe0419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 -- 172.21.15.60:0/2627909230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041a52d0 con 0x7efe04101490 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe09d75640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe0419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0ad77640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 0x7efe041a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0a576640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d770 tx=0x7efe0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efdfb7fe640 1 -- 172.21.15.60:0/2627909230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00017070 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041a55c0 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efdfb7fe640 1 -- 172.21.15.60:0/2627909230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe000054d0 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efdfb7fe640 1 -- 172.21.15.60:0/2627909230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00004c30 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.892+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041abc80 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efdfb7fe640 1 -- 172.21.15.60:0/2627909230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7efe00004e10 con 0x7efe04101490 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 Processor -- start 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 -- start start 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f0fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 0x564f4f10a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f4f072720 con 0x564f4f0fdb60 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f4f061c20 con 0x564f4f0fc600 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f4f065c60 con 0x564f4f108430 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.895+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51576/0 (socket says 172.21.15.60:51576) 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 -- 172.21.15.60:0/2648739005 learned_addr learned my addr 172.21.15.60:0/2648739005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0a576640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f0fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe09d75640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 0x564f4f10a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 -- 172.21.15.60:0/2648739005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 msgr2=0x564f4f10a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 0x564f4f10a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 -- 172.21.15.60:0/2648739005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 msgr2=0x564f4f0fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f0fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 -- 172.21.15.60:0/2648739005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4efeb1d0 con 0x564f4f0fdb60 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0a576640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f0fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe09d75640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 0x564f4f10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0ad77640 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0098e0 tx=0x7efdfc009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637e9ca90cc57227 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efdfb7fe640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 0x7efdf00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efdf8ff9640 1 -- 172.21.15.60:0/2648739005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc031df0 con 0x564f4f0fdb60 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efdf8ff9640 1 -- 172.21.15.60:0/2648739005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc036070 con 0x564f4f0fdb60 2024-07-24T21:47:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efdf8ff9640 1 -- 172.21.15.60:0/2648739005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc004380 con 0x564f4f0fdb60 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.896+0000 7efe0a576640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 0x7efdf00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efdfb7fe640 1 -- 172.21.15.60:0/2627909230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efe000e3bd0 con 0x7efe04101490 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0a576640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 0x7efdf00b6200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe000e13d0 tx=0x7efe00009f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- 172.21.15.60:0/2648739005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 msgr2=0x564f4f0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0098e0 tx=0x7efdfc009910 comp rx=0 tx=0).stop 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- 172.21.15.60:0/2648739005 shutdown_connections 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f108430 0x564f4f10a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f0fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/2648739005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fdb60 0x564f4f0fc0c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- 172.21.15.60:0/2648739005 >> 172.21.15.60:0/2648739005 conn(0x564f4f09a6a0 msgr2=0x564f4f0f9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- 172.21.15.60:0/2648739005 shutdown_connections 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- 172.21.15.60:0/2648739005 wait complete. 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.897+0000 7efe0eb8b740 1 -- start start 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 Processor -- start 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fc600 0x564f4f107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fdb60 0x564f4f1050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f4eec55f0 con 0x564f4f108430 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f4f0044a0 con 0x564f4f0fdb60 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f4f10b1f0 con 0x564f4f0fc600 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0eb8b740 10 client.0 _ll_register_callbacks cb 0x564f4f05a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51586/0 (socket says 172.21.15.60:51586) 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 -- 172.21.15.60:0/3133754742 learned_addr learned my addr 172.21.15.60:0/3133754742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fc600 0x564f4f107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe09d75640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fdb60 0x564f4f1050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 -- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fc600 msgr2=0x564f4f107c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fc600 0x564f4f107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 -- 172.21.15.60:0/3133754742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fdb60 msgr2=0x564f4f1050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fdb60 0x564f4f1050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.898+0000 7efe0ad77640 1 -- 172.21.15.60:0/3133754742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4eff7530 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0ad77640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003fd0 tx=0x7efdfc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1022773f4cfc490f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc049040 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efdfc031df0 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc036070 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdfc004b40 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 10 client.15564 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f4edc2540 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc036070 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efdfc031550 con 0x564f4f108430 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 10 client.15564 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 20 client.15564 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f119e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 4 client.15564 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.899+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4f074670 con 0x564f4f1189d0 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.900+0000 7efe09d75640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f119e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.900+0000 7efe09d75640 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f119e40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c768e056cf207989 server_cookie=4264a04daf1f499a in_seq=0 out_seq=0 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.900+0000 7efdedffb640 10 client.15564 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efdedffb640 1 -- 172.21.15.60:0/3133754742 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f4f074670 con 0x564f4f1189d0 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efdedffb640 10 client.15564 handle_command_reply: tid=0 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 client.15564 shutdown 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 msgr2=0x564f4f105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003fd0 tx=0x7efdfc004040 comp rx=0 tx=0).stop 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 shutdown_connections 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f119e40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f105630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fdb60 0x564f4f1050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efdedffb640 0 client.15564 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/3133754742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fc600 0x564f4f107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 >> 172.21.15.60:0/3133754742 conn(0x564f4f09a6a0 msgr2=0x564f4f0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.901+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 shutdown_connections 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.902+0000 7efe0eb8b740 1 -- 172.21.15.60:0/3133754742 wait complete. 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.902+0000 7efe0eb8b740 20 client.15564 trim_cache size 0 max 16384 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.920+0000 7efe0eb8b740 1 Processor -- start 2024-07-24T21:47:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.920+0000 7efe0eb8b740 1 -- start start 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f0f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f108430 0x564f4f100190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f4f118180 con 0x564f4f0fc600 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f4f118850 con 0x564f4f108430 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f4f118f20 con 0x564f4f0fdb60 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51588/0 (socket says 172.21.15.60:51588) 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 -- 172.21.15.60:0/472201533 learned_addr learned my addr 172.21.15.60:0/472201533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe09d75640 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f0f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 -- 172.21.15.60:0/472201533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 msgr2=0x564f4f0f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f0f9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 -- 172.21.15.60:0/472201533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f108430 msgr2=0x564f4f100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f108430 0x564f4f100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 -- 172.21.15.60:0/472201533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4f0fd770 con 0x564f4f0fc600 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe09d75640 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f0f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efe0ad77640 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efdfc031a90 tx=0x7efdfc02f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3821141127098d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efdeffff640 1 -- 172.21.15.60:0/472201533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02f830 con 0x564f4f0fc600 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efdeffff640 1 -- 172.21.15.60:0/472201533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc036040 con 0x564f4f0fc600 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.921+0000 7efdeffff640 1 -- 172.21.15.60:0/472201533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc003970 con 0x564f4f0fc600 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- 172.21.15.60:0/472201533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 msgr2=0x564f4f0fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efdfc031a90 tx=0x7efdfc02f760 comp rx=0 tx=0).stop 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- 172.21.15.60:0/472201533 shutdown_connections 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f108430 0x564f4f100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f0f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/472201533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f0fc600 0x564f4f0fbdf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- 172.21.15.60:0/472201533 >> 172.21.15.60:0/472201533 conn(0x564f4f09a6a0 msgr2=0x564f4f106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- 172.21.15.60:0/472201533 shutdown_connections 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- 172.21.15.60:0/472201533 wait complete. 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.922+0000 7efe0eb8b740 1 -- start start 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 Processor -- start 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f1220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f1225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f4f119cc0 con 0x564f4f108430 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f4f1195f0 con 0x564f4f0fc600 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f4f11a390 con 0x564f4f0fdb60 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0eb8b740 10 client.0 _ll_register_callbacks cb 0x7efdfc009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:03.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51594/0 (socket says 172.21.15.60:51594) 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 -- 172.21.15.60:0/1991577189 learned_addr learned my addr 172.21.15.60:0/1991577189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f1220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe09d75640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f1225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 -- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 msgr2=0x564f4f1225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f1225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 -- 172.21.15.60:0/1991577189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 msgr2=0x564f4f1220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f1220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.923+0000 7efe0ad77640 1 -- 172.21.15.60:0/1991577189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4f11b630 con 0x564f4f108430 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0ad77640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003fd0 tx=0x7efdfc02f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7734096616a684 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02fd60 con 0x564f4f108430 2024-07-24T21:47:03.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efdfc003970 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc036040 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdfc003b70 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 10 client.15570 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f4f1839c0 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc036040 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efdfc0305c0 con 0x564f4f108430 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 10 client.15570 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 20 client.15570 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f186f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 4 client.15570 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.924+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "status"}) -- 0x564f4f070110 con 0x564f4f1189d0 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe09d75640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f186f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:03.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efdedffb640 10 client.15570 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe09d75640 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f186f40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88cbd14cd706fd79 server_cookie=7e008b20b35be408 in_seq=0 out_seq=0 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efdedffb640 1 -- 172.21.15.60:0/1991577189 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564f4f070110 con 0x564f4f1189d0 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efdedffb640 10 client.15570 handle_command_reply: tid=0 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 client.15570 shutdown 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 msgr2=0x564f4f11b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003fd0 tx=0x7efdfc02f4f0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 shutdown_connections 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f4f1189d0 0x564f4f186f40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.925+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f4f108430 0x564f4f11b040 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f4f0fdb60 0x564f4f1225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 1 --2- 172.21.15.60:0/1991577189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4f0fc600 0x564f4f1220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efdedffb640 0 client.15570 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 >> 172.21.15.60:0/1991577189 conn(0x564f4f09a6a0 msgr2=0x564f4f103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 shutdown_connections 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 1 -- 172.21.15.60:0/1991577189 wait complete. 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0eb8b740 20 client.15570 trim_cache size 0 max 16384 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "646042b4-4a04-11ef-bcad-c7b262605968", 2024-07-24T21:47:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24610, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 229.63651618200001, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 26, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 254.698997378 2024-07-24T21:47:03.947 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 msgr2=0x7efdf00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 0x7efdf00b6200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe000e13d0 tx=0x7efe00009f90 comp rx=0 tx=0).stop 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 msgr2=0x7efe0419f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d770 tx=0x7efe0000dc30 comp rx=0 tx=0).stop 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 shutdown_connections 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdf00b3d40 0x7efdf00b6200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe041068d0 0x7efe041a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe04101db0 0x7efe0419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 --2- 172.21.15.60:0/2627909230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe04101490 0x7efe0419f220 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:03.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 >> 172.21.15.60:0/2627909230 conn(0x7efe040fd100 msgr2=0x7efe041041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:03.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 shutdown_connections 2024-07-24T21:47:03.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:03.926+0000 7efe0c7da640 1 -- 172.21.15.60:0/2627909230 wait complete. 2024-07-24T21:47:04.622 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-07-24T21:47:05.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:05 smithi123 ceph-mon[37143]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:05 smithi086 ceph-mon[37128]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:05.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:05 smithi060 ceph-mon[33000]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:07.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:07 smithi060 ceph-mon[33000]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:07.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:07 smithi123 ceph-mon[37143]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:07 smithi086 ceph-mon[37128]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:07.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:47:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:09.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:09 smithi060 ceph-mon[33000]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-07-24T21:47:09.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:09 smithi123 ceph-mon[37143]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-07-24T21:47:09.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:09 smithi086 ceph-mon[37128]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 -- 172.21.15.60:0/3430522857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 msgr2=0x7f130c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 --2- 172.21.15.60:0/3430522857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c10e1e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1304009ca0 tx=0x7f130402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 -- 172.21.15.60:0/3430522857 shutdown_connections 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 --2- 172.21.15.60:0/3430522857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c10e1e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 --2- 172.21.15.60:0/3430522857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 0x7f130c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 --2- 172.21.15.60:0/3430522857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 -- 172.21.15.60:0/3430522857 >> 172.21.15.60:0/3430522857 conn(0x7f130c0fd0c0 msgr2=0x7f130c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 -- 172.21.15.60:0/3430522857 shutdown_connections 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.595+0000 7f1310b05640 1 -- 172.21.15.60:0/3430522857 wait complete. 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 Processor -- start 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 -- start start 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c0fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 0x7f130c0fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f130c111610 con 0x7f130c1079e0 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f130c111490 con 0x7f130c107060 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1310b05640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f130c111790 con 0x7f130c101650 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f130ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f130ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49766/0 (socket says 172.21.15.60:49766) 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f130ad9d640 1 -- 172.21.15.60:0/750013728 learned_addr learned my addr 172.21.15.60:0/750013728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.596+0000 7f1309d9b640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 0x7f130c0fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130a59c640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c0fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 -- 172.21.15.60:0/750013728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 msgr2=0x7f130c0fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c0fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 -- 172.21.15.60:0/750013728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 msgr2=0x7f130c0fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 0x7f130c0fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 -- 172.21.15.60:0/750013728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c1a7040 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130a59c640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c0fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f130ad9d640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1304005cc0 tx=0x7f1304031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f12fb7fe640 1 -- 172.21.15.60:0/750013728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304003b70 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c1a72d0 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f12fb7fe640 1 -- 172.21.15.60:0/750013728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13040318f0 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f12fb7fe640 1 -- 172.21.15.60:0/750013728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130402f950 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.597+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c1a7810 con 0x7f130c1079e0 2024-07-24T21:47:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f12fb7fe640 1 -- 172.21.15.60:0/750013728 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f130402fb50 con 0x7f130c1079e0 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 Processor -- start 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 -- start start 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04879a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564e047e4990 con 0x564e0486d6b0 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e047d3e90 con 0x564e04879f60 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.600+0000 7f1312eb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e047d7ed0 con 0x564e0486dfd0 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49778/0 (socket says 172.21.15.60:49778) 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 -- 172.21.15.60:0/2904606533 learned_addr learned my addr 172.21.15.60:0/2904606533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f130a59c640 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04879a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f12fb7fe640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 0x7f12f40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 -- 172.21.15.60:0/2904606533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 msgr2=0x564e04879a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04879a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 -- 172.21.15.60:0/2904606533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 msgr2=0x564e0487c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 -- 172.21.15.60:0/2904606533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0475d300 con 0x564e0486d6b0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f130a59c640 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04879a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f12fb7fe640 1 -- 172.21.15.60:0/750013728 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1304105370 con 0x7f130c1079e0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f130ad9d640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 0x7f12f40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.601+0000 7f1309d9b640 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1300009560 tx=0x7f1300011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e3cfa91b1e2e11 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f12f8ff9640 1 -- 172.21.15.60:0/2904606533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130001e070 con 0x564e0486d6b0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f12f8ff9640 1 -- 172.21.15.60:0/2904606533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1300004250 con 0x564e0486d6b0 2024-07-24T21:47:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f12f8ff9640 1 -- 172.21.15.60:0/2904606533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300019690 con 0x564e0486d6b0 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 -- 172.21.15.60:0/2904606533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 msgr2=0x564e0486da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f130ad9d640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 0x7f12f40b6200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1304005bc0 tx=0x7f130403e040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1300009560 tx=0x7f1300011810 comp rx=0 tx=0).stop 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 -- 172.21.15.60:0/2904606533 shutdown_connections 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04879a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 --2- 172.21.15.60:0/2904606533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486d6b0 0x564e0486da90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.602+0000 7f1312eb6740 1 -- 172.21.15.60:0/2904606533 >> 172.21.15.60:0/2904606533 conn(0x564e0486b4e0 msgr2=0x564e0486c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- 172.21.15.60:0/2904606533 shutdown_connections 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- 172.21.15.60:0/2904606533 wait complete. 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- start start 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 Processor -- start 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e04876ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e04877950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564e04637700 con 0x564e0486dfd0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e047765d0 con 0x564e04879f60 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e0487cbe0 con 0x564e0486d6b0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f1312eb6740 10 client.0 _ll_register_callbacks cb 0x564e04796940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.603+0000 7f130a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49790/0 (socket says 172.21.15.60:49790) 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 -- 172.21.15.60:0/322121573 learned_addr learned my addr 172.21.15.60:0/322121573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f1309d9b640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e04876ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f1309d9b640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e04877950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 -- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 msgr2=0x564e04876ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e04876ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 -- 172.21.15.60:0/322121573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 msgr2=0x564e04877950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e04877950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 -- 172.21.15.60:0/322121573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e04769660 con 0x564e0486dfd0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f130a59c640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f12fc013790 tx=0x7f12fc013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb55d7bef47513f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f12f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011dd0 con 0x564e0486dfd0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f12fc005020 con 0x564e0486dfd0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1300004250 con 0x564e0486dfd0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12fc005220 con 0x564e0486dfd0 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f1312eb6740 10 client.15582 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.604+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e04534650 con 0x564e0486dfd0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1300004250 con 0x564e0486dfd0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f12fc01a030 con 0x564e0486dfd0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 20 client.15582 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e04889b80 0x564e0488bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 4 client.15582 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e047e68e0 con 0x564e04889b80 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.605+0000 7f1309d9b640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e04889b80 0x564e0488bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.606+0000 7f1309d9b640 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e04889b80 0x564e0488bfa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04b58630405fe3c server_cookie=5187825d68e2e1c1 in_seq=0 out_seq=0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.606+0000 7f12f1ffb640 10 client.15582 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.606+0000 7f12f1ffb640 1 -- 172.21.15.60:0/322121573 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e047e68e0 con 0x564e04889b80 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.606+0000 7f12f1ffb640 10 client.15582 handle_command_reply: tid=0 2024-07-24T21:47:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.606+0000 7f1312eb6740 1 client.15582 shutdown 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 msgr2=0x564e04877410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f12fc013790 tx=0x7f12fc013c50 comp rx=0 tx=0).stop 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 shutdown_connections 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e04889b80 0x564e0488bfa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e04877950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e04877410 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f12f1ffb640 0 client.15582 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 --2- 172.21.15.60:0/322121573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e04876ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 >> 172.21.15.60:0/322121573 conn(0x564e0486b4e0 msgr2=0x564e04871080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 shutdown_connections 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 1 -- 172.21.15.60:0/322121573 wait complete. 2024-07-24T21:47:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.607+0000 7f1312eb6740 20 client.15582 trim_cache size 0 max 16384 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.625+0000 7f1312eb6740 1 Processor -- start 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 -- start start 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e0486d6b0 0x564e04891870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04891db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b2b70 con 0x564e04879f60 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1b0440 con 0x564e0486d6b0 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1312eb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1ae290 con 0x564e0486dfd0 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49802/0 (socket says 172.21.15.60:49802) 2024-07-24T21:47:09.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 -- 172.21.15.60:0/947953565 learned_addr learned my addr 172.21.15.60:0/947953565 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e0486d6b0 0x564e04891870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f130a59c640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04891db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 -- 172.21.15.60:0/947953565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 msgr2=0x564e04891db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04891db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 -- 172.21.15.60:0/947953565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e0486d6b0 msgr2=0x564e04891870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e0486d6b0 0x564e04891870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f1309d9b640 1 -- 172.21.15.60:0/947953565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0486b310 con 0x564e04879f60 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.626+0000 7f130a59c640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04891db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.627+0000 7f1309d9b640 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1300009b10 tx=0x7f1300011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25ff9ff0cae7475 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.627+0000 7f12f3fff640 1 -- 172.21.15.60:0/947953565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130001e040 con 0x564e04879f60 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.627+0000 7f12f3fff640 1 -- 172.21.15.60:0/947953565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130001f040 con 0x564e04879f60 2024-07-24T21:47:09.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.627+0000 7f12f3fff640 1 -- 172.21.15.60:0/947953565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300004680 con 0x564e04879f60 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- 172.21.15.60:0/947953565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 msgr2=0x564e04875750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1300009b10 tx=0x7f1300011810 comp rx=0 tx=0).stop 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- 172.21.15.60:0/947953565 shutdown_connections 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e04879f60 0x564e04875750 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486dfd0 0x564e04891db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- 172.21.15.60:0/947953565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e0486d6b0 0x564e04891870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- 172.21.15.60:0/947953565 >> 172.21.15.60:0/947953565 conn(0x564e0486b4e0 msgr2=0x564e04871010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- 172.21.15.60:0/947953565 shutdown_connections 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- 172.21.15.60:0/947953565 wait complete. 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- start start 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 Processor -- start 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e048822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1ae900 con 0x564e0486dfd0 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1afd70 con 0x564e04879f60 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1aefd0 con 0x564e0486d6b0 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.628+0000 7f1312eb6740 10 client.0 _ll_register_callbacks cb 0x7f12fc009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49818/0 (socket says 172.21.15.60:49818) 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 -- 172.21.15.60:0/182291706 learned_addr learned my addr 172.21.15.60:0/182291706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f1309d9b640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e048822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f1309d9b640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 -- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 msgr2=0x564e048822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e048822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 -- 172.21.15.60:0/182291706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 msgr2=0x564e0487dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 -- 172.21.15.60:0/182291706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0487e270 con 0x564e0486dfd0 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f130a59c640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009910 tx=0x7f12fc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421fa479ce51020b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f12f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc009e80 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f12fc00fb50 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.629+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130001f040 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12fc03b050 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 10 client.15588 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e048f4640 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130001f040 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f12fc01a040 con 0x564e0486dfd0 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 20 client.15588 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e048f4de0 0x564e048f7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 4 client.15588 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:47:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.630+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e047e2380 con 0x564e048f4de0 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1309d9b640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e048f4de0 0x564e048f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1309d9b640 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e048f4de0 0x564e048f7200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2af5e9fe433bdb server_cookie=e754cc894330fa06 in_seq=0 out_seq=0 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f12f1ffb640 10 client.15588 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f12f1ffb640 1 -- 172.21.15.60:0/182291706 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564e047e2380 con 0x564e048f4de0 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f12f1ffb640 10 client.15588 handle_command_reply: tid=0 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 client.15588 shutdown 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 msgr2=0x564e0487d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009910 tx=0x7f12fc011e00 comp rx=0 tx=0).stop 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 shutdown_connections 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564e048f4de0 0x564e048f7200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e04879f60 0x564e0487dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e0486dfd0 0x564e0487d770 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f12f1ffb640 0 client.15588 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 --2- 172.21.15.60:0/182291706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e0486d6b0 0x564e048822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.631+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 >> 172.21.15.60:0/182291706 conn(0x564e0486b4e0 msgr2=0x564e04878500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 shutdown_connections 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1312eb6740 1 -- 172.21.15.60:0/182291706 wait complete. 2024-07-24T21:47:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1312eb6740 20 client.15588 trim_cache size 0 max 16384 2024-07-24T21:47:09.640 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:09.640 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:47:09.640 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:47:09.640 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 msgr2=0x7f12f40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 0x7f12f40b6200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1304005bc0 tx=0x7f130403e040 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 msgr2=0x7f130c0fe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1304005cc0 tx=0x7f1304031700 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 shutdown_connections 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12f40b3d40 0x7f12f40b6200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c1079e0 0x7f130c0fe040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f130c107060 0x7f130c0fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 --2- 172.21.15.60:0/750013728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f130c101650 0x7f130c0fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 >> 172.21.15.60:0/750013728 conn(0x7f130c0fd0c0 msgr2=0x7f130c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.632+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 shutdown_connections 2024-07-24T21:47:09.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:09.633+0000 7f1310b05640 1 -- 172.21.15.60:0/750013728 wait complete. 2024-07-24T21:47:10.374 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:47:10.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:47:10.652 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:11.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:11 smithi060 ceph-mon[33000]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:11.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:11 smithi123 ceph-mon[37143]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 -- 172.21.15.60:0/4255781723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a81015e0 msgr2=0x7f00a81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 --2- 172.21.15.60:0/4255781723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a81015e0 0x7f00a81019c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f009c009900 tx=0x7f009c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 -- 172.21.15.60:0/4255781723 shutdown_connections 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 --2- 172.21.15.60:0/4255781723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8107910 0x7f00a810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 --2- 172.21.15.60:0/4255781723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a8106f90 0x7f00a81073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 --2- 172.21.15.60:0/4255781723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a81015e0 0x7f00a81019c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 -- 172.21.15.60:0/4255781723 >> 172.21.15.60:0/4255781723 conn(0x7f00a80782e0 msgr2=0x7f00a80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 -- 172.21.15.60:0/4255781723 shutdown_connections 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.513+0000 7f00aea95640 1 -- 172.21.15.60:0/4255781723 wait complete. 2024-07-24T21:47:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 Processor -- start 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 -- start start 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a81015e0 0x7f00a8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 0x7f00a8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8111560 con 0x7f00a8107910 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81113e0 con 0x7f00a8106f90 2024-07-24T21:47:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00aea95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81116e0 con 0x7f00a81015e0 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00ad032640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 0x7f00a8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00ad032640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49842/0 (socket says 172.21.15.60:49842) 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00ad032640 1 -- 172.21.15.60:0/615520587 learned_addr learned my addr 172.21.15.60:0/615520587 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.514+0000 7f00ac831640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a81015e0 0x7f00a8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 -- 172.21.15.60:0/615520587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a81015e0 msgr2=0x7f00a8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a81015e0 0x7f00a8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 -- 172.21.15.60:0/615520587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 msgr2=0x7f00a8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 0x7f00a8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 -- 172.21.15.60:0/615520587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81a4e30 con 0x7f00a8107910 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00a7fff640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 0x7f00a8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00ad032640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009fd0 tx=0x7f00a000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00a5ffb640 1 -- 172.21.15.60:0/615520587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0017070 con 0x7f00a8107910 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00a5ffb640 1 -- 172.21.15.60:0/615520587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a0003e20 con 0x7f00a8107910 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a81a5120 con 0x7f00a8107910 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00a5ffb640 1 -- 172.21.15.60:0/615520587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0010460 con 0x7f00a8107910 2024-07-24T21:47:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.515+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81a5620 con 0x7f00a8107910 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.518+0000 7f00b17ea740 1 Processor -- start 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.518+0000 7f00a5ffb640 1 -- 172.21.15.60:0/615520587 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f00a0003970 con 0x7f00a8107910 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 -- start start 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919b31e0 0x5574919bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557491928280 con 0x5574919bf170 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557491917780 con 0x5574919b31e0 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00b17ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55749191b7c0 con 0x5574919b28c0 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49850/0 (socket says 172.21.15.60:49850) 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 -- 172.21.15.60:0/1352069490 learned_addr learned my addr 172.21.15.60:0/1352069490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00ac831640 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919b31e0 0x5574919bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 -- 172.21.15.60:0/1352069490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 msgr2=0x5574919b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 -- 172.21.15.60:0/1352069490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919b31e0 msgr2=0x5574919bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919b31e0 0x5574919bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.519+0000 7f00a7fff640 1 -- 172.21.15.60:0/1352069490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749167b790 con 0x5574919bf170 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f00a5ffb640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 0x7f00980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f00ac831640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 0x7f00980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0094012900 tx=0x7f009400bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23223d832d607834 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f00a5ffb640 1 -- 172.21.15.60:0/615520587 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f00a00e3d20 con 0x7f00a8107910 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f0092ffd640 1 -- 172.21.15.60:0/1352069490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094004220 con 0x5574919bf170 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f0092ffd640 1 -- 172.21.15.60:0/1352069490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00940043c0 con 0x5574919bf170 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.520+0000 7f0092ffd640 1 -- 172.21.15.60:0/1352069490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094003c00 con 0x5574919bf170 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- 172.21.15.60:0/1352069490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 msgr2=0x5574919c1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00ac831640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 0x7f00980b6200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f009c031440 tx=0x7f009c005890 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0094012900 tx=0x7f009400bc40 comp rx=0 tx=0).stop 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- 172.21.15.60:0/1352069490 shutdown_connections 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919bf170 0x5574919c1560 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919b31e0 0x5574919bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1352069490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- 172.21.15.60:0/1352069490 >> 172.21.15.60:0/1352069490 conn(0x5574919b06f0 msgr2=0x5574919b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- 172.21.15.60:0/1352069490 shutdown_connections 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- 172.21.15.60:0/1352069490 wait complete. 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 -- start start 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.521+0000 7f00b17ea740 1 Processor -- start 2024-07-24T21:47:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919be930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557491851ab0 con 0x5574919b31e0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55749189f9e0 con 0x5574919bf170 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5574919c1e50 con 0x5574919b28c0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00b17ea740 10 client.0 _ll_register_callbacks cb 0x557491627a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49854/0 (socket says 172.21.15.60:49854) 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 -- 172.21.15.60:0/713580909 learned_addr learned my addr 172.21.15.60:0/713580909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00ad032640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00ac831640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 -- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 msgr2=0x5574919be930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919be930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 -- 172.21.15.60:0/713580909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 msgr2=0x5574919bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 -- 172.21.15.60:0/713580909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574916f4620 con 0x5574919b31e0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f00a7fff640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0094003820 tx=0x7f00940042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7108c468293fc8f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f0087fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.522+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940044c0 con 0x5574919b31e0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0094003b90 con 0x5574919b31e0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574918c6ce0 con 0x5574919b31e0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00940035f0 con 0x5574919b31e0 2024-07-24T21:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574919cd660 con 0x5574919b31e0 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5574918c6ce0 con 0x5574919b31e0 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f009401c030 con 0x5574919b31e0 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 20 client.15600 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574919cea60 0x5574919d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 4 client.15600 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55749192a1d0 con 0x5574919cea60 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.523+0000 7f00ac831640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574919cea60 0x5574919d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.524+0000 7f00ac831640 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574919cea60 0x5574919d0e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62368a4ccc8acc99 server_cookie=28fa2fd3decbfc9d in_seq=0 out_seq=0 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.524+0000 7f0087fff640 10 client.15600 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f0087fff640 1 -- 172.21.15.60:0/713580909 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55749192a1d0 con 0x5574919cea60 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f0087fff640 10 client.15600 handle_command_reply: tid=0 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 client.15600 shutdown 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 msgr2=0x5574919bbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0094003820 tx=0x7f00940042d0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 shutdown_connections 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574919cea60 0x5574919d0e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f0087fff640 0 client.15600 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b31e0 0x5574919bbe10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 --2- 172.21.15.60:0/713580909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b28c0 0x5574919be930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 >> 172.21.15.60:0/713580909 conn(0x5574919b06f0 msgr2=0x5574919b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 shutdown_connections 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 1 -- 172.21.15.60:0/713580909 wait complete. 2024-07-24T21:47:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.525+0000 7f00b17ea740 20 client.15600 trim_cache size 0 max 16384 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.540+0000 7f00b17ea740 1 Processor -- start 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.540+0000 7f00b17ea740 1 -- start start 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.540+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00b17ea740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81b09a0 con 0x5574919b28c0 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00b17ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81ae270 con 0x5574919bf170 2024-07-24T21:47:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00b17ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81ac0c0 con 0x5574919b31e0 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49862/0 (socket says 172.21.15.60:49862) 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 -- 172.21.15.60:0/1638721978 learned_addr learned my addr 172.21.15.60:0/1638721978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ad032640 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 -- 172.21.15.60:0/1638721978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 msgr2=0x5574919bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 -- 172.21.15.60:0/1638721978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 msgr2=0x5574919d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 -- 172.21.15.60:0/1638721978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574919b1060 con 0x5574919b28c0 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00ac831640 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f009c031630 tx=0x7f009c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23be846552d29c28 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00927fc640 1 -- 172.21.15.60:0/1638721978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c035040 con 0x5574919b28c0 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00927fc640 1 -- 172.21.15.60:0/1638721978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f009c005040 con 0x5574919b28c0 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.541+0000 7f00927fc640 1 -- 172.21.15.60:0/1638721978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c004ad0 con 0x5574919b28c0 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 -- 172.21.15.60:0/1638721978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 msgr2=0x5574919caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f009c031630 tx=0x7f009c03e000 comp rx=0 tx=0).stop 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 -- 172.21.15.60:0/1638721978 shutdown_connections 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1638721978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919caf70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 -- 172.21.15.60:0/1638721978 >> 172.21.15.60:0/1638721978 conn(0x5574919b06f0 msgr2=0x5574919b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 -- 172.21.15.60:0/1638721978 shutdown_connections 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.542+0000 7f00b17ea740 1 -- 172.21.15.60:0/1638721978 wait complete. 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 -- start start 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 Processor -- start 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81ac730 con 0x5574919b28c0 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81adba0 con 0x5574919bf170 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81ace00 con 0x5574919b31e0 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00b17ea740 10 client.0 _ll_register_callbacks cb 0x7f00940127e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ac831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ac831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49876/0 (socket says 172.21.15.60:49876) 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ac831640 1 -- 172.21.15.60:0/1376838138 learned_addr learned my addr 172.21.15.60:0/1376838138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ad032640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ac831640 1 -- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 msgr2=0x5574919c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.543+0000 7f00ac831640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00ac831640 1 -- 172.21.15.60:0/1376838138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 msgr2=0x5574919c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00ac831640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00ac831640 1 -- 172.21.15.60:0/1376838138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574919c19e0 con 0x5574919b28c0 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00ad032640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00ac831640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f009c009310 tx=0x7f009c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cde2a75482008b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c0051f0 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f009c04f040 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f009c005040 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f009c069050 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00b17ea740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574919b35c0 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f009c005040 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f009c0036f0 con 0x5574919b28c0 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.544+0000 7f00b17ea740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00b17ea740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00b17ea740 20 client.15606 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557491a38bd0 0x557491a3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00b17ea740 4 client.15606 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:47:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557491925c70 con 0x557491a38bd0 2024-07-24T21:47:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557491a38bd0 0x557491a3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f00a7fff640 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557491a38bd0 0x557491a3aff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb06abca79da1ba9 server_cookie=446f1870a19feed0 in_seq=0 out_seq=0 2024-07-24T21:47:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.545+0000 7f0087fff640 10 client.15606 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:11.554 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:11 smithi086 ceph-mon[37128]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f0087fff640 1 -- 172.21.15.60:0/1376838138 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557491925c70 con 0x557491a38bd0 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f0087fff640 10 client.15606 handle_command_reply: tid=0 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 client.15606 shutdown 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 msgr2=0x5574919c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f009c009310 tx=0x7f009c031770 comp rx=0 tx=0).stop 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 shutdown_connections 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557491a38bd0 0x557491a3aff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574919bf170 0x5574919c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f0087fff640 0 client.15606 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574919b31e0 0x5574919c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.552+0000 7f00b17ea740 1 --2- 172.21.15.60:0/1376838138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574919b28c0 0x5574919c79d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 >> 172.21.15.60:0/1376838138 conn(0x5574919b06f0 msgr2=0x5574919aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 shutdown_connections 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00b17ea740 1 -- 172.21.15.60:0/1376838138 wait complete. 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00b17ea740 20 client.15606 trim_cache size 0 max 16384 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:47:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:47:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 msgr2=0x7f00980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 0x7f00980b6200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f009c031440 tx=0x7f009c005890 comp rx=0 tx=0).stop 2024-07-24T21:47:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 msgr2=0x7f00a819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009fd0 tx=0x7f00a000b260 comp rx=0 tx=0).stop 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 shutdown_connections 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f00980b3d40 0x7f00980b6200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a8107910 0x7f00a819a4f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a8106f90 0x7f00a8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 --2- 172.21.15.60:0/615520587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00a81015e0 0x7f00a8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.553+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 >> 172.21.15.60:0/615520587 conn(0x7f00a80782e0 msgr2=0x7f00a8105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.554+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 shutdown_connections 2024-07-24T21:47:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:11.554+0000 7f00aea95640 1 -- 172.21.15.60:0/615520587 wait complete. 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 -- 172.21.15.60:0/427409927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d010eca0 msgr2=0x7f44d0111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 --2- 172.21.15.60:0/427409927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d010eca0 0x7f44d0111110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009ca0 tx=0x7f44c802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 -- 172.21.15.60:0/427409927 shutdown_connections 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 --2- 172.21.15.60:0/427409927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d010eca0 0x7f44d0111110 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 --2- 172.21.15.60:0/427409927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d01019f0 0x7f44d010e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 --2- 172.21.15.60:0/427409927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 0x7f44d01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 -- 172.21.15.60:0/427409927 >> 172.21.15.60:0/427409927 conn(0x7f44d0076890 msgr2=0x7f44d0076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 -- 172.21.15.60:0/427409927 shutdown_connections 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.467+0000 7f44d73d6640 1 -- 172.21.15.60:0/427409927 wait complete. 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 Processor -- start 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 -- start start 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 0x7f44d019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d010eca0 0x7f44d01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f44d01148e0 con 0x7f44d01019f0 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44d0114760 con 0x7f44d010eca0 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d73d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44d0114a60 con 0x7f44d0100ff0 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49894/0 (socket says 172.21.15.60:49894) 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 -- 172.21.15.60:0/142933718 learned_addr learned my addr 172.21.15.60:0/142933718 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d5172640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 0x7f44d019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d5973640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d010eca0 0x7f44d01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 -- 172.21.15.60:0/142933718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 msgr2=0x7f44d019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 0x7f44d019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 -- 172.21.15.60:0/142933718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d010eca0 msgr2=0x7f44d01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d010eca0 0x7f44d01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.468+0000 7f44d4971640 1 -- 172.21.15.60:0/142933718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d01a0e80 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44d4971640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c890 tx=0x7f44c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0004220 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c00043c0 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01a1170 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0004a00 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.469+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01a7a00 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.470+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d01013d0 con 0x7f44d01019f0 2024-07-24T21:47:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.472+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f44c00079b0 con 0x7f44d01019f0 2024-07-24T21:47:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.473+0000 7f44c67fc640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 0x7f44bc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.473+0000 7f44d5172640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 0x7f44bc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.473+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f44c00e41d0 con 0x7f44d01019f0 2024-07-24T21:47:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.473+0000 7f44d5172640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 0x7f44bc0b6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0058c0 tx=0x7f44cc00a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.476+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f44c0014030 con 0x7f44d01019f0 2024-07-24T21:47:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.737+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44d01a7bd0 con 0x7f44d01019f0 2024-07-24T21:47:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.738+0000 7f44c67fc640 1 -- 172.21.15.60:0/142933718 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f44c00ab560 con 0x7f44d01019f0 2024-07-24T21:47:12.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:47:12.740 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:47:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 msgr2=0x7f44bc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 0x7f44bc0b6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0058c0 tx=0x7f44cc00a4c0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 msgr2=0x7f44d019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c890 tx=0x7f44c000cd50 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 shutdown_connections 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f44bc0b3dc0 0x7f44bc0b6280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44d010eca0 0x7f44d01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d01019f0 0x7f44d019b310 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 --2- 172.21.15.60:0/142933718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d0100ff0 0x7f44d019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.740+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 >> 172.21.15.60:0/142933718 conn(0x7f44d0076890 msgr2=0x7f44d010f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.741+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 shutdown_connections 2024-07-24T21:47:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:12.741+0000 7f44d73d6640 1 -- 172.21.15.60:0/142933718 wait complete. 2024-07-24T21:47:13.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:13 smithi123 ceph-mon[37143]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:13.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/142933718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:13.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:13 smithi086 ceph-mon[37128]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:13.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/142933718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:13.546 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:13 smithi060 ceph-mon[33000]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-24T21:47:13.546 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/142933718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:13.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:47:13.860 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:15.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:15 smithi060 ceph-mon[33000]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-07-24T21:47:15.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:15 smithi123 ceph-mon[37143]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-07-24T21:47:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:15 smithi086 ceph-mon[37128]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-07-24T21:47:15.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1323120748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 msgr2=0x7f3ed01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1323120748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01111b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009ca0 tx=0x7f3ec802a950 comp rx=0 tx=0).stop 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1323120748 shutdown_connections 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1323120748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01111b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1323120748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101b30 0x7f3ed010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1323120748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101210 0x7f3ed01015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1323120748 >> 172.21.15.60:0/1323120748 conn(0x7f3ed00fd0e0 msgr2=0x7f3ed00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1323120748 shutdown_connections 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.615+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1323120748 wait complete. 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 Processor -- start 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 -- start start 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101210 0x7f3ed019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 0x7f3ed019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0114940 con 0x7f3ed010edc0 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01147c0 con 0x7f3ed0101210 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed72c2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0114ac0 con 0x7f3ed0101b30 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed585f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed585f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49914/0 (socket says 172.21.15.60:49914) 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.616+0000 7f3ed585f640 1 -- 172.21.15.60:0/1731378873 learned_addr learned my addr 172.21.15.60:0/1731378873 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed505e640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101210 0x7f3ed019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed485d640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 0x7f3ed019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 -- 172.21.15.60:0/1731378873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 msgr2=0x7f3ed019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 0x7f3ed019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 -- 172.21.15.60:0/1731378873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101210 msgr2=0x7f3ed019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101210 0x7f3ed019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 -- 172.21.15.60:0/1731378873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed01a0d80 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed485d640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 0x7f3ed019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed585f640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec802f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ec27fc640 1 -- 172.21.15.60:0/1731378873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8034040 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed01a1010 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ec27fc640 1 -- 172.21.15.60:0/1731378873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec8003e00 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ec27fc640 1 -- 172.21.15.60:0/1731378873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8045480 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.617+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01a7730 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.620+0000 7f3ed9673740 1 Processor -- start 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.620+0000 7f3ec27fc640 1 -- 172.21.15.60:0/1731378873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3ec8037020 con 0x7f3ed010edc0 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.620+0000 7f3ed9673740 1 -- start start 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfcff0 0x556b35e08a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35e08f80 0x556b35e0b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556b35d739b0 con 0x556b35dfc6d0 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b35d62eb0 con 0x556b35dfcff0 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed9673740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b35d66ef0 con 0x556b35e08f80 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49916/0 (socket says 172.21.15.60:49916) 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 -- 172.21.15.60:0/328641033 learned_addr learned my addr 172.21.15.60:0/328641033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed505e640 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfcff0 0x556b35e08a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35e08f80 0x556b35e0b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ec27fc640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 0x7f3ebc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 -- 172.21.15.60:0/328641033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35e08f80 msgr2=0x556b35e0b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35e08f80 0x556b35e0b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 -- 172.21.15.60:0/328641033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfcff0 msgr2=0x556b35e08a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfcff0 0x556b35e08a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed485d640 1 -- 172.21.15.60:0/328641033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b35cec320 con 0x556b35dfc6d0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.621+0000 7f3ed585f640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 0x7f3ebc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ed485d640 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc008fc0 tx=0x7f3ecc008640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93803d32c2652c96 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ec27fc640 1 -- 172.21.15.60:0/1731378873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3ec8105070 con 0x7f3ed010edc0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ebb7fe640 1 -- 172.21.15.60:0/328641033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018030 con 0x556b35dfc6d0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ebb7fe640 1 -- 172.21.15.60:0/328641033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc010af0 con 0x556b35dfc6d0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ebb7fe640 1 -- 172.21.15.60:0/328641033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc01c9e0 con 0x556b35dfc6d0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.622+0000 7f3ed585f640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 0x7f3ebc0b6200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80cbdb0 tx=0x7f3ec80982d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- 172.21.15.60:0/328641033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 msgr2=0x556b35dfcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc008fc0 tx=0x7f3ecc008640 comp rx=0 tx=0).stop 2024-07-24T21:47:15.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- 172.21.15.60:0/328641033 shutdown_connections 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35e08f80 0x556b35e0b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfcff0 0x556b35e08a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 --2- 172.21.15.60:0/328641033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfc6d0 0x556b35dfcab0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- 172.21.15.60:0/328641033 >> 172.21.15.60:0/328641033 conn(0x556b35dfa500 msgr2=0x556b35dfb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- 172.21.15.60:0/328641033 shutdown_connections 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- 172.21.15.60:0/328641033 wait complete. 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 -- start start 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.623+0000 7f3ed9673740 1 Processor -- start 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35e08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35e05e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556b35bc6700 con 0x556b35e08f80 2024-07-24T21:47:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b35d055f0 con 0x556b35dfc6d0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b35e0bc00 con 0x556b35dfcff0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed9673740 10 client.0 _ll_register_callbacks cb 0x556b35d25960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49928/0 (socket says 172.21.15.60:49928) 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 -- 172.21.15.60:0/3969509576 learned_addr learned my addr 172.21.15.60:0/3969509576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35e08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed505e640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35e05e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 -- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 msgr2=0x556b35e05e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35e05e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 -- 172.21.15.60:0/3969509576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 msgr2=0x556b35e08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35e08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.624+0000 7f3ed485d640 1 -- 172.21.15.60:0/3969509576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b35cf8680 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed485d640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc01c4e0 tx=0x7f3ecc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73baebcc1ab34bec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018040 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3ecc0107f0 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ecc010af0 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ecc005050 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 10 client.15624 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b35ac3650 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc010af0 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3ecc01cce0 con 0x556b35e08f80 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 20 client.15624 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e189d0 0x556b35e1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 4 client.15624 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.625+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b35d75900 con 0x556b35e189d0 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.626+0000 7f3ed505e640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e189d0 0x556b35e1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.626+0000 7f3ed505e640 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e189d0 0x556b35e1adc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d1443d60692dfcf server_cookie=ebf78fa064126a13 in_seq=0 out_seq=0 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.626+0000 7f3eb8ff9640 10 client.15624 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/3969509576 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b35d75900 con 0x556b35e189d0 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3eb8ff9640 10 client.15624 handle_command_reply: tid=0 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 client.15624 shutdown 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 msgr2=0x556b35e06610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc01c4e0 tx=0x7f3ecc01c540 comp rx=0 tx=0).stop 2024-07-24T21:47:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 shutdown_connections 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e189d0 0x556b35e1adc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e06610 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35e05e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3eb8ff9640 0 client.15624 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 --2- 172.21.15.60:0/3969509576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35e08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.627+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 >> 172.21.15.60:0/3969509576 conn(0x556b35dfa500 msgr2=0x556b35e000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.628+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 shutdown_connections 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.628+0000 7f3ed9673740 1 -- 172.21.15.60:0/3969509576 wait complete. 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.628+0000 7f3ed9673740 20 client.15624 trim_cache size 0 max 16384 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 Processor -- start 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 -- start start 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35dfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35df56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b2a10 con 0x556b35e08f80 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b02e0 con 0x556b35dfc6d0 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed9673740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01ae130 con 0x556b35dfcff0 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed485d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49932/0 (socket says 172.21.15.60:49932) 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed485d640 1 -- 172.21.15.60:0/911678292 learned_addr learned my addr 172.21.15.60:0/911678292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed485d640 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35dfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.643+0000 7f3ed505e640 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35df56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 -- 172.21.15.60:0/911678292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 msgr2=0x556b35df56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35df56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 -- 172.21.15.60:0/911678292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 msgr2=0x556b35dfbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35dfbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 -- 172.21.15.60:0/911678292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b35dfad10 con 0x556b35e08f80 2024-07-24T21:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ed485d640 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0048d0 tx=0x7f3ecc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d042876e3eaf397 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ebaffd640 1 -- 172.21.15.60:0/911678292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018070 con 0x556b35e08f80 2024-07-24T21:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ebaffd640 1 -- 172.21.15.60:0/911678292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc008310 con 0x556b35e08f80 2024-07-24T21:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.644+0000 7f3ebaffd640 1 -- 172.21.15.60:0/911678292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc010930 con 0x556b35e08f80 2024-07-24T21:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- 172.21.15.60:0/911678292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 msgr2=0x556b35e20530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0048d0 tx=0x7f3ecc014040 comp rx=0 tx=0).stop 2024-07-24T21:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- 172.21.15.60:0/911678292 shutdown_connections 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35e08f80 0x556b35e20530 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfcff0 0x556b35df56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 --2- 172.21.15.60:0/911678292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35dfc6d0 0x556b35dfbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- 172.21.15.60:0/911678292 >> 172.21.15.60:0/911678292 conn(0x556b35dfa500 msgr2=0x556b35dfb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- 172.21.15.60:0/911678292 shutdown_connections 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- 172.21.15.60:0/911678292 wait complete. 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 -- start start 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.645+0000 7f3ed9673740 1 Processor -- start 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfc6d0 0x556b35e11310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 0x556b35e0a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01ae7a0 con 0x556b35dfcff0 2024-07-24T21:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01afc10 con 0x556b35e08f80 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01aee70 con 0x556b35dfc6d0 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed9673740 10 client.0 _ll_register_callbacks cb 0x7f3ec400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49934/0 (socket says 172.21.15.60:49934) 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 -- 172.21.15.60:0/2118284483 learned_addr learned my addr 172.21.15.60:0/2118284483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed485d640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfc6d0 0x556b35e11310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed485d640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 0x556b35e0a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 -- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfc6d0 msgr2=0x556b35e11310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfc6d0 0x556b35e11310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 -- 172.21.15.60:0/2118284483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 msgr2=0x556b35e0a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 0x556b35e0a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed505e640 1 -- 172.21.15.60:0/2118284483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b35e0a7e0 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.646+0000 7f3ed485d640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 0x556b35e0a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3ed505e640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x556b35e1b4d0 tx=0x7f3ec401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e347c6e5ed8aaa1b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3eb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4010cf0 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3ec4025410 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ecc008310 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4033050 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3ed9673740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b35e83500 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.647+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc008310 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3ec401e5e0 con 0x556b35dfcff0 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:47:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 20 client.15630 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e83ae0 0x556b35e85f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 4 client.15630 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b35d713a0 con 0x556b35e83ae0 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed485d640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e83ae0 0x556b35e85f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3ed485d640 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e83ae0 0x556b35e85f00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1dd79690f25e192d server_cookie=7b5292046698498b in_seq=0 out_seq=0 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.648+0000 7f3eb8ff9640 10 client.15630 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3eb8ff9640 1 -- 172.21.15.60:0/2118284483 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b35d713a0 con 0x556b35e83ae0 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3eb8ff9640 10 client.15630 handle_command_reply: tid=0 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 client.15630 shutdown 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 msgr2=0x556b35e09cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x556b35e1b4d0 tx=0x7f3ec401fae0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 shutdown_connections 2024-07-24T21:47:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x556b35e83ae0 0x556b35e85f00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b35e08f80 0x556b35e0a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b35dfcff0 0x556b35e09cb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 --2- 172.21.15.60:0/2118284483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b35dfc6d0 0x556b35e11310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3eb8ff9640 0 client.15630 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 >> 172.21.15.60:0/2118284483 conn(0x556b35dfa500 msgr2=0x556b35dff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 shutdown_connections 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 1 -- 172.21.15.60:0/2118284483 wait complete. 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.649+0000 7f3ed9673740 20 client.15630 trim_cache size 0 max 16384 2024-07-24T21:47:15.666 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:15.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 msgr2=0x7f3ebc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 0x7f3ebc0b6200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80cbdb0 tx=0x7f3ec80982d0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 msgr2=0x7f3ed01a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec802f640 comp rx=0 tx=0).stop 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 shutdown_connections 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3ebc0b3d40 0x7f3ebc0b6200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed010edc0 0x7f3ed01a05b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ed0101b30 0x7f3ed019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 --2- 172.21.15.60:0/1731378873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0101210 0x7f3ed019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 >> 172.21.15.60:0/1731378873 conn(0x7f3ed00fd0e0 msgr2=0x7f3ed010f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 shutdown_connections 2024-07-24T21:47:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:15.650+0000 7f3ed72c2640 1 -- 172.21.15.60:0/1731378873 wait complete. 2024-07-24T21:47:16.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:47:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:17.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:17 smithi123 ceph-mon[37143]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-07-24T21:47:17.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:17 smithi086 ceph-mon[37128]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-07-24T21:47:17.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:17 smithi060 ceph-mon[33000]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-07-24T21:47:18.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:47:18.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:47:18.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:47:18.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:47:18.534 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:47:18.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:47:18.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:47:18.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:47:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 -- 172.21.15.60:0/1024986899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101b40 msgr2=0x7ff63810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 --2- 172.21.15.60:0/1024986899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101b40 0x7ff63810e890 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009900 tx=0x7ff62c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 -- 172.21.15.60:0/1024986899 shutdown_connections 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 --2- 172.21.15.60:0/1024986899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63810edd0 0x7ff6381111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 --2- 172.21.15.60:0/1024986899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101b40 0x7ff63810e890 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 --2- 172.21.15.60:0/1024986899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff638101220 0x7ff638101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 -- 172.21.15.60:0/1024986899 >> 172.21.15.60:0/1024986899 conn(0x7ff6380fd0b0 msgr2=0x7ff6380ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 -- 172.21.15.60:0/1024986899 shutdown_connections 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.530+0000 7ff63d668640 1 -- 172.21.15.60:0/1024986899 wait complete. 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 Processor -- start 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 -- start start 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 0x7ff63810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 0x7ff63810b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 0x7ff63810bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff638114950 con 0x7ff638101220 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381147d0 con 0x7ff63810edd0 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.531+0000 7ff63d668640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff638114ad0 con 0x7ff638101b40 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 0x7ff63810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 0x7ff63810bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 0x7ff63810e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48730/0 (socket says 172.21.15.60:48730) 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff636ffd640 1 -- 172.21.15.60:0/2214332403 learned_addr learned my addr 172.21.15.60:0/2214332403 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 0x7ff63810b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 -- 172.21.15.60:0/2214332403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 msgr2=0x7ff63810b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 0x7ff63810b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 -- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 msgr2=0x7ff63810e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 0x7ff63810e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 -- 172.21.15.60:0/2214332403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6381ab670 con 0x7ff63810edd0 2024-07-24T21:47:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 0x7ff63810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.532+0000 7ff6377fe640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 0x7ff63810bf30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff630005fd0 tx=0x7ff63000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.533+0000 7ff61ffff640 1 -- 172.21.15.60:0/2214332403 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63000cdf0 con 0x7ff63810edd0 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.533+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6381ab840 con 0x7ff63810edd0 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.533+0000 7ff61ffff640 1 -- 172.21.15.60:0/2214332403 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff630005020 con 0x7ff63810edd0 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.533+0000 7ff61ffff640 1 -- 172.21.15.60:0/2214332403 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630010690 con 0x7ff63810edd0 2024-07-24T21:47:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.533+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381abd50 con 0x7ff63810edd0 2024-07-24T21:47:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.535+0000 7ff63fa19740 1 Processor -- start 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff61ffff640 1 -- 172.21.15.60:0/2214332403 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff630005460 con 0x7ff63810edd0 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 -- start start 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013b82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 0x55e013b91370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e013af99b0 con 0x55e013b8ef80 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e013ae8eb0 con 0x55e013b82ff0 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.536+0000 7ff63fa19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e013aecef0 con 0x55e013b826d0 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51258/0 (socket says 172.21.15.60:51258) 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 -- 172.21.15.60:0/1171350545 learned_addr learned my addr 172.21.15.60:0/1171350545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff61ffff640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 0x7ff6200b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 -- 172.21.15.60:0/1171350545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 msgr2=0x55e013b82ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013b82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 0x55e013b91370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 -- 172.21.15.60:0/1171350545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 msgr2=0x55e013b91370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 0x55e013b91370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff636ffd640 1 -- 172.21.15.60:0/1171350545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e013a72320 con 0x55e013b82ff0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 0x55e013b91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.537+0000 7ff61ffff640 1 -- 172.21.15.60:0/2214332403 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff6300e45c0 con 0x7ff63810edd0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.538+0000 7ff6377fe640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 0x7ff6200b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.538+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff62800bda0 tx=0x7ff6280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3e07ef7bff40ff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.538+0000 7ff61d7fa640 1 -- 172.21.15.60:0/1171350545 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62801c680 con 0x55e013b82ff0 2024-07-24T21:47:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.538+0000 7ff61d7fa640 1 -- 172.21.15.60:0/1171350545 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6280118f0 con 0x55e013b82ff0 2024-07-24T21:47:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.538+0000 7ff61d7fa640 1 -- 172.21.15.60:0/1171350545 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62801a800 con 0x55e013b82ff0 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 -- 172.21.15.60:0/1171350545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 msgr2=0x55e013b8ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff62800bda0 tx=0x7ff6280107a0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff6377fe640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 0x7ff6200b6200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff6300e1dc0 tx=0x7ff630005750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 -- 172.21.15.60:0/1171350545 shutdown_connections 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b8ef80 0x55e013b91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8ea40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1171350545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013b82ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 -- 172.21.15.60:0/1171350545 >> 172.21.15.60:0/1171350545 conn(0x55e013b80500 msgr2=0x55e013b816a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 -- 172.21.15.60:0/1171350545 shutdown_connections 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.539+0000 7ff63fa19740 1 -- 172.21.15.60:0/1171350545 wait complete. 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 -- start start 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 Processor -- start 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b8c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e01394c700 con 0x55e013b826d0 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e013a8b5f0 con 0x55e013b82ff0 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e013b91c00 con 0x55e013b8ef80 2024-07-24T21:47:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48750/0 (socket says 172.21.15.60:48750) 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff6367fc640 1 -- 172.21.15.60:0/2582236226 learned_addr learned my addr 172.21.15.60:0/2582236226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.540+0000 7ff63fa19740 10 client.0 _ll_register_callbacks cb 0x55e013aab960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b8c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 -- 172.21.15.60:0/2582236226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 msgr2=0x55e013b8c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b8c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 -- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 msgr2=0x55e013b8c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 -- 172.21.15.60:0/2582236226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e013a7e680 con 0x55e013b826d0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009900 tx=0x7ff62c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b68f052a12c84 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff61aff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c035070 con 0x55e013b826d0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff62c004450 con 0x55e013b826d0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6280118f0 con 0x55e013b826d0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff62c005020 con 0x55e013b826d0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff63fa19740 10 client.15639 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.541+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e013849650 con 0x55e013b826d0 2024-07-24T21:47:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6280118f0 con 0x55e013b826d0 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff62c0035f0 con 0x55e013b826d0 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 10 client.15639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 20 client.15639 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013b9ea20 0x55e013ba0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 4 client.15639 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e013afb900 con 0x55e013b9ea20 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.542+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013b9ea20 0x55e013ba0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013b9ea20 0x55e013ba0e40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61857962fdb46be5 server_cookie=4ece9ccf81f8d9ea in_seq=0 out_seq=0 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff61aff5640 10 client.15639 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff61aff5640 1 -- 172.21.15.60:0/2582236226 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e013afb900 con 0x55e013b9ea20 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff61aff5640 10 client.15639 handle_command_reply: tid=0 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff63fa19740 1 client.15639 shutdown 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 msgr2=0x55e013b8bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.543+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009900 tx=0x7ff62c032ed0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 shutdown_connections 2024-07-24T21:47:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013b9ea20 0x55e013ba0e40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b82ff0 0x55e013b8c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2582236226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b826d0 0x55e013b8bf10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff61aff5640 0 client.15639 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 >> 172.21.15.60:0/2582236226 conn(0x55e013b80500 msgr2=0x55e013b860a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 shutdown_connections 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 1 -- 172.21.15.60:0/2582236226 wait complete. 2024-07-24T21:47:18.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.544+0000 7ff63fa19740 20 client.15639 trim_cache size 0 max 16384 2024-07-24T21:47:18.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:47:18.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:47:18.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:47:18.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:47:18.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 Processor -- start 2024-07-24T21:47:18.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 -- start start 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013ba66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b8ef80 0x55e013b89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b7030 con 0x55e013b82ff0 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b4900 con 0x55e013b8ef80 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff63fa19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b2750 con 0x55e013b826d0 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48752/0 (socket says 172.21.15.60:48752) 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff636ffd640 1 -- 172.21.15.60:0/2188728208 learned_addr learned my addr 172.21.15.60:0/2188728208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.558+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b8ef80 0x55e013b89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff6367fc640 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013ba66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 -- 172.21.15.60:0/2188728208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 msgr2=0x55e013ba66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013ba66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 -- 172.21.15.60:0/2188728208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b8ef80 msgr2=0x55e013b89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b8ef80 0x55e013b89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 -- 172.21.15.60:0/2188728208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e013ba14a0 con 0x55e013b82ff0 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff636ffd640 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff628010ca0 tx=0x7ff62800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6162b1c8a782a998 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff61cff9640 1 -- 172.21.15.60:0/2188728208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280052d0 con 0x55e013b82ff0 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff61cff9640 1 -- 172.21.15.60:0/2188728208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff628011ad0 con 0x55e013b82ff0 2024-07-24T21:47:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.559+0000 7ff61cff9640 1 -- 172.21.15.60:0/2188728208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628003b50 con 0x55e013b82ff0 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- 172.21.15.60:0/2188728208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 msgr2=0x55e013ba6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff628010ca0 tx=0x7ff62800b6e0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- 172.21.15.60:0/2188728208 shutdown_connections 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b8ef80 0x55e013b89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b82ff0 0x55e013ba6c10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 --2- 172.21.15.60:0/2188728208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b826d0 0x55e013ba66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- 172.21.15.60:0/2188728208 >> 172.21.15.60:0/2188728208 conn(0x55e013b80500 msgr2=0x55e013b8dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- 172.21.15.60:0/2188728208 shutdown_connections 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- 172.21.15.60:0/2188728208 wait complete. 2024-07-24T21:47:18.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 -- start start 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.560+0000 7ff63fa19740 1 Processor -- start 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b826d0 0x55e013b97200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b2dc0 con 0x55e013b94010 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b4230 con 0x55e013b826d0 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b3490 con 0x55e013b8ef80 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff63fa19740 10 client.0 _ll_register_callbacks cb 0x7ff62c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b826d0 0x55e013b97200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48756/0 (socket says 172.21.15.60:48756) 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 -- 172.21.15.60:0/1103721664 learned_addr learned my addr 172.21.15.60:0/1103721664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:18.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 -- 172.21.15.60:0/1103721664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 msgr2=0x55e013b97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 -- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b826d0 msgr2=0x55e013b97200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b826d0 0x55e013b97200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 -- 172.21.15.60:0/1103721664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e013c096b0 con 0x55e013b94010 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff62c003c00 tx=0x7ff62c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56b6def4a52d18c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.561+0000 7ff61aff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c035040 con 0x55e013b94010 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff62c0053d0 con 0x55e013b94010 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628011ad0 con 0x55e013b94010 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff62c0055d0 con 0x55e013b94010 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 10 client.15645 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:18.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e013c0a7c0 con 0x55e013b94010 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff628011ad0 con 0x55e013b94010 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff62c032030 con 0x55e013b94010 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 10 client.15645 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 20 client.15645 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013c0ade0 0x55e013c0d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 4 client.15645 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.562+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e013af73a0 con 0x55e013c0ade0 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013c0ade0 0x55e013c0d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff636ffd640 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013c0ade0 0x55e013c0d200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddcfdd00fc30d94d server_cookie=4b1b4443a9d199a6 in_seq=0 out_seq=0 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff61aff5640 10 client.15645 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff61aff5640 1 -- 172.21.15.60:0/1103721664 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e013af73a0 con 0x55e013c0ade0 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff61aff5640 10 client.15645 handle_command_reply: tid=0 2024-07-24T21:47:18.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 client.15645 shutdown 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 msgr2=0x55e013b94490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff62c003c00 tx=0x7ff62c032ed0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 shutdown_connections 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55e013c0ade0 0x55e013c0d200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e013b94010 0x55e013b94490 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e013b8ef80 0x55e013b97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff61aff5640 0 client.15645 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 --2- 172.21.15.60:0/1103721664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e013b826d0 0x55e013b97200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.563+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 >> 172.21.15.60:0/1103721664 conn(0x55e013b80500 msgr2=0x55e013b8bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 shutdown_connections 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63fa19740 1 -- 172.21.15.60:0/1103721664 wait complete. 2024-07-24T21:47:18.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63fa19740 20 client.15645 trim_cache size 0 max 16384 2024-07-24T21:47:18.571 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:18.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 msgr2=0x7ff6200b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 0x7ff6200b6200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff6300e1dc0 tx=0x7ff630005750 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 msgr2=0x7ff63810bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 0x7ff63810bf30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff630005fd0 tx=0x7ff63000ef10 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 shutdown_connections 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6200b3d40 0x7ff6200b6200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63810edd0 0x7ff63810bf30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff638101b40 0x7ff63810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 --2- 172.21.15.60:0/2214332403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff638101220 0x7ff63810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.564+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 >> 172.21.15.60:0/2214332403 conn(0x7ff6380fd0b0 msgr2=0x7ff6380ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.565+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 shutdown_connections 2024-07-24T21:47:18.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:18.565+0000 7ff63d668640 1 -- 172.21.15.60:0/2214332403 wait complete. 2024-07-24T21:47:19.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:47:19.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:19 smithi123 ceph-mon[37143]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:47:19.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:19 smithi086 ceph-mon[37128]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:47:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:19 smithi060 ceph-mon[33000]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-07-24T21:47:19.624 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-24T21:47:19.624 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-07-24T21:47:19.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:47:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:20.839 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:21.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:21 smithi123 ceph-mon[37143]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:21 smithi086 ceph-mon[37128]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:21.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:21 smithi060 ceph-mon[33000]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 -- 172.21.15.60:0/2509977847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103f80 msgr2=0x7facc010c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 --2- 172.21.15.60:0/2509977847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103f80 0x7facc010c470 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb002a410 comp rx=0 tx=0).stop 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 -- 172.21.15.60:0/2509977847 shutdown_connections 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 --2- 172.21.15.60:0/2509977847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 0x7facc010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 --2- 172.21.15.60:0/2509977847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103f80 0x7facc010c470 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 --2- 172.21.15.60:0/2509977847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103660 0x7facc0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 -- 172.21.15.60:0/2509977847 >> 172.21.15.60:0/2509977847 conn(0x7facc00fd0c0 msgr2=0x7facc00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 -- 172.21.15.60:0/2509977847 shutdown_connections 2024-07-24T21:47:22.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.538+0000 7facc807e640 1 -- 172.21.15.60:0/2509977847 wait complete. 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 Processor -- start 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 -- start start 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 0x7facc010bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 0x7facc01a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7facc0112760 con 0x7facc0103660 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc01125e0 con 0x7facc010cac0 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc807e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facc01128e0 con 0x7facc0103f80 2024-07-24T21:47:22.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.539+0000 7facc5e1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48786/0 (socket says 172.21.15.60:48786) 2024-07-24T21:47:22.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 -- 172.21.15.60:0/1016129722 learned_addr learned my addr 172.21.15.60:0/1016129722 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:22.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5619640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 0x7facc010bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc661b640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 0x7facc01a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 -- 172.21.15.60:0/1016129722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 msgr2=0x7facc010bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 0x7facc010bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 -- 172.21.15.60:0/1016129722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 msgr2=0x7facc01a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 0x7facc01a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 -- 172.21.15.60:0/1016129722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01a20b0 con 0x7facc0103660 2024-07-24T21:47:22.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5619640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 0x7facc010bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.540+0000 7facc5e1a640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7facbc00e9f0 tx=0x7facbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.541+0000 7facb6ffd640 1 -- 172.21.15.60:0/1016129722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc00cd80 con 0x7facc0103660 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.541+0000 7facb6ffd640 1 -- 172.21.15.60:0/1016129722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facbc0054d0 con 0x7facc0103660 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.541+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01a23a0 con 0x7facc0103660 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.541+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc006c560 con 0x7facc0103660 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.542+0000 7facb6ffd640 1 -- 172.21.15.60:0/1016129722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc00cd80 con 0x7facc0103660 2024-07-24T21:47:22.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.544+0000 7facca42f740 1 Processor -- start 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 -- start start 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb4773ff0 0x559bb477fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4782370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559bb46ea9b0 con 0x559bb47736d0 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559bb46d9eb0 con 0x559bb4773ff0 2024-07-24T21:47:22.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facca42f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559bb46ddef0 con 0x559bb477ff80 2024-07-24T21:47:22.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facc661b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facc661b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48794/0 (socket says 172.21.15.60:48794) 2024-07-24T21:47:22.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.545+0000 7facc661b640 1 -- 172.21.15.60:0/979589284 learned_addr learned my addr 172.21.15.60:0/979589284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:22.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 -- 172.21.15.60:0/979589284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 msgr2=0x559bb4782370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4782370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 -- 172.21.15.60:0/979589284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb4773ff0 msgr2=0x559bb477fa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb4773ff0 0x559bb477fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 -- 172.21.15.60:0/979589284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb4663320 con 0x559bb47736d0 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facc661b640 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fb0 tx=0x7facb80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a94463e8ee4310 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facb6ffd640 1 -- 172.21.15.60:0/1016129722 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7facbc004e90 con 0x7facc0103660 2024-07-24T21:47:22.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facabfff640 1 -- 172.21.15.60:0/979589284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8004d20 con 0x559bb47736d0 2024-07-24T21:47:22.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facabfff640 1 -- 172.21.15.60:0/979589284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb8004050 con 0x559bb47736d0 2024-07-24T21:47:22.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.546+0000 7facabfff640 1 -- 172.21.15.60:0/979589284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8005280 con 0x559bb47736d0 2024-07-24T21:47:22.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.547+0000 7facca42f740 1 -- 172.21.15.60:0/979589284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 msgr2=0x559bb4773ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.547+0000 7facca42f740 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fb0 tx=0x7facb80118d0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.547+0000 7facca42f740 1 -- 172.21.15.60:0/979589284 shutdown_connections 2024-07-24T21:47:22.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facca42f740 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4782370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facca42f740 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb4773ff0 0x559bb477fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facca42f740 1 --2- 172.21.15.60:0/979589284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb47736d0 0x559bb4773ab0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facca42f740 1 -- 172.21.15.60:0/979589284 >> 172.21.15.60:0/979589284 conn(0x559bb4771500 msgr2=0x559bb47726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facb6ffd640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 0x7facac0b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facc5619640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 0x7facac0b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.548+0000 7facb6ffd640 1 -- 172.21.15.60:0/1016129722 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7facbc106110 con 0x7facc0103660 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.549+0000 7facca42f740 1 -- 172.21.15.60:0/979589284 shutdown_connections 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.549+0000 7facca42f740 1 -- 172.21.15.60:0/979589284 wait complete. 2024-07-24T21:47:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.549+0000 7facc5619640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 0x7facac0b63b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facb0006010 tx=0x7facb0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.549+0000 7facca42f740 1 -- start start 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.550+0000 7facca42f740 1 Processor -- start 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.550+0000 7facca42f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 0x559bb477ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.550+0000 7facca42f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.550+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.550+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48798/0 (socket says 172.21.15.60:48798) 2024-07-24T21:47:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5619640 1 -- 172.21.15.60:0/3089458962 learned_addr learned my addr 172.21.15.60:0/3089458962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 0x559bb477d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559bb453d700 con 0x559bb4773ff0 2024-07-24T21:47:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559bb467c5f0 con 0x559bb477ff80 2024-07-24T21:47:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559bb4782ca0 con 0x559bb47736d0 2024-07-24T21:47:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 0x559bb477ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc661b640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 0x559bb477d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 -- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 msgr2=0x559bb477d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 0x559bb477d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 -- 172.21.15.60:0/3089458962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 msgr2=0x559bb477d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facc5e1a640 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb466f680 con 0x559bb47736d0 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.551+0000 7facca42f740 10 client.0 _ll_register_callbacks cb 0x559bb469c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facc661b640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 0x559bb477d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb8004050 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facc5619640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 0x559bb477ced0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facbc103860 tx=0x7facbc0cc550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9c15e4ee54f068 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7faca97fa640 10 client.25225 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7faca97fa640 1 -- 172.21.15.60:0/3089458962 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0cd420 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7faca97fa640 1 -- 172.21.15.60:0/3089458962 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7facbc0d5940 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7faca97fa640 1 -- 172.21.15.60:0/3089458962 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7facbc0d5b40 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facca42f740 10 client.25225 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.552+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559bb443a650 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7faca97fa640 1 -- 172.21.15.60:0/3089458962 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7facbc0cbf80 con 0x559bb47736d0 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 10 client.25225 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 10 client.25225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 20 client.25225 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb478fe20 0x559bb4792250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 4 client.25225 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bb46ec900 con 0x559bb478fe20 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.553+0000 7facc5619640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb478fe20 0x559bb4792250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.554+0000 7faca97fa640 10 client.25225 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.554+0000 7facc5619640 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb478fe20 0x559bb4792250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d05c618f80abfb5 server_cookie=27445d44d57daee0 in_seq=0 out_seq=0 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.554+0000 7faca97fa640 1 -- 172.21.15.60:0/3089458962 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bb46ec900 con 0x559bb478fe20 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.554+0000 7faca97fa640 10 client.25225 handle_command_reply: tid=0 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.554+0000 7facca42f740 1 client.25225 shutdown 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 msgr2=0x559bb477ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 0x559bb477ced0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facbc103860 tx=0x7facbc0cc550 comp rx=0 tx=0).stop 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 shutdown_connections 2024-07-24T21:47:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb478fe20 0x559bb4792250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb477ff80 0x559bb477d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb477d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7faca97fa640 0 client.25225 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 --2- 172.21.15.60:0/3089458962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb47736d0 0x559bb477ced0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 >> 172.21.15.60:0/3089458962 conn(0x559bb4771500 msgr2=0x559bb47770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 shutdown_connections 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 1 -- 172.21.15.60:0/3089458962 wait complete. 2024-07-24T21:47:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.555+0000 7facca42f740 20 client.25225 trim_cache size 0 max 16384 2024-07-24T21:47:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 Processor -- start 2024-07-24T21:47:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 -- start start 2024-07-24T21:47:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb4797900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb477ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7facc01ae820 con 0x559bb4773ff0 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc01ac0f0 con 0x559bb47736d0 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.569+0000 7facca42f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facc0103a40 con 0x559bb477ff80 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48814/0 (socket says 172.21.15.60:48814) 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 -- 172.21.15.60:0/3987256484 learned_addr learned my addr 172.21.15.60:0/3987256484 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb4797900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc661b640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb477ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 -- 172.21.15.60:0/3987256484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 msgr2=0x559bb477ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb477ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 -- 172.21.15.60:0/3987256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 msgr2=0x559bb4797900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb4797900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 -- 172.21.15.60:0/3987256484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb477b3d0 con 0x559bb4773ff0 2024-07-24T21:47:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5e1a640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb4797900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.570+0000 7facc5619640 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7facb002aa00 tx=0x7facb003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddac94d8641143f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facab7fe640 1 -- 172.21.15.60:0/3987256484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0035050 con 0x559bb4773ff0 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facab7fe640 1 -- 172.21.15.60:0/3987256484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb0004b70 con 0x559bb4773ff0 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facab7fe640 1 -- 172.21.15.60:0/3987256484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0003b40 con 0x559bb4773ff0 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facca42f740 1 -- 172.21.15.60:0/3987256484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 msgr2=0x559bb4797e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facca42f740 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7facb002aa00 tx=0x7facb003c000 comp rx=0 tx=0).stop 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facca42f740 1 -- 172.21.15.60:0/3987256484 shutdown_connections 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.571+0000 7facca42f740 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb477ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4797e40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 --2- 172.21.15.60:0/3987256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb4797900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 -- 172.21.15.60:0/3987256484 >> 172.21.15.60:0/3987256484 conn(0x559bb4771500 msgr2=0x559bb47781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 -- 172.21.15.60:0/3987256484 shutdown_connections 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 -- 172.21.15.60:0/3987256484 wait complete. 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 -- start start 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.572+0000 7facca42f740 1 Processor -- start 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb47883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4783dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7facc0073610 con 0x559bb4773ff0 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc0074a80 con 0x559bb47736d0 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facc0073ce0 con 0x559bb477ff80 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facca42f740 10 client.0 _ll_register_callbacks cb 0x7facbc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:22.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48816/0 (socket says 172.21.15.60:48816) 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 -- 172.21.15.60:0/429048953 learned_addr learned my addr 172.21.15.60:0/429048953 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc661b640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4783dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 -- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 msgr2=0x559bb4783dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4783dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 -- 172.21.15.60:0/429048953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 msgr2=0x559bb47883c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb47883c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.573+0000 7facc5619640 1 -- 172.21.15.60:0/429048953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb47843c0 con 0x559bb4773ff0 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7facc5619640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7facb0005370 tx=0x7facb00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec179953d24c1297 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7faca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb005c070 con 0x559bb4773ff0 2024-07-24T21:47:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7facb00048c0 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb0004b70 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7facb0067050 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7facca42f740 10 client.15663 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559bb47fa6f0 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.574+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb0004b70 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7facb0032850 con 0x559bb4773ff0 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 10 client.15663 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 20 client.15663 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb47fad00 0x559bb47fd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 4 client.15663 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bb46e83a0 con 0x559bb47fad00 2024-07-24T21:47:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.575+0000 7facc5e1a640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb47fad00 0x559bb47fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7faca97fa640 10 client.15663 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facc5e1a640 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb47fad00 0x559bb47fd120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a08eb01f6d6177d7 server_cookie=11f7e0a10999a59f in_seq=0 out_seq=0 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7faca97fa640 1 -- 172.21.15.60:0/429048953 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bb46e83a0 con 0x559bb47fad00 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7faca97fa640 10 client.15663 handle_command_reply: tid=0 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 client.15663 shutdown 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 msgr2=0x559bb4783890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7facb0005370 tx=0x7facb00056a0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 shutdown_connections 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559bb47fad00 0x559bb47fd120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559bb477ff80 0x559bb4783dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7faca97fa640 0 client.15663 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559bb4773ff0 0x559bb4783890 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 --2- 172.21.15.60:0/429048953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bb47736d0 0x559bb47883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.576+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 >> 172.21.15.60:0/429048953 conn(0x559bb4771500 msgr2=0x559bb477ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 shutdown_connections 2024-07-24T21:47:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facca42f740 1 -- 172.21.15.60:0/429048953 wait complete. 2024-07-24T21:47:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facca42f740 20 client.15663 trim_cache size 0 max 16384 2024-07-24T21:47:22.582 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 msgr2=0x7facac0b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 0x7facac0b63b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facb0006010 tx=0x7facb0031400 comp rx=0 tx=0).stop 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 msgr2=0x7facc010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.577+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7facbc00e9f0 tx=0x7facbc00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 shutdown_connections 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facac0b3ef0 0x7facac0b63b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc010cac0 0x7facc01a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103f80 0x7facc010bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 --2- 172.21.15.60:0/1016129722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc0103660 0x7facc010b840 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 >> 172.21.15.60:0/1016129722 conn(0x7facc00fd0c0 msgr2=0x7facc010ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 shutdown_connections 2024-07-24T21:47:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:22.578+0000 7facc807e640 1 -- 172.21.15.60:0/1016129722 wait complete. 2024-07-24T21:47:23.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:23 smithi123 ceph-mon[37143]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:23.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:23 smithi060 ceph-mon[33000]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:23 smithi086 ceph-mon[37128]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.771+0000 7fd388f8e640 1 -- 172.21.15.60:0/3697883490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 msgr2=0x7fd384110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.771+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3697883490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd384110ba0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd374009900 tx=0x7fd37402a480 comp rx=0 tx=0).stop 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 -- 172.21.15.60:0/3697883490 shutdown_connections 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3697883490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38411ba90 0x7fd38411de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3697883490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd384109f20 0x7fd38410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3697883490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd384110ba0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 -- 172.21.15.60:0/3697883490 >> 172.21.15.60:0/3697883490 conn(0x7fd38406d980 msgr2=0x7fd38406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:23.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 -- 172.21.15.60:0/3697883490 shutdown_connections 2024-07-24T21:47:23.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.772+0000 7fd388f8e640 1 -- 172.21.15.60:0/3697883490 wait complete. 2024-07-24T21:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.773+0000 7fd388f8e640 1 Processor -- start 2024-07-24T21:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.773+0000 7fd388f8e640 1 -- start start 2024-07-24T21:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.773+0000 7fd388f8e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.773+0000 7fd388f8e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd384109f20 0x7fd3841a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd388f8e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 0x7fd3841ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd388f8e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd384121500 con 0x7fd384109b40 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd388f8e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd384121380 con 0x7fd38411ba90 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd388f8e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd384121680 con 0x7fd384109f20 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48824/0 (socket says 172.21.15.60:48824) 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 -- 172.21.15.60:0/3938227455 learned_addr learned my addr 172.21.15.60:0/3938227455 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd382d9d640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 0x7fd3841ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd381d9b640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd384109f20 0x7fd3841a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 -- 172.21.15.60:0/3938227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd384109f20 msgr2=0x7fd3841a7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd384109f20 0x7fd3841a7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 -- 172.21.15.60:0/3938227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 msgr2=0x7fd3841ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 0x7fd3841ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:23.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 -- 172.21.15.60:0/3938227455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3841ada60 con 0x7fd384109b40 2024-07-24T21:47:23.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd382d9d640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 0x7fd3841ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:23.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.774+0000 7fd38259c640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd374005940 tx=0x7fd3740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:23.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.775+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37403f030 con 0x7fd384109b40 2024-07-24T21:47:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.775+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd374003cb0 con 0x7fd384109b40 2024-07-24T21:47:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.775+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374035070 con 0x7fd384109b40 2024-07-24T21:47:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.775+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3841adcf0 con 0x7fd384109b40 2024-07-24T21:47:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.775+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3841ae0c0 con 0x7fd384109b40 2024-07-24T21:47:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.777+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd384074300 con 0x7fd384109b40 2024-07-24T21:47:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.779+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd374042030 con 0x7fd384109b40 2024-07-24T21:47:23.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.780+0000 7fd3737fe640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 0x7fd36c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:23.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.780+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd3741047a0 con 0x7fd384109b40 2024-07-24T21:47:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.781+0000 7fd381d9b640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 0x7fd36c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.781+0000 7fd381d9b640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 0x7fd36c0b6280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd38411e210 tx=0x7fd37800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:23.784+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd3841ae0c0 con 0x7fd384109b40 2024-07-24T21:47:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.018+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3841a8ae0 con 0x7fd384109b40 2024-07-24T21:47:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.019+0000 7fd3737fe640 1 -- 172.21.15.60:0/3938227455 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fd3841a8ae0 con 0x7fd384109b40 2024-07-24T21:47:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:47:24.021 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:47:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 msgr2=0x7fd36c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 0x7fd36c0b6280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd38411e210 tx=0x7fd37800b390 comp rx=0 tx=0).stop 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 msgr2=0x7fd3841a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd374005940 tx=0x7fd3740042f0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 shutdown_connections 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd36c0b3dc0 0x7fd36c0b6280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38411ba90 0x7fd3841ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd384109f20 0x7fd3841a7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 --2- 172.21.15.60:0/3938227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd384109b40 0x7fd3841a79b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 >> 172.21.15.60:0/3938227455 conn(0x7fd38406d980 msgr2=0x7fd38411c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 shutdown_connections 2024-07-24T21:47:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:24.023+0000 7fd388f8e640 1 -- 172.21.15.60:0/3938227455 wait complete. 2024-07-24T21:47:24.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:24 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3938227455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:24.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:24 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3938227455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:24.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:47:24.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:47:24.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:24 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3938227455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:24.694 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-07-24T21:47:24.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.695+0000 7f02a9dfa640 1 -- 172.21.15.60:0/4208432696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a41034d0 msgr2=0x7f02a4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/4208432696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a41034d0 0x7f02a4103910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f028c009900 tx=0x7f028c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 -- 172.21.15.60:0/4208432696 shutdown_connections 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/4208432696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a4103e50 0x7f02a410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/4208432696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a41034d0 0x7f02a4103910 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/4208432696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 -- 172.21.15.60:0/4208432696 >> 172.21.15.60:0/4208432696 conn(0x7f02a4076860 msgr2=0x7f02a4076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 -- 172.21.15.60:0/4208432696 shutdown_connections 2024-07-24T21:47:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.696+0000 7f02a9dfa640 1 -- 172.21.15.60:0/4208432696 wait complete. 2024-07-24T21:47:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.697+0000 7f02a9dfa640 1 Processor -- start 2024-07-24T21:47:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.697+0000 7f02a9dfa640 1 -- start start 2024-07-24T21:47:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a41969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4110460 con 0x7f02a4103e50 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41102e0 con 0x7f02a4101180 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a9dfa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41105e0 con 0x7f02a41034d0 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42708/0 (socket says 172.21.15.60:42708) 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48836/0 (socket says 172.21.15.60:48836) 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a2ffd640 1 -- 172.21.15.60:0/3476170043 learned_addr learned my addr 172.21.15.60:0/3476170043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.698+0000 7f02a37fe640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a41969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 -- 172.21.15.60:0/3476170043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 msgr2=0x7f02a4196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 -- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 msgr2=0x7f02a419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 -- 172.21.15.60:0/3476170043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a419ca60 con 0x7f02a4101180 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a3fff640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a37fe640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a41969c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9f0 tx=0x7f029400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a0ff9640 1 -- 172.21.15.60:0/3476170043 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400cd80 con 0x7f02a4101180 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a419cd50 con 0x7f02a4101180 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a0ff9640 1 -- 172.21.15.60:0/3476170043 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02940035f0 con 0x7f02a4101180 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41a35f0 con 0x7f02a4101180 2024-07-24T21:47:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.699+0000 7f02a0ff9640 1 -- 172.21.15.60:0/3476170043 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294010690 con 0x7f02a4101180 2024-07-24T21:47:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 Processor -- start 2024-07-24T21:47:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 -- start start 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 0x564454a6c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 0x564454a6c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5644549e1280 con 0x564454a80bf0 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644549d0780 con 0x564454a6bc90 2024-07-24T21:47:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02acc2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5644549d47c0 con 0x564454a75ff0 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02a0ff9640 1 -- 172.21.15.60:0/3476170043 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f02940037a0 con 0x7f02a4101180 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48846/0 (socket says 172.21.15.60:48846) 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.703+0000 7f02a2ffd640 1 -- 172.21.15.60:0/99258121 learned_addr learned my addr 172.21.15.60:0/99258121 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a3fff640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 0x564454a6c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a37fe640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 0x564454a6c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 -- 172.21.15.60:0/99258121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 msgr2=0x564454a6c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 0x564454a6c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 -- 172.21.15.60:0/99258121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 msgr2=0x564454a6c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 0x564454a6c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 -- 172.21.15.60:0/99258121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564454734790 con 0x564454a80bf0 2024-07-24T21:47:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a3fff640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 0x564454a6c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a37fe640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 0x564454a6c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.704+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f028c000c00 tx=0x7f028c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a87d7956e9797a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.705+0000 7f0299ffb640 1 -- 172.21.15.60:0/99258121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c035040 con 0x564454a80bf0 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.705+0000 7f02a0ff9640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 0x7f02900b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.705+0000 7f0299ffb640 1 -- 172.21.15.60:0/99258121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c004030 con 0x564454a80bf0 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.705+0000 7f02a3fff640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 0x7f02900b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.705+0000 7f02a0ff9640 1 -- 172.21.15.60:0/3476170043 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f02940e4270 con 0x7f02a4101180 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 -- 172.21.15.60:0/99258121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 msgr2=0x564454a82fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02a3fff640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 0x7f02900b6280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f029c00a380 tx=0x7f029c00a610 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f028c000c00 tx=0x7f028c0058d0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 -- 172.21.15.60:0/99258121 shutdown_connections 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a82fe0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a75ff0 0x564454a6c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 --2- 172.21.15.60:0/99258121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a6bc90 0x564454a6c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 -- 172.21.15.60:0/99258121 >> 172.21.15.60:0/99258121 conn(0x564454a69ac0 msgr2=0x564454a6ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 -- 172.21.15.60:0/99258121 shutdown_connections 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.706+0000 7f02acc2f740 1 -- 172.21.15.60:0/99258121 wait complete. 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.707+0000 7f02acc2f740 1 -- start start 2024-07-24T21:47:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.707+0000 7f02acc2f740 1 Processor -- start 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56445490aab0 con 0x564454a80bf0 2024-07-24T21:47:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644549589e0 con 0x564454a75ff0 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564454a83a20 con 0x564454a6bc90 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42720/0 (socket says 172.21.15.60:42720) 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 -- 172.21.15.60:0/1976123227 learned_addr learned my addr 172.21.15.60:0/1976123227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02acc2f740 10 client.0 _ll_register_callbacks cb 0x5644546e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 -- 172.21.15.60:0/1976123227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 msgr2=0x564454a72ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a72ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 -- 172.21.15.60:0/1976123227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 msgr2=0x564454a73000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.708+0000 7f02a37fe640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a73000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.709+0000 7f02a37fe640 1 -- 172.21.15.60:0/1976123227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644547ad620 con 0x564454a6bc90 2024-07-24T21:47:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.709+0000 7f02a3fff640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.709+0000 7f02a37fe640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f029400cfd0 tx=0x7f0294003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a1a5d0ec973ead server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.709+0000 7f0282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.709+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940b3f30 con 0x564454a6bc90 2024-07-24T21:47:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.710+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56445497fce0 con 0x564454a6bc90 2024-07-24T21:47:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.710+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f02940a7110 con 0x564454a6bc90 2024-07-24T21:47:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.713+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02940aa0e0 con 0x564454a6bc90 2024-07-24T21:47:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.713+0000 7f02acc2f740 10 client.25240 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.713+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564454a8f070 con 0x564454a6bc90 2024-07-24T21:47:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.713+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02940aa280 con 0x564454a6bc90 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.713+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f02940b0610 con 0x564454a6bc90 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 10 client.25240 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 10 client.25240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 20 client.25240 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454a90940 0x564454a92d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 4 client.25240 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445497fce0 con 0x564454a90940 2024-07-24T21:47:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454a90940 0x564454a92d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454a90940 0x564454a92d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3d95302d2863ec server_cookie=3fefae7da193af4e in_seq=0 out_seq=0 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.714+0000 7f0282ffd640 10 client.25240 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f0282ffd640 1 -- 172.21.15.60:0/1976123227 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56445497fce0 con 0x564454a90940 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f0282ffd640 10 client.25240 handle_command_reply: tid=0 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f02acc2f740 1 client.25240 shutdown 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 msgr2=0x564454a75590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f029400cfd0 tx=0x7f0294003cf0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 shutdown_connections 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.715+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454a90940 0x564454a92d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a72ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 --2- 172.21.15.60:0/1976123227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a75590 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 >> 172.21.15.60:0/1976123227 conn(0x564454a69ac0 msgr2=0x564454a6ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f0282ffd640 0 client.25240 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 shutdown_connections 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 1 -- 172.21.15.60:0/1976123227 wait complete. 2024-07-24T21:47:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.716+0000 7f02acc2f740 20 client.25240 trim_cache size 0 max 16384 2024-07-24T21:47:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.744+0000 7f02acc2f740 1 Processor -- start 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.744+0000 7f02acc2f740 1 -- start start 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a6c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ae720 con 0x564454a80bf0 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41abff0 con 0x564454a75ff0 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02acc2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41a9e40 con 0x564454a6bc90 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51400/0 (socket says 172.21.15.60:51400) 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02a2ffd640 1 -- 172.21.15.60:0/741282433 learned_addr learned my addr 172.21.15.60:0/741282433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.745+0000 7f02a3fff640 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a6c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 -- 172.21.15.60:0/741282433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 msgr2=0x564454a69490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a69490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 -- 172.21.15.60:0/741282433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 msgr2=0x564454a6c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a6c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 -- 172.21.15.60:0/741282433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564454a933e0 con 0x564454a75ff0 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f028c005ec0 tx=0x7f028c005130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b7e42ceaeecb37 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.746+0000 7f02997fa640 1 -- 172.21.15.60:0/741282433 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c048040 con 0x564454a75ff0 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.747+0000 7f02997fa640 1 -- 172.21.15.60:0/741282433 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c031b60 con 0x564454a75ff0 2024-07-24T21:47:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.747+0000 7f02997fa640 1 -- 172.21.15.60:0/741282433 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c035040 con 0x564454a75ff0 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- 172.21.15.60:0/741282433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 msgr2=0x564454a6c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f028c005ec0 tx=0x7f028c005130 comp rx=0 tx=0).stop 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- 172.21.15.60:0/741282433 shutdown_connections 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a6c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a6c070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 --2- 172.21.15.60:0/741282433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a69490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- 172.21.15.60:0/741282433 >> 172.21.15.60:0/741282433 conn(0x564454a69ac0 msgr2=0x564454a72a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- 172.21.15.60:0/741282433 shutdown_connections 2024-07-24T21:47:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- 172.21.15.60:0/741282433 wait complete. 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 -- start start 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.748+0000 7f02acc2f740 1 Processor -- start 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a88d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a97810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41aa4b0 con 0x564454a80bf0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41ab920 con 0x564454a75ff0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41aab80 con 0x564454a6bc90 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02acc2f740 10 client.0 _ll_register_callbacks cb 0x7f0294003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51410/0 (socket says 172.21.15.60:51410) 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 -- 172.21.15.60:0/2667627312 learned_addr learned my addr 172.21.15.60:0/2667627312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a3fff640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a97810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a37fe640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a88d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 -- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 msgr2=0x564454a88d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a88d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 -- 172.21.15.60:0/2667627312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 msgr2=0x564454a97810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a97810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a2ffd640 1 -- 172.21.15.60:0/2667627312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564454a6caf0 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.749+0000 7f02a3fff640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f02a2ffd640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f028c009a20 tx=0x7f028c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8123b62a9a8cb7cb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f0282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c05b060 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f028c031b60 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f028c031d10 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f028c035040 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f02acc2f740 10 client.25250 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.750+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564454afacb0 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c031b60 con 0x564454a75ff0 2024-07-24T21:47:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f028c004ce0 con 0x564454a75ff0 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 10 client.25250 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 20 client.25250 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454afaf30 0x564454afd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 4 client.25250 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644549e31d0 con 0x564454afaf30 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.751+0000 7f02a37fe640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454afaf30 0x564454afd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.752+0000 7f02a37fe640 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454afaf30 0x564454afd320 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43bd83b78f65de3 server_cookie=4880bf86573a3fb3 in_seq=0 out_seq=0 2024-07-24T21:47:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.752+0000 7f0282ffd640 10 client.25250 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.755+0000 7f0282ffd640 1 -- 172.21.15.60:0/2667627312 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644549e31d0 con 0x564454afaf30 2024-07-24T21:47:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.755+0000 7f0282ffd640 10 client.25250 handle_command_reply: tid=0 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.755+0000 7f02acc2f740 1 client.25250 shutdown 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 msgr2=0x564454a892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f028c009a20 tx=0x7f028c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 shutdown_connections 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564454afaf30 0x564454afd320 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f0282ffd640 0 client.25250 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564454a80bf0 0x564454a97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564454a75ff0 0x564454a892a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 --2- 172.21.15.60:0/2667627312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564454a6bc90 0x564454a88d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 >> 172.21.15.60:0/2667627312 conn(0x564454a69ac0 msgr2=0x564454a72a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 shutdown_connections 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 1 -- 172.21.15.60:0/2667627312 wait complete. 2024-07-24T21:47:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.756+0000 7f02acc2f740 20 client.25250 trim_cache size 0 max 16384 2024-07-24T21:47:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:47:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:47:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:47:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 msgr2=0x7f02900b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 0x7f02900b6280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f029c00a380 tx=0x7f029c00a610 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 msgr2=0x7f02a41969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a41969c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9f0 tx=0x7f029400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 shutdown_connections 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02900b3dc0 0x7f02900b6280 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02a4103e50 0x7f02a419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02a41034d0 0x7f02a4196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 --2- 172.21.15.60:0/3476170043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a4101180 0x7f02a41969c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 >> 172.21.15.60:0/3476170043 conn(0x7f02a4076860 msgr2=0x7f02a4104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 shutdown_connections 2024-07-24T21:47:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:24.757+0000 7f02a9dfa640 1 -- 172.21.15.60:0/3476170043 wait complete. 2024-07-24T21:47:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:25.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:25 smithi060 ceph-mon[33000]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:25.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:25 smithi123 ceph-mon[37143]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:25.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:25 smithi086 ceph-mon[37128]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:26.059 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 -- 172.21.15.60:0/1462766262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 msgr2=0x7feec810c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1462766262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec810c370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009860 tx=0x7feeb802a340 comp rx=0 tx=0).stop 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 -- 172.21.15.60:0/1462766262 shutdown_connections 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1462766262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1462766262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec810c370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1462766262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 0x7feec81015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 -- 172.21.15.60:0/1462766262 >> 172.21.15.60:0/1462766262 conn(0x7feec8076890 msgr2=0x7feec8076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 -- 172.21.15.60:0/1462766262 shutdown_connections 2024-07-24T21:47:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.777+0000 7feecfd5a640 1 -- 172.21.15.60:0/1462766262 wait complete. 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 Processor -- start 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 -- start start 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 0x7feec8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feec8112720 con 0x7feec81011c0 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec81125a0 con 0x7feec8101bc0 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.778+0000 7feecfd5a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec81128a0 con 0x7feec810ca80 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecdaf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 0x7feec8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51632/0 (socket says 172.21.15.60:51632) 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 -- 172.21.15.60:0/1831155208 learned_addr learned my addr 172.21.15.60:0/1831155208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feece2f7640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 -- 172.21.15.60:0/1831155208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 msgr2=0x7feec819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 -- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 msgr2=0x7feec8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 0x7feec8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 -- 172.21.15.60:0/1831155208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec819ed40 con 0x7feec8101bc0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feece2f7640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feecd2f5640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb8032d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feebeffd640 1 -- 172.21.15.60:0/1831155208 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8040020 con 0x7feec8101bc0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feebeffd640 1 -- 172.21.15.60:0/1831155208 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb8030d10 con 0x7feec8101bc0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.779+0000 7feebeffd640 1 -- 172.21.15.60:0/1831155208 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8037040 con 0x7feec8101bc0 2024-07-24T21:47:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.780+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec819efd0 con 0x7feec8101bc0 2024-07-24T21:47:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.780+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81a56f0 con 0x7feec8101bc0 2024-07-24T21:47:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.783+0000 7feed210b740 1 Processor -- start 2024-07-24T21:47:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feebeffd640 1 -- 172.21.15.60:0/1831155208 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7feeb8043020 con 0x7feec8101bc0 2024-07-24T21:47:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 -- start start 2024-07-24T21:47:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feece2f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feece2f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51634/0 (socket says 172.21.15.60:51634) 2024-07-24T21:47:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feece2f7640 1 -- 172.21.15.60:0/1585006788 learned_addr learned my addr 172.21.15.60:0/1585006788 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ff139179f0 con 0x55ff139a0840 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff13906ef0 con 0x55ff1399ff20 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.784+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff1390af30 con 0x55ff139ac7d0 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 -- 172.21.15.60:0/1585006788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 msgr2=0x55ff139aebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 -- 172.21.15.60:0/1585006788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 msgr2=0x55ff139ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 -- 172.21.15.60:0/1585006788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff13667930 con 0x55ff1399ff20 2024-07-24T21:47:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.785+0000 7feece2f7640 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feec0009fd0 tx=0x7feec00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8879b8203c69f0c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.786+0000 7feebeffd640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 0x7feeb40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.786+0000 7feeb3fff640 1 -- 172.21.15.60:0/1585006788 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0011760 con 0x55ff1399ff20 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.786+0000 7feeb3fff640 1 -- 172.21.15.60:0/1585006788 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec0011db0 con 0x55ff1399ff20 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.786+0000 7feeb3fff640 1 -- 172.21.15.60:0/1585006788 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec000e3c0 con 0x55ff1399ff20 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.786+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 0x7feeb40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feebeffd640 1 -- 172.21.15.60:0/1831155208 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7feeb8103ee0 con 0x7feec8101bc0 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 msgr2=0x55ff139a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 0x7feeb40b6200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55ff1399ec30 tx=0x7feec4015040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feec0009fd0 tx=0x7feec00108e0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 shutdown_connections 2024-07-24T21:47:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 --2- 172.21.15.60:0/1585006788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a0300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 >> 172.21.15.60:0/1585006788 conn(0x55ff1399dd50 msgr2=0x55ff1399eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 shutdown_connections 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.787+0000 7feed210b740 1 -- 172.21.15.60:0/1585006788 wait complete. 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 -- start start 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 Processor -- start 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feecdaf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46876/0 (socket says 172.21.15.60:46876) 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feecdaf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51642/0 (socket says 172.21.15.60:51642) 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feecdaf6640 1 -- 172.21.15.60:0/3041687643 learned_addr learned my addr 172.21.15.60:0/3041687643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ff1376a890 con 0x55ff139a0840 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff138a9390 con 0x55ff1399ff20 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.788+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ff139af450 con 0x55ff139ac7d0 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 -- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 msgr2=0x55ff139aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 -- 172.21.15.60:0/3041687643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 msgr2=0x55ff139a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff138a3d60 con 0x55ff1399ff20 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feed210b740 10 client.0 _ll_register_callbacks cb 0x55ff138b3500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec0011db0 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec4013770 tx=0x7feec4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529925873e7a8afe server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feeb17fa640 10 client.25262 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.789+0000 7feeb17fa640 1 -- 172.21.15.60:0/3041687643 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec402c060 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feeb17fa640 1 -- 172.21.15.60:0/3041687643 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7feec401fad0 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feeb17fa640 1 -- 172.21.15.60:0/3041687643 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feec401fcd0 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 10 client.25262 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ff139b2370 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feeb17fa640 1 -- 172.21.15.60:0/3041687643 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7feec4027440 con 0x55ff1399ff20 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 10 client.25262 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 10 client.25262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 20 client.25262 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff139b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 4 client.25262 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff13919940 con 0x55ff139b3710 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.790+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff139b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.791+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff139b5b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db216d3d72ace0c server_cookie=56d253e539bc107f in_seq=0 out_seq=0 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.791+0000 7feeb17fa640 10 client.25262 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feeb17fa640 1 -- 172.21.15.60:0/3041687643 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff13919940 con 0x55ff139b3710 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feeb17fa640 10 client.25262 handle_command_reply: tid=0 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 client.25262 shutdown 2024-07-24T21:47:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 msgr2=0x55ff139a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec4013770 tx=0x7feec4004730 comp rx=0 tx=0).stop 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 shutdown_connections 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff139b5b80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feeb17fa640 0 client.25262 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 --2- 172.21.15.60:0/3041687643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1399ff20 0x55ff139a9800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 >> 172.21.15.60:0/3041687643 conn(0x55ff1399dd50 msgr2=0x55ff139a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.792+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 shutdown_connections 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.793+0000 7feed210b740 1 -- 172.21.15.60:0/3041687643 wait complete. 2024-07-24T21:47:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.793+0000 7feed210b740 20 client.25262 trim_cache size 0 max 16384 2024-07-24T21:47:26.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.829+0000 7feed210b740 1 Processor -- start 2024-07-24T21:47:26.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 -- start start 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139a0840 0x55ff139a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feec81ae700 con 0x55ff1399ff20 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec81abfd0 con 0x55ff139a0840 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feed210b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec81a9e20 con 0x55ff139ac7d0 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feecdaf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feecdaf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46880/0 (socket says 172.21.15.60:46880) 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feecdaf6640 1 -- 172.21.15.60:0/1938043344 learned_addr learned my addr 172.21.15.60:0/1938043344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feece2f7640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.830+0000 7feecd2f5640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139a0840 0x55ff139a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 -- 172.21.15.60:0/1938043344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 msgr2=0x55ff139a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 -- 172.21.15.60:0/1938043344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139a0840 msgr2=0x55ff139a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139a0840 0x55ff139a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 -- 172.21.15.60:0/1938043344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff13998290 con 0x55ff1399ff20 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feece2f7640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feecdaf6640 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feec4013f90 tx=0x7feec4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93574590bfd5b198 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feeb37fe640 1 -- 172.21.15.60:0/1938043344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4027440 con 0x55ff1399ff20 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feeb37fe640 1 -- 172.21.15.60:0/1938043344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec0011db0 con 0x55ff1399ff20 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.831+0000 7feeb37fe640 1 -- 172.21.15.60:0/1938043344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4005640 con 0x55ff1399ff20 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 -- 172.21.15.60:0/1938043344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 msgr2=0x55ff139ab900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feec4013f90 tx=0x7feec4022000 comp rx=0 tx=0).stop 2024-07-24T21:47:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 -- 172.21.15.60:0/1938043344 shutdown_connections 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff139ac7d0 0x55ff139a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139a0840 0x55ff139a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 --2- 172.21.15.60:0/1938043344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff1399ff20 0x55ff139ab900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.832+0000 7feed210b740 1 -- 172.21.15.60:0/1938043344 >> 172.21.15.60:0/1938043344 conn(0x55ff1399dd50 msgr2=0x55ff139a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.833+0000 7feed210b740 1 -- 172.21.15.60:0/1938043344 shutdown_connections 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.833+0000 7feed210b740 1 -- 172.21.15.60:0/1938043344 wait complete. 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.833+0000 7feed210b740 1 -- start start 2024-07-24T21:47:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.833+0000 7feed210b740 1 Processor -- start 2024-07-24T21:47:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 0x55ff139b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139ac7d0 0x55ff13a1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feec81aa490 con 0x55ff139a0840 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec81ab900 con 0x55ff139ac7d0 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec81aab60 con 0x55ff1399ff20 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feed210b740 10 client.0 _ll_register_callbacks cb 0x7feec4013d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46894/0 (socket says 172.21.15.60:46894) 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 -- 172.21.15.60:0/3897696566 learned_addr learned my addr 172.21.15.60:0/3897696566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 0x55ff139b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feece2f7640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139ac7d0 0x55ff13a1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 -- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 msgr2=0x55ff139b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 0x55ff139b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 -- 172.21.15.60:0/3897696566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139ac7d0 msgr2=0x55ff13a1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139ac7d0 0x55ff13a1f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecd2f5640 1 -- 172.21.15.60:0/3897696566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff13a1f730 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.834+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 0x55ff139b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feecd2f5640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feeb8031000 tx=0x7feeb802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeceb8ecd0904f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feeb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80328f0 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7feeb80d3820 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec0011db0 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feeb80ca710 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feed210b740 10 client.15705 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ff13a27d70 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.835+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80cb300 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7feeb80b2070 con 0x55ff139a0840 2024-07-24T21:47:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 10 client.15705 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 20 client.15705 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff13a2a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 4 client.15705 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7feec0011db0 con 0x55ff139b3710 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff13a2a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feecdaf6640 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff13a2a450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2ccc297b6cf222 server_cookie=34bf7e19bce27aec in_seq=0 out_seq=0 2024-07-24T21:47:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.836+0000 7feeb17fa640 10 client.15705 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.837+0000 7feeb17fa640 1 -- 172.21.15.60:0/3897696566 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7feec0011db0 con 0x55ff139b3710 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.837+0000 7feeb17fa640 10 client.15705 handle_command_reply: tid=0 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.837+0000 7feed210b740 1 client.15705 shutdown 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.837+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 msgr2=0x55ff139ba3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.837+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feeb8031000 tx=0x7feeb802a340 comp rx=0 tx=0).stop 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 shutdown_connections 2024-07-24T21:47:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ff139b3710 0x55ff13a2a450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff139ac7d0 0x55ff13a1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff139a0840 0x55ff139ba3e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feeb17fa640 0 client.15705 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 --2- 172.21.15.60:0/3897696566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ff1399ff20 0x55ff139b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 >> 172.21.15.60:0/3897696566 conn(0x55ff1399dd50 msgr2=0x55ff139a97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 shutdown_connections 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 1 -- 172.21.15.60:0/3897696566 wait complete. 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.838+0000 7feed210b740 20 client.15705 trim_cache size 0 max 16384 2024-07-24T21:47:26.842 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "4af92ae3-bdb9-4582-9a9a-88ffdd14bef0", 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 msgr2=0x7feeb40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 0x7feeb40b6200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55ff1399ec30 tx=0x7feec4015040 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 msgr2=0x7feec81991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb8032d40 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 shutdown_connections 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7feeb40b3d40 0x7feeb40b6200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec810ca80 0x7feec819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec8101bc0 0x7feec81991d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 --2- 172.21.15.60:0/1831155208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec81011c0 0x7feec8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 >> 172.21.15.60:0/1831155208 conn(0x7feec8076890 msgr2=0x7feec80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 shutdown_connections 2024-07-24T21:47:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:26.839+0000 7feecfd5a640 1 -- 172.21.15.60:0/1831155208 wait complete. 2024-07-24T21:47:27.220 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:27 smithi060 ceph-mon[33000]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:47:27.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:27 smithi123 ceph-mon[37143]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:47:27.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:27 smithi086 ceph-mon[37128]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-24T21:47:28.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:28 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:47:28.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:28 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:28.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:28 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:47:28.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:28 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:28.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:28 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:47:28.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:28 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:28.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:28 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:47:28.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:28 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:28.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:47:28.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:28 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:47:28.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:28 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:28.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:28 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:47:28.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:28 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 -- 172.21.15.60:0/2912635498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5354101650 msgr2=0x7f5354101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 --2- 172.21.15.60:0/2912635498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5354101650 0x7f5354101a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5350009900 tx=0x7f535002a410 comp rx=0 tx=0).stop 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 -- 172.21.15.60:0/2912635498 shutdown_connections 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 --2- 172.21.15.60:0/2912635498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f535410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 --2- 172.21.15.60:0/2912635498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f53541074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 --2- 172.21.15.60:0/2912635498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5354101650 0x7f5354101a30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 -- 172.21.15.60:0/2912635498 >> 172.21.15.60:0/2912635498 conn(0x7f53540fd0c0 msgr2=0x7f53540ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.113+0000 7f535b535640 1 -- 172.21.15.60:0/2912635498 shutdown_connections 2024-07-24T21:47:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.114+0000 7f535b535640 1 -- 172.21.15.60:0/2912635498 wait complete. 2024-07-24T21:47:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.115+0000 7f535b535640 1 Processor -- start 2024-07-24T21:47:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.115+0000 7f535b535640 1 -- start start 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.115+0000 7f535b535640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f5354197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f535b535640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f535b535640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 0x7f535419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f53592d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f5354197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f5358ad0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f5359ad2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 0x7f535419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f5358ad0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46380/0 (socket says 172.21.15.60:46380) 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f5359ad2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 0x7f535419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46910/0 (socket says 172.21.15.60:46910) 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f53592d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f5354197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51678/0 (socket says 172.21.15.60:51678) 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f5358ad0640 1 -- 172.21.15.60:0/696872515 learned_addr learned my addr 172.21.15.60:0/696872515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5354111640 con 0x7f535419d630 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53541114c0 con 0x7f5354107060 2024-07-24T21:47:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.116+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53541117c0 con 0x7f53541079e0 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 -- 172.21.15.60:0/696872515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 msgr2=0x7f5354197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f5354197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 -- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 msgr2=0x7f535419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 0x7f535419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535419e1c0 con 0x7f53541079e0 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f5358ad0640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f534400fb60 tx=0x7f5344009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f534a7fc640 1 -- 172.21.15.60:0/696872515 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400fc50 con 0x7f53541079e0 2024-07-24T21:47:29.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535406a470 con 0x7f53541079e0 2024-07-24T21:47:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.117+0000 7f534a7fc640 1 -- 172.21.15.60:0/696872515 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534400fdf0 con 0x7f53541079e0 2024-07-24T21:47:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.118+0000 7f534a7fc640 1 -- 172.21.15.60:0/696872515 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344004a00 con 0x7f53541079e0 2024-07-24T21:47:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.119+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535406aa00 con 0x7f53541079e0 2024-07-24T21:47:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.122+0000 7f534a7fc640 1 -- 172.21.15.60:0/696872515 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5344004020 con 0x7f53541079e0 2024-07-24T21:47:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.123+0000 7f535d8e6740 1 Processor -- start 2024-07-24T21:47:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 -- start start 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d844e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5591d83c35e0 con 0x5591d845a590 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591d83b2ae0 con 0x5591d844dce0 2024-07-24T21:47:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f535d8e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5591d83b6b20 con 0x5591d844e600 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.124+0000 7f534a7fc640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 0x7f53400b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f53592d1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f534a7fc640 1 -- 172.21.15.60:0/696872515 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f53440e5e00 con 0x7f53541079e0 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f53592d1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46390/0 (socket says 172.21.15.60:46390) 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f53592d1640 1 -- 172.21.15.60:0/127778672 learned_addr learned my addr 172.21.15.60:0/127778672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f5359ad2640 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d844e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f5359ad2640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 0x7f53400b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.125+0000 7f5358ad0640 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 -- 172.21.15.60:0/127778672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 msgr2=0x5591d844e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d844e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 -- 172.21.15.60:0/127778672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 msgr2=0x5591d845c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 -- 172.21.15.60:0/127778672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d83482f0 con 0x5591d844e600 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f53592d1640 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f5350004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddb39d82ffba415 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.126+0000 7f5359ad2640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 0x7f53400b6200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f534c005fd0 tx=0x7f534c009b00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.127+0000 7f533f7fe640 1 -- 172.21.15.60:0/127778672 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535003f070 con 0x5591d844e600 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.127+0000 7f533f7fe640 1 -- 172.21.15.60:0/127778672 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500054f0 con 0x5591d844e600 2024-07-24T21:47:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.127+0000 7f533f7fe640 1 -- 172.21.15.60:0/127778672 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350035070 con 0x5591d844e600 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 -- 172.21.15.60:0/127778672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 msgr2=0x5591d845a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f5350004160 comp rx=0 tx=0).stop 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 -- 172.21.15.60:0/127778672 shutdown_connections 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845a050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 --2- 172.21.15.60:0/127778672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d844e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.128+0000 7f535d8e6740 1 -- 172.21.15.60:0/127778672 >> 172.21.15.60:0/127778672 conn(0x5591d844bb10 msgr2=0x5591d844ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.129+0000 7f535d8e6740 1 -- 172.21.15.60:0/127778672 shutdown_connections 2024-07-24T21:47:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.129+0000 7f535d8e6740 1 -- 172.21.15.60:0/127778672 wait complete. 2024-07-24T21:47:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.129+0000 7f535d8e6740 1 -- start start 2024-07-24T21:47:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.129+0000 7f535d8e6740 1 Processor -- start 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8457a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f53592d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f53592d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46932/0 (socket says 172.21.15.60:46932) 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f53592d1640 1 -- 172.21.15.60:0/1337655488 learned_addr learned my addr 172.21.15.60:0/1337655488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.130+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8457f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8457a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5591d82167b0 con 0x5591d844dce0 2024-07-24T21:47:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591d8355100 con 0x5591d844e600 2024-07-24T21:47:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f5359ad2640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8457f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5591d845d250 con 0x5591d845a590 2024-07-24T21:47:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f53592d1640 1 -- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 msgr2=0x5591d8457f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f53592d1640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8457f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f53592d1640 1 -- 172.21.15.60:0/1337655488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 msgr2=0x5591d8457a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.131+0000 7f53592d1640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8457a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f53592d1640 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d8358e50 con 0x5591d844dce0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f535d8e6740 10 client.0 _ll_register_callbacks cb 0x5591d83ab630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f5359ad2640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8457f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53500054f0 con 0x5591d844dce0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8457a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f53592d1640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f535002a450 tx=0x7f53500048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fcd36aa946ec23 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f533cff9640 10 client.15717 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.132+0000 7f533cff9640 1 -- 172.21.15.60:0/1337655488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535004d060 con 0x5591d844dce0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.133+0000 7f533cff9640 1 -- 172.21.15.60:0/1337655488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5350005040 con 0x5591d844dce0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.133+0000 7f533cff9640 1 -- 172.21.15.60:0/1337655488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f535003f070 con 0x5591d844dce0 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.133+0000 7f535d8e6740 10 client.15717 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.133+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5591d8122ba0 con 0x5591d844dce0 2024-07-24T21:47:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.133+0000 7f533cff9640 1 -- 172.21.15.60:0/1337655488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f53500035f0 con 0x5591d844dce0 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 10 client.15717 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 20 client.15717 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d8469ca0 0x5591d846c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 4 client.15717 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591d83c5530 con 0x5591d8469ca0 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d8469ca0 0x5591d846c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d8469ca0 0x5591d846c090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f14fca279879e8 server_cookie=cc345871d06748e8 in_seq=0 out_seq=0 2024-07-24T21:47:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.134+0000 7f533cff9640 10 client.15717 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.135+0000 7f533cff9640 1 -- 172.21.15.60:0/1337655488 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591d83c5530 con 0x5591d8469ca0 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.135+0000 7f533cff9640 10 client.15717 handle_command_reply: tid=0 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.135+0000 7f535d8e6740 1 client.15717 shutdown 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.135+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 msgr2=0x5591d84574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.135+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f535002a450 tx=0x7f53500048d0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 shutdown_connections 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d8469ca0 0x5591d846c090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8457f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8457a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1337655488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d84574f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 >> 172.21.15.60:0/1337655488 conn(0x5591d844bb10 msgr2=0x5591d84516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f533cff9640 0 client.15717 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 shutdown_connections 2024-07-24T21:47:29.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 1 -- 172.21.15.60:0/1337655488 wait complete. 2024-07-24T21:47:29.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.136+0000 7f535d8e6740 20 client.15717 trim_cache size 0 max 16384 2024-07-24T21:47:29.147 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:29.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.168+0000 7f535d8e6740 1 Processor -- start 2024-07-24T21:47:29.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 -- start start 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f53592d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f53592d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46944/0 (socket says 172.21.15.60:46944) 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f53592d1640 1 -- 172.21.15.60:0/3188978339 learned_addr learned my addr 172.21.15.60:0/3188978339 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8451e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8456a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53541aed50 con 0x5591d844dce0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53541ac620 con 0x5591d844e600 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.169+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53541aa470 con 0x5591d845a590 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f5359ad2640 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8456a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 -- 172.21.15.60:0/3188978339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 msgr2=0x5591d8456a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8456a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 -- 172.21.15.60:0/3188978339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 msgr2=0x5591d8451e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8451e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 -- 172.21.15.60:0/3188978339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d845cdf0 con 0x5591d844dce0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.170+0000 7f53592d1640 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5350005c40 tx=0x7f5350048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea63107af215771d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.171+0000 7f533effd640 1 -- 172.21.15.60:0/3188978339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350035040 con 0x5591d844dce0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.171+0000 7f533effd640 1 -- 172.21.15.60:0/3188978339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500054f0 con 0x5591d844dce0 2024-07-24T21:47:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.171+0000 7f533effd640 1 -- 172.21.15.60:0/3188978339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500050a0 con 0x5591d844dce0 2024-07-24T21:47:29.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 msgr2=0x5591d8459190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5350005c40 tx=0x7f5350048000 comp rx=0 tx=0).stop 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 shutdown_connections 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d845a590 0x5591d8456a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844e600 0x5591d8451e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 --2- 172.21.15.60:0/3188978339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d844dce0 0x5591d8459190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 >> 172.21.15.60:0/3188978339 conn(0x5591d844bb10 msgr2=0x5591d844abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 shutdown_connections 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.172+0000 7f535d8e6740 1 -- 172.21.15.60:0/3188978339 wait complete. 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.173+0000 7f535d8e6740 1 -- start start 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.173+0000 7f535d8e6740 1 Processor -- start 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.173+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d8462ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.173+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.173+0000 7f535d8e6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f53592d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d8462ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46422/0 (socket says 172.21.15.60:46422) 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5359ad2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f53592d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d8462ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51710/0 (socket says 172.21.15.60:51710) 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5359ad2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46954/0 (socket says 172.21.15.60:46954) 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 -- 172.21.15.60:0/1721994666 learned_addr learned my addr 172.21.15.60:0/1721994666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53541aaae0 con 0x5591d845a590 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53541abf50 con 0x5591d844dce0 2024-07-24T21:47:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53541ab1b0 con 0x5591d844e600 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 -- 172.21.15.60:0/1721994666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 msgr2=0x5591d8462ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d8462ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.174+0000 7f5358ad0640 1 -- 172.21.15.60:0/1721994666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 msgr2=0x5591d845e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f5358ad0640 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d84e46a0 con 0x5591d844e600 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f5359ad2640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f535d8e6740 10 client.0 _ll_register_callbacks cb 0x7f53440a8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53500054f0 con 0x5591d844e600 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f5358ad0640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5344009b10 tx=0x7f5344094040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf99d5793ab80569 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f533cff9640 10 client.25273 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f533cff9640 1 -- 172.21.15.60:0/1721994666 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53440c6070 con 0x5591d844e600 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f533cff9640 1 -- 172.21.15.60:0/1721994666 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f53440c7260 con 0x5591d844e600 2024-07-24T21:47:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f533cff9640 1 -- 172.21.15.60:0/1721994666 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f534401f040 con 0x5591d844e600 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.175+0000 7f535d8e6740 10 client.25273 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5591d84e5830 con 0x5591d844e600 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f533cff9640 1 -- 172.21.15.60:0/1721994666 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f53440adce0 con 0x5591d844e600 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 10 client.25273 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 10 client.25273 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 20 client.25273 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d84ee560 0x5591d84f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 4 client.25273 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.176+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x7f53500054f0 con 0x5591d84ee560 2024-07-24T21:47:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.177+0000 7f53592d1640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d84ee560 0x5591d84f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.177+0000 7f53592d1640 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d84ee560 0x5591d84f0970 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc78a3f376ee23e server_cookie=7b2c39ccbc5e62d9 in_seq=0 out_seq=0 2024-07-24T21:47:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.177+0000 7f533cff9640 10 client.25273 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.187+0000 7f533cff9640 1 -- 172.21.15.60:0/1721994666 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f53500054f0 con 0x5591d84ee560 2024-07-24T21:47:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.187+0000 7f533cff9640 10 client.25273 handle_command_reply: tid=0 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 client.25273 shutdown 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 msgr2=0x5591d845e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5344009b10 tx=0x7f5344094040 comp rx=0 tx=0).stop 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 shutdown_connections 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591d84ee560 0x5591d84f0970 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591d845a590 0x5591d845e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f533cff9640 0 client.25273 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591d844e600 0x5591d845e1b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 --2- 172.21.15.60:0/1721994666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591d844dce0 0x5591d8462ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 >> 172.21.15.60:0/1721994666 conn(0x5591d844bb10 msgr2=0x5591d844a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 shutdown_connections 2024-07-24T21:47:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.188+0000 7f535d8e6740 1 -- 172.21.15.60:0/1721994666 wait complete. 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.189+0000 7f535d8e6740 20 client.25273 trim_cache size 0 max 16384 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 26, 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 2, 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:47:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: "98c11f33": { 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 2, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.3, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:47:29.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:29.193 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.189+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 msgr2=0x7f53400b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.189+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 0x7f53400b6200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f534c005fd0 tx=0x7f534c009b00 comp rx=0 tx=0).stop 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.189+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 msgr2=0x7f5354198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.189+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f534400fb60 tx=0x7f5344009610 comp rx=0 tx=0).stop 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 shutdown_connections 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53400b3d40 0x7f53400b6200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f535419d630 0x7f535419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53541079e0 0x7f5354198290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 --2- 172.21.15.60:0/696872515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5354107060 0x7f5354197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 >> 172.21.15.60:0/696872515 conn(0x7f53540fd0c0 msgr2=0x7f53540fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 shutdown_connections 2024-07-24T21:47:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:29.190+0000 7f535b535640 1 -- 172.21.15.60:0/696872515 wait complete. 2024-07-24T21:47:29.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:29 smithi060 ceph-mon[33000]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-07-24T21:47:29.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:29 smithi123 ceph-mon[37143]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-07-24T21:47:29.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:29 smithi086 ceph-mon[37128]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-07-24T21:47:30.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:30 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:47:30.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:30 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:47:30.541 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '98c11f33', quiesce took 10.9 seconds. Will release after: 9.450000000000001 2024-07-24T21:47:30.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:30 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:47:31.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:31 smithi123 ceph-mon[37143]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.494+0000 7fe6ecad4640 1 -- 172.21.15.60:0/459745860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 msgr2=0x7fe6e8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.494+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/459745860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8101a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009900 tx=0x7fe6d802a410 comp rx=0 tx=0).stop 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 -- 172.21.15.60:0/459745860 shutdown_connections 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/459745860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/459745860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 0x7fe6e81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/459745860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8101a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 -- 172.21.15.60:0/459745860 >> 172.21.15.60:0/459745860 conn(0x7fe6e80fd0c0 msgr2=0x7fe6e80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 -- 172.21.15.60:0/459745860 shutdown_connections 2024-07-24T21:47:31.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.495+0000 7fe6ecad4640 1 -- 172.21.15.60:0/459745860 wait complete. 2024-07-24T21:47:31.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:31 smithi060 ceph-mon[33000]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.496+0000 7fe6ecad4640 1 Processor -- start 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.496+0000 7fe6ecad4640 1 -- start start 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.496+0000 7fe6ecad4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6ecad4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 0x7fe6e8074690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6ecad4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6ecad4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81105e0 con 0x7fe6e81079e0 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6ecad4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8110460 con 0x7fe6e8107060 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6ecad4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8110760 con 0x7fe6e8101650 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 0x7fe6e8074690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46964/0 (socket says 172.21.15.60:46964) 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 -- 172.21.15.60:0/4069709215 learned_addr learned my addr 172.21.15.60:0/4069709215 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:31.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e659c640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 -- 172.21.15.60:0/4069709215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 msgr2=0x7fe6e8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 -- 172.21.15.60:0/4069709215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 msgr2=0x7fe6e8074690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 0x7fe6e8074690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.497+0000 7fe6e6d9d640 1 -- 172.21.15.60:0/4069709215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e806dd60 con 0x7fe6e81079e0 2024-07-24T21:47:31.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.498+0000 7fe6e6d9d640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000efd0 tx=0x7fe6e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.498+0000 7fe6d77fe640 1 -- 172.21.15.60:0/4069709215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0003b70 con 0x7fe6e81079e0 2024-07-24T21:47:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.498+0000 7fe6d77fe640 1 -- 172.21.15.60:0/4069709215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e0003d10 con 0x7fe6e81079e0 2024-07-24T21:47:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.498+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e806e050 con 0x7fe6e81079e0 2024-07-24T21:47:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.498+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e8072f90 con 0x7fe6e81079e0 2024-07-24T21:47:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.499+0000 7fe6d77fe640 1 -- 172.21.15.60:0/4069709215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00108a0 con 0x7fe6e81079e0 2024-07-24T21:47:31.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.502+0000 7fe6eee85740 1 Processor -- start 2024-07-24T21:47:31.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6d77fe640 1 -- 172.21.15.60:0/4069709215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe6e0010b00 con 0x7fe6e81079e0 2024-07-24T21:47:31.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 -- start start 2024-07-24T21:47:31.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3fa0 0x557c720cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557c7203a960 con 0x557c720cff30 2024-07-24T21:47:31.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c72029e60 con 0x557c720c3680 2024-07-24T21:47:31.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.503+0000 7fe6eee85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c7202dea0 con 0x557c720c3fa0 2024-07-24T21:47:31.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51736/0 (socket says 172.21.15.60:51736) 2024-07-24T21:47:31.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/1964666364 learned_addr learned my addr 172.21.15.60:0/1964666364 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:31.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6d77fe640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 0x7fe6d00b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.504+0000 7fe6e6d9d640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 0x7fe6d00b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/1964666364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3fa0 msgr2=0x557c720cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3fa0 0x557c720cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/1964666364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 msgr2=0x557c720c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720c3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/1964666364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c71fb32d0 con 0x557c720cff30 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6d77fe640 1 -- 172.21.15.60:0/4069709215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6e00e4170 con 0x7fe6e81079e0 2024-07-24T21:47:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.505+0000 7fe6e6d9d640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 0x7fe6d00b6380 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005c40 tx=0x7fe6e0009bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.506+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720d2320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0174b0 tx=0x7fe6dc00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d36dadc24030dc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.506+0000 7fe6d4ff9640 1 -- 172.21.15.60:0/1964666364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0165f0 con 0x557c720cff30 2024-07-24T21:47:31.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.507+0000 7fe6d4ff9640 1 -- 172.21.15.60:0/1964666364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc016c40 con 0x557c720cff30 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.507+0000 7fe6d4ff9640 1 -- 172.21.15.60:0/1964666364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc01b9f0 con 0x557c720cff30 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.507+0000 7fe6eee85740 1 -- 172.21.15.60:0/1964666364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 msgr2=0x557c720d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.507+0000 7fe6eee85740 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720d2320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0174b0 tx=0x7fe6dc00b600 comp rx=0 tx=0).stop 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 -- 172.21.15.60:0/1964666364 shutdown_connections 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720d2320 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3fa0 0x557c720cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 --2- 172.21.15.60:0/1964666364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 -- 172.21.15.60:0/1964666364 >> 172.21.15.60:0/1964666364 conn(0x557c720c14b0 msgr2=0x557c720c2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 -- 172.21.15.60:0/1964666364 shutdown_connections 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.508+0000 7fe6eee85740 1 -- 172.21.15.60:0/1964666364 wait complete. 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.509+0000 7fe6eee85740 1 -- start start 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.509+0000 7fe6eee85740 1 Processor -- start 2024-07-24T21:47:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.509+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51750/0 (socket says 172.21.15.60:51750) 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3125706710 learned_addr learned my addr 172.21.15.60:0/3125706710 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557c71e8d700 con 0x557c720c3fa0 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c71fcc5a0 con 0x557c720c3680 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c720d2c00 con 0x557c720cff30 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6eee85740 10 client.0 _ll_register_callbacks cb 0x557c71fec910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 msgr2=0x557c720cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3125706710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 msgr2=0x557c720ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.510+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c71fbf630 con 0x557c720c3680 2024-07-24T21:47:31.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005fd0 tx=0x7fe6dc00b270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6066d1dffb1fb43b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc028070 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6dc016a00 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc016c40 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.511+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc031050 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 10 client.25292 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c71d8a650 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc01b550 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe6dc02b020 con 0x557c720c3680 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 10 client.25292 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 20 client.25292 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:31.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c720e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 4 client.25292 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:31.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.512+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe6dc016c40 con 0x557c720df8e0 2024-07-24T21:47:31.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.513+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c720e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.513+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c720e1cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cab4aaff0bc05ef server_cookie=5fbad5cbc76a99c3 in_seq=0 out_seq=0 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.513+0000 7fe6cdffb640 10 client.25292 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3125706710 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe6dc016c40 con 0x557c720df8e0 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6cdffb640 10 client.25292 handle_command_reply: tid=0 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 client.25292 shutdown 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 msgr2=0x557c720cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005fd0 tx=0x7fe6dc00b270 comp rx=0 tx=0).stop 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 shutdown_connections 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c720e1cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6cdffb640 0 client.25292 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3125706710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720cf980 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 >> 172.21.15.60:0/3125706710 conn(0x557c720c14b0 msgr2=0x557c720c7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 shutdown_connections 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 1 -- 172.21.15.60:0/3125706710 wait complete. 2024-07-24T21:47:31.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.514+0000 7fe6eee85740 20 client.25292 trim_cache size 0 max 16384 2024-07-24T21:47:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:31 smithi086 ceph-mon[37128]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.551+0000 7fe6eee85740 1 Processor -- start 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.551+0000 7fe6eee85740 1 -- start start 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3680 0x557c720c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c80110e0 con 0x557c720cff30 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800e9b0 con 0x557c720c3fa0 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6eee85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800c800 con 0x557c720c3680 2024-07-24T21:47:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51754/0 (socket says 172.21.15.60:51754) 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6e659c640 1 -- 172.21.15.60:0/3826484930 learned_addr learned my addr 172.21.15.60:0/3826484930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.552+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3680 0x557c720c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 -- 172.21.15.60:0/3826484930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3680 msgr2=0x557c720c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3680 0x557c720c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 -- 172.21.15.60:0/3826484930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 msgr2=0x557c720e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 -- 172.21.15.60:0/3826484930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c720dca40 con 0x557c720c3fa0 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80092a0 tx=0x7fe6d8004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ed327b57673959 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6cffff640 1 -- 172.21.15.60:0/3826484930 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d804a030 con 0x557c720c3fa0 2024-07-24T21:47:31.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6cffff640 1 -- 172.21.15.60:0/3826484930 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d80054f0 con 0x557c720c3fa0 2024-07-24T21:47:31.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.553+0000 7fe6cffff640 1 -- 172.21.15.60:0/3826484930 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8035040 con 0x557c720c3fa0 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 -- 172.21.15.60:0/3826484930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 msgr2=0x557c720e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80092a0 tx=0x7fe6d8004500 comp rx=0 tx=0).stop 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 -- 172.21.15.60:0/3826484930 shutdown_connections 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720cff30 0x557c720e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3fa0 0x557c720e74f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3826484930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720c3680 0x557c720c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 -- 172.21.15.60:0/3826484930 >> 172.21.15.60:0/3826484930 conn(0x557c720c14b0 msgr2=0x557c720c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 -- 172.21.15.60:0/3826484930 shutdown_connections 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.554+0000 7fe6eee85740 1 -- 172.21.15.60:0/3826484930 wait complete. 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.555+0000 7fe6eee85740 1 -- start start 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.555+0000 7fe6eee85740 1 Processor -- start 2024-07-24T21:47:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.555+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.555+0000 7fe6eee85740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51760/0 (socket says 172.21.15.60:51760) 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3784276089 learned_addr learned my addr 172.21.15.60:0/3784276089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800ce70 con 0x557c720c3fa0 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800e2e0 con 0x557c720c3680 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800d540 con 0x557c720cff30 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6eee85740 10 client.0 _ll_register_callbacks cb 0x7fe6dc017380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 msgr2=0x557c720ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3784276089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 msgr2=0x557c720c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c720ca8d0 con 0x557c720c3680 2024-07-24T21:47:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6e5d9b640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005d00 tx=0x7fe6dc005930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9776140c34669805 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.556+0000 7fe6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc03f060 con 0x557c720c3680 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d80054f0 con 0x557c720c3680 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6dc003bb0 con 0x557c720c3680 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc028040 con 0x557c720c3680 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6eee85740 10 client.25298 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.557+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c7214a520 con 0x557c720c3680 2024-07-24T21:47:31.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc003db0 con 0x557c720c3680 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe6dc02b070 con 0x557c720c3680 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 10 client.25298 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 10 client.25298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 20 client.25298 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c7214cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 4 client.25298 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.558+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe6d80054f0 con 0x557c720df8e0 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.559+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c7214cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.559+0000 7fe6cdffb640 10 client.25298 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.559+0000 7fe6e659c640 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c7214cc10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d90a96c1283ded8 server_cookie=8b9d2257c4010e8b in_seq=0 out_seq=0 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6cdffb640 1 -- 172.21.15.60:0/3784276089 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe6d80054f0 con 0x557c720df8e0 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6cdffb640 10 client.25298 handle_command_reply: tid=0 2024-07-24T21:47:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 client.25298 shutdown 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 msgr2=0x557c720d8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005d00 tx=0x7fe6dc005930 comp rx=0 tx=0).stop 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 shutdown_connections 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c720df8e0 0x557c7214cc10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c720cff30 0x557c720ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c720c3fa0 0x557c720c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 --2- 172.21.15.60:0/3784276089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c720c3680 0x557c720d8560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 >> 172.21.15.60:0/3784276089 conn(0x557c720c14b0 msgr2=0x557c720c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6cdffb640 0 client.25298 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 shutdown_connections 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 1 -- 172.21.15.60:0/3784276089 wait complete. 2024-07-24T21:47:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.560+0000 7fe6eee85740 20 client.25298 trim_cache size 0 max 16384 2024-07-24T21:47:31.565 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:31.565 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:47:31.565 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:47:31.565 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 msgr2=0x7fe6d00b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 0x7fe6d00b6380 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005c40 tx=0x7fe6e0009bd0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 msgr2=0x7fe6e806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000efd0 tx=0x7fe6e000c6a0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 shutdown_connections 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d00b3ec0 0x7fe6d00b6380 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6e81079e0 0x7fe6e806d4a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8107060 0x7fe6e8074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 --2- 172.21.15.60:0/4069709215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6e8101650 0x7fe6e8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 >> 172.21.15.60:0/4069709215 conn(0x7fe6e80fd0c0 msgr2=0x7fe6e80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.561+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 shutdown_connections 2024-07-24T21:47:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:31.562+0000 7fe6ecad4640 1 -- 172.21.15.60:0/4069709215 wait complete. 2024-07-24T21:47:32.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:47:32.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:47:32.780 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:33.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:33 smithi123 ceph-mon[37143]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-24T21:47:33.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:33 smithi086 ceph-mon[37128]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-24T21:47:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:33 smithi060 ceph-mon[33000]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-07-24T21:47:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.426+0000 7f18c5d77640 1 -- 172.21.15.60:0/461453855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 msgr2=0x7f18c0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.426+0000 7f18c5d77640 1 --2- 172.21.15.60:0/461453855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c0102b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009900 tx=0x7f18b002a410 comp rx=0 tx=0).stop 2024-07-24T21:47:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 -- 172.21.15.60:0/461453855 shutdown_connections 2024-07-24T21:47:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 --2- 172.21.15.60:0/461453855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108ae0 0x7f18c010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 --2- 172.21.15.60:0/461453855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108160 0x7f18c01085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 --2- 172.21.15.60:0/461453855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c0102b30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 -- 172.21.15.60:0/461453855 >> 172.21.15.60:0/461453855 conn(0x7f18c00fe200 msgr2=0x7f18c0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 -- 172.21.15.60:0/461453855 shutdown_connections 2024-07-24T21:47:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.427+0000 7f18c5d77640 1 -- 172.21.15.60:0/461453855 wait complete. 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.428+0000 7f18c5d77640 1 Processor -- start 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.428+0000 7f18c5d77640 1 -- start start 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.428+0000 7f18c5d77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.428+0000 7f18c5d77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 0x7f18c010e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47026/0 (socket says 172.21.15.60:47026) 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18bf7fe640 1 -- 172.21.15.60:0/64016120 learned_addr learned my addr 172.21.15.60:0/64016120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108ae0 0x7f18c01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0112760 con 0x7f18c0102750 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01125e0 con 0x7f18c0108160 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18c01128e0 con 0x7f18c0108ae0 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18beffd640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 0x7f18c010e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18bffff640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108ae0 0x7f18c01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18beffd640 1 -- 172.21.15.60:0/64016120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108ae0 msgr2=0x7f18c01a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18beffd640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108ae0 0x7f18c01a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.429+0000 7f18beffd640 1 -- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 msgr2=0x7f18c010e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18beffd640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18beffd640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01a1f00 con 0x7f18c0108160 2024-07-24T21:47:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18bf7fe640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18beffd640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 0x7f18c010e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18b400d720 tx=0x7f18b400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4017070 con 0x7f18c0108160 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c01a21f0 con 0x7f18c0108160 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01abb60 con 0x7f18c0108160 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.430+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b400de30 con 0x7f18c0108160 2024-07-24T21:47:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.431+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40047a0 con 0x7f18c0108160 2024-07-24T21:47:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.432+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ac0052c0 con 0x7f18c0108160 2024-07-24T21:47:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.434+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f18b40099d0 con 0x7f18c0108160 2024-07-24T21:47:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.435+0000 7f18bcff9640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 0x7f18a80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.435+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f18b40e39a0 con 0x7f18c0108160 2024-07-24T21:47:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.435+0000 7f18bf7fe640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 0x7f18a80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.436+0000 7f18bf7fe640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 0x7f18a80b6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18b00096c0 tx=0x7f18b0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.440+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f18b4020020 con 0x7f18c0108160 2024-07-24T21:47:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.921+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ac005880 con 0x7f18c0108160 2024-07-24T21:47:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.922+0000 7f18bcff9640 1 -- 172.21.15.60:0/64016120 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f18b40aacb0 con 0x7f18c0108160 2024-07-24T21:47:34.923 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:47:34.924 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:47:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:47:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 msgr2=0x7f18a80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 0x7f18a80b6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18b00096c0 tx=0x7f18b0031400 comp rx=0 tx=0).stop 2024-07-24T21:47:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 msgr2=0x7f18c010e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 0x7f18c010e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18b400d720 tx=0x7f18b400dbe0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 shutdown_connections 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18a80b3dc0 0x7f18a80b6280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c0108ae0 0x7f18c01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c0108160 0x7f18c010e9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 --2- 172.21.15.60:0/64016120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18c0102750 0x7f18c010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 >> 172.21.15.60:0/64016120 conn(0x7f18c00fe200 msgr2=0x7f18c0074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.925+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 shutdown_connections 2024-07-24T21:47:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:34.926+0000 7f18c5d77640 1 -- 172.21.15.60:0/64016120 wait complete. 2024-07-24T21:47:35.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:35 smithi060 ceph-mon[33000]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:35.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/64016120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:35.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:35 smithi123 ceph-mon[37143]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:35.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/64016120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:35 smithi086 ceph-mon[37128]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-07-24T21:47:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/64016120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:35.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:47:36.105 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:37.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:37 smithi123 ceph-mon[37143]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:37.514 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:37 smithi060 ceph-mon[33000]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:37.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:37 smithi086 ceph-mon[37128]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/3847392411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78106f90 msgr2=0x7f3c781073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/3847392411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78106f90 0x7f3c781073d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009860 tx=0x7f3c6802a340 comp rx=0 tx=0).stop 2024-07-24T21:47:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/3847392411 shutdown_connections 2024-07-24T21:47:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/3847392411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 0x7f3c7810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/3847392411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78106f90 0x7f3c781073d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/3847392411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78101590 0x7f3c78101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.900+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/3847392411 >> 172.21.15.60:0/3847392411 conn(0x7f3c780fd010 msgr2=0x7f3c780ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.901+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/3847392411 shutdown_connections 2024-07-24T21:47:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.901+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/3847392411 wait complete. 2024-07-24T21:47:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.902+0000 7f3c7fa9a640 1 Processor -- start 2024-07-24T21:47:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.902+0000 7f3c7fa9a640 1 -- start start 2024-07-24T21:47:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 0x7f3c78196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 0x7f3c7819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7d035640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78110460 con 0x7f3c78106f90 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7d035640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40648/0 (socket says 172.21.15.60:40648) 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7d035640 1 -- 172.21.15.60:0/4274138577 learned_addr learned my addr 172.21.15.60:0/4274138577 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7d836640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 0x7f3c78196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781102e0 con 0x7f3c78101590 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781105e0 con 0x7f3c78107910 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.903+0000 7f3c7e037640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 0x7f3c7819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 -- 172.21.15.60:0/4274138577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 msgr2=0x7f3c7819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 0x7f3c7819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 -- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 msgr2=0x7f3c78196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 -- 172.21.15.60:0/4274138577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7819cac0 con 0x7f3c78101590 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d035640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c7d836640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 0x7f3c78196a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c760 tx=0x7f3c7400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c6effd640 1 -- 172.21.15.60:0/4274138577 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74009a40 con 0x7f3c78101590 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c6effd640 1 -- 172.21.15.60:0/4274138577 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c74009be0 con 0x7f3c78101590 2024-07-24T21:47:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.904+0000 7f3c6effd640 1 -- 172.21.15.60:0/4274138577 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74003ce0 con 0x7f3c78101590 2024-07-24T21:47:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.905+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7819cdb0 con 0x7f3c78101590 2024-07-24T21:47:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.905+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a3470 con 0x7f3c78101590 2024-07-24T21:47:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.909+0000 7f3c827f9740 1 Processor -- start 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.909+0000 7f3c827f9740 1 -- start start 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.909+0000 7f3c6effd640 1 -- 172.21.15.60:0/4274138577 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3c7401d030 con 0x7f3c78101590 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 0x5557c2cd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5557c2c34280 con 0x5557c2cbe8c0 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557c2c23780 con 0x5557c2ccf580 2024-07-24T21:47:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c827f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5557c2c277c0 con 0x5557c2cbf1e0 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7e037640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7e037640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40658/0 (socket says 172.21.15.60:40658) 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7e037640 1 -- 172.21.15.60:0/1605189620 learned_addr learned my addr 172.21.15.60:0/1605189620 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7d035640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 0x5557c2cd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7d836640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7d035640 1 -- 172.21.15.60:0/1605189620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2cbf1e0 msgr2=0x5557c2ccf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7d035640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.910+0000 7f3c7d035640 1 -- 172.21.15.60:0/1605189620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 msgr2=0x5557c2cbeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c7d035640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c7d035640 1 -- 172.21.15.60:0/1605189620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c2987790 con 0x5557c2ccf580 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c7e037640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c7d035640 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 0x5557c2cd1970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009830 tx=0x7f3c68005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edae1fc74e9c0c1b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c63fff640 1 -- 172.21.15.60:0/1605189620 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68037040 con 0x5557c2ccf580 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c63fff640 1 -- 172.21.15.60:0/1605189620 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c68030dd0 con 0x5557c2ccf580 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.911+0000 7f3c63fff640 1 -- 172.21.15.60:0/1605189620 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68032a50 con 0x5557c2ccf580 2024-07-24T21:47:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.912+0000 7f3c6effd640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 0x7f3c640b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.912+0000 7f3c6effd640 1 -- 172.21.15.60:0/4274138577 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3c740e2ec0 con 0x7f3c78101590 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.912+0000 7f3c827f9740 1 -- 172.21.15.60:0/1605189620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 msgr2=0x5557c2cd1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.912+0000 7f3c827f9740 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 0x5557c2cd1970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009830 tx=0x7f3c68005b20 comp rx=0 tx=0).stop 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c7e037640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 0x7f3c640b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c7e037640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 0x7f3c640b6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005f40 tx=0x7f3c700096a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c827f9740 1 -- 172.21.15.60:0/1605189620 shutdown_connections 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c827f9740 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2ccf580 0x5557c2cd1970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c827f9740 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c827f9740 1 --2- 172.21.15.60:0/1605189620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2cbeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.913+0000 7f3c827f9740 1 -- 172.21.15.60:0/1605189620 >> 172.21.15.60:0/1605189620 conn(0x5557c2cbc6f0 msgr2=0x5557c2cbd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.914+0000 7f3c827f9740 1 -- 172.21.15.60:0/1605189620 shutdown_connections 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.914+0000 7f3c827f9740 1 -- 172.21.15.60:0/1605189620 wait complete. 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.914+0000 7f3c827f9740 1 -- start start 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.914+0000 7f3c827f9740 1 Processor -- start 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2ccecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2ccc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5557c2b5dab0 con 0x5557c2cbf1e0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557c2bab9e0 con 0x5557c2cbe8c0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5557c2cd2200 con 0x5557c2ccf580 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c827f9740 10 client.0 _ll_register_callbacks cb 0x5557c2933a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40672/0 (socket says 172.21.15.60:40672) 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 -- 172.21.15.60:0/229536035 learned_addr learned my addr 172.21.15.60:0/229536035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d836640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2ccecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7e037640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2ccc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 -- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 msgr2=0x5557c2ccc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2ccc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 -- 172.21.15.60:0/229536035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 msgr2=0x5557c2ccecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.915+0000 7f3c7d035640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2ccecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c7d035640 1 -- 172.21.15.60:0/229536035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c2a00620 con 0x5557c2cbf1e0 2024-07-24T21:47:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c7d836640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2ccecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c7d035640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3c6802a380 tx=0x7f3c68030560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639d0bd4a874d759 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68037030 con 0x5557c2cbf1e0 2024-07-24T21:47:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3c68004450 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.916+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557c2bd2ce0 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.917+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c68056050 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.917+0000 7f3c827f9740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.917+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557c2cdd980 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.917+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5557c2bd2ce0 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c68003a30 con 0x5557c2cbf1e0 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 20 client.15759 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2ce10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 4 client.15759 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c2c361d0 con 0x5557c2cdecd0 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.918+0000 7f3c7d836640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2ce10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.919+0000 7f3c7d836640 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2ce10c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5d2b2c4c069d0f server_cookie=3d27416cc3d5793 in_seq=0 out_seq=0 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.919+0000 7f3c617fa640 10 client.15759 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.919+0000 7f3c617fa640 1 -- 172.21.15.60:0/229536035 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557c2c361d0 con 0x5557c2cdecd0 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.919+0000 7f3c617fa640 10 client.15759 handle_command_reply: tid=0 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 client.15759 shutdown 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 msgr2=0x5557c2ccc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3c6802a380 tx=0x7f3c68030560 comp rx=0 tx=0).stop 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 shutdown_connections 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2ce10c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2ccc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c617fa640 0 client.15759 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2ccc1a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 --2- 172.21.15.60:0/229536035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2ccecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 >> 172.21.15.60:0/229536035 conn(0x5557c2cbc6f0 msgr2=0x5557c2cc2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 shutdown_connections 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 1 -- 172.21.15.60:0/229536035 wait complete. 2024-07-24T21:47:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.920+0000 7f3c827f9740 20 client.15759 trim_cache size 0 max 16384 2024-07-24T21:47:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.940+0000 7f4d71d9b640 1 -- 172.21.15.60:0/2446841677 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d640318a0 con 0x7f4d74101650 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.940+0000 7f4d79162640 1 -- 172.21.15.60:0/2446841677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 msgr2=0x7f4d74101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.940+0000 7f4d79162640 1 --2- 172.21.15.60:0/2446841677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74101a30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d6402a410 comp rx=0 tx=0).stop 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 -- 172.21.15.60:0/2446841677 shutdown_connections 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 --2- 172.21.15.60:0/2446841677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 --2- 172.21.15.60:0/2446841677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d741074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 --2- 172.21.15.60:0/2446841677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74101a30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 -- 172.21.15.60:0/2446841677 >> 172.21.15.60:0/2446841677 conn(0x7f4d740fd0c0 msgr2=0x7f4d740ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 -- 172.21.15.60:0/2446841677 shutdown_connections 2024-07-24T21:47:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.941+0000 7f4d79162640 1 -- 172.21.15.60:0/2446841677 wait complete. 2024-07-24T21:47:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.943+0000 7f4d79162640 1 Processor -- start 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.943+0000 7f4d79162640 1 -- start start 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d79162640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d79162640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d72d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d72d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40698/0 (socket says 172.21.15.60:40698) 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d79162640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d79162640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741116e0 con 0x7f4d74101650 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d7259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d7259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44260/0 (socket says 172.21.15.60:44260) 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d7259c640 1 -- 172.21.15.60:0/3098828487 learned_addr learned my addr 172.21.15.60:0/3098828487 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d79162640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74111560 con 0x7f4d74107060 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.944+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74111860 con 0x7f4d741079e0 2024-07-24T21:47:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7259c640 1 -- 172.21.15.60:0/3098828487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 msgr2=0x7f4d7419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7259c640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7259c640 1 -- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 msgr2=0x7f4d74197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7259c640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7259c640 1 -- 172.21.15.60:0/3098828487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7419dc80 con 0x7f4d74107060 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.945+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d7259c640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800bea0 tx=0x7f4d6800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d5bfff640 1 -- 172.21.15.60:0/3098828487 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68016070 con 0x7f4d74107060 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d5bfff640 1 -- 172.21.15.60:0/3098828487 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d68009c30 con 0x7f4d74107060 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7419df70 con 0x7f4d74107060 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d5bfff640 1 -- 172.21.15.60:0/3098828487 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680196a0 con 0x7f4d74107060 2024-07-24T21:47:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.946+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7406a4f0 con 0x7f4d74107060 2024-07-24T21:47:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 Processor -- start 2024-07-24T21:47:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 -- start start 2024-07-24T21:47:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d5bfff640 1 -- 172.21.15.60:0/3098828487 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4d68019840 con 0x7f4d74107060 2024-07-24T21:47:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597236fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723713a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5597236739b0 con 0x559723706a30 2024-07-24T21:47:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559723662eb0 con 0x559723711630 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.950+0000 7f4d7b513740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559723666ef0 con 0x5597236fc6d0 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.951+0000 7f4d72d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.951+0000 7f4d72d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40706/0 (socket says 172.21.15.60:40706) 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.951+0000 7f4d72d9d640 1 -- 172.21.15.60:0/806500013 learned_addr learned my addr 172.21.15.60:0/806500013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 -- 172.21.15.60:0/806500013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 msgr2=0x5597236fcab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597236fcab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 -- 172.21.15.60:0/806500013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 msgr2=0x559723713a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723713a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 -- 172.21.15.60:0/806500013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597235ec320 con 0x559723706a30 2024-07-24T21:47:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d5bfff640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 0x7f4d5c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f3c827f9740 1 Processor -- start 2024-07-24T21:47:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f3c827f9740 1 -- start start 2024-07-24T21:47:37.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d5bfff640 1 -- 172.21.15.60:0/3098828487 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d68010140 con 0x7f4d74107060 2024-07-24T21:47:37.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.952+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d640315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f892896cd6b2519e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 0x7f4d5c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f4d597fa640 1 -- 172.21.15.60:0/806500013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64040070 con 0x559723706a30 2024-07-24T21:47:37.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f4d597fa640 1 -- 172.21.15.60:0/806500013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d64005040 con 0x559723706a30 2024-07-24T21:47:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cc1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.953+0000 7f4d597fa640 1 -- 172.21.15.60:0/806500013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64035040 con 0x559723706a30 2024-07-24T21:47:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c827f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781ae730 con 0x5557c2cbf1e0 2024-07-24T21:47:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c827f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781ac000 con 0x5557c2cbe8c0 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44272/0 (socket says 172.21.15.60:44272) 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 -- 172.21.15.60:0/365791164 learned_addr learned my addr 172.21.15.60:0/365791164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781a9e50 con 0x5557c2ccf580 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d035640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7e037640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cc1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 -- 172.21.15.60:0/365791164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 msgr2=0x5557c2cc1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cc1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 -- 172.21.15.60:0/365791164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 msgr2=0x5557c2cd1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 -- 172.21.15.60:0/806500013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 msgr2=0x5597236fcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d640315c0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 0x7f4d5c0b63d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00a7c0 tx=0x7f4d6c00d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d836640 1 -- 172.21.15.60:0/365791164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c2cb6c00 con 0x5557c2cbe8c0 2024-07-24T21:47:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f3c7d035640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:37.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 -- 172.21.15.60:0/806500013 shutdown_connections 2024-07-24T21:47:37.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723713a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723706a30 0x5597236fcff0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 --2- 172.21.15.60:0/806500013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597236fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.954+0000 7f4d7b513740 1 -- 172.21.15.60:0/806500013 >> 172.21.15.60:0/806500013 conn(0x5597236fa500 msgr2=0x5597236fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f3c7d836640 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005fd0 tx=0x7f3c740e5290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2461d80695d46dff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f4d7b513740 1 -- 172.21.15.60:0/806500013 shutdown_connections 2024-07-24T21:47:37.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f4d7b513740 1 -- 172.21.15.60:0/806500013 wait complete. 2024-07-24T21:47:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f3c637fe640 1 -- 172.21.15.60:0/365791164 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740ce040 con 0x5557c2cbe8c0 2024-07-24T21:47:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f3c637fe640 1 -- 172.21.15.60:0/365791164 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c740b0030 con 0x5557c2cbe8c0 2024-07-24T21:47:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f3c637fe640 1 -- 172.21.15.60:0/365791164 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74009e20 con 0x5557c2cbe8c0 2024-07-24T21:47:37.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f4d7b513740 1 -- start start 2024-07-24T21:47:37.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.955+0000 7f4d7b513740 1 Processor -- start 2024-07-24T21:47:37.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x559723703a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723703fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5597234c6700 con 0x559723711630 2024-07-24T21:47:37.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597236055f0 con 0x559723706a30 2024-07-24T21:47:37.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597237143c0 con 0x5597236fc6d0 2024-07-24T21:47:37.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7b513740 10 client.0 _ll_register_callbacks cb 0x559723625960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 msgr2=0x5557c2cdb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005fd0 tx=0x7f3c740e5290 comp rx=0 tx=0).stop 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 shutdown_connections 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cc1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 --2- 172.21.15.60:0/365791164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbe8c0 0x5557c2cdb320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 >> 172.21.15.60:0/365791164 conn(0x5557c2cbc6f0 msgr2=0x5557c2cd1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 shutdown_connections 2024-07-24T21:47:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40722/0 (socket says 172.21.15.60:40722) 2024-07-24T21:47:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7359e640 1 -- 172.21.15.60:0/3472717864 learned_addr learned my addr 172.21.15.60:0/3472717864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f3c827f9740 1 -- 172.21.15.60:0/365791164 wait complete. 2024-07-24T21:47:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x559723703a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.956+0000 7f4d7259c640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723703fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 -- start start 2024-07-24T21:47:37.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 -- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 msgr2=0x559723703a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x559723703a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 -- 172.21.15.60:0/3472717864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 msgr2=0x559723703fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723703fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 -- 172.21.15.60:0/3472717864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597235f8680 con 0x559723711630 2024-07-24T21:47:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 Processor -- start 2024-07-24T21:47:37.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7359e640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009fd0 tx=0x7f4d6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff7b8fd01469ad1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2ce6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cd7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781aa4c0 con 0x5557c2cbe8c0 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781ab930 con 0x5557c2cbf1e0 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781aab90 con 0x5557c2ccf580 2024-07-24T21:47:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f3c827f9740 10 client.0 _ll_register_callbacks cb 0x7f3c70009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:37.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c025060 con 0x559723711630 2024-07-24T21:47:37.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.957+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d64005040 con 0x559723711630 2024-07-24T21:47:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7e037640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cd7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d035640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7e037640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cd7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56582/0 (socket says 172.21.15.60:56582) 2024-07-24T21:47:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7e037640 1 -- 172.21.15.60:0/2262636278 learned_addr learned my addr 172.21.15.60:0/2262636278 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2ce6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d6c0205f0 con 0x559723711630 2024-07-24T21:47:37.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d6c0207f0 con 0x559723711630 2024-07-24T21:47:37.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f4d7b513740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:37.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597233c3650 con 0x559723711630 2024-07-24T21:47:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 -- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 msgr2=0x5557c2cd7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cd7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 -- 172.21.15.60:0/2262636278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbf1e0 msgr2=0x5557c2cd7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.958+0000 7f3c7d836640 1 -- 172.21.15.60:0/2262636278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c2ce5760 con 0x5557c2cbe8c0 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c7d836640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2ce6eb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c740abe30 tx=0x7f3c7400c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a614f1f4b9e0c5b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740ce070 con 0x5557c2cbe8c0 2024-07-24T21:47:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3c740aaed0 con 0x5557c2cbe8c0 2024-07-24T21:47:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c740b0030 con 0x5557c2cbe8c0 2024-07-24T21:47:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c74009e00 con 0x5557c2cbe8c0 2024-07-24T21:47:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c827f9740 10 client.15777 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:37.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d64005040 con 0x559723711630 2024-07-24T21:47:37.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4d6c015590 con 0x559723711630 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557c2d48a60 con 0x5557c2cbe8c0 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c740b0030 con 0x5557c2cbe8c0 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c740a75e0 con 0x5557c2cbe8c0 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 20 client.15777 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:37.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:47:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 20 client.15774 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x559723721300 0x559723723720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 4 client.15774 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.959+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559723675900 con 0x559723721300 2024-07-24T21:47:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2d4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 4 client.15777 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:47:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557c2c31c70 con 0x5557c2cdecd0 2024-07-24T21:47:37.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x559723721300 0x559723723720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c7d035640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2d4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c617fa640 10 client.15777 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f3c7d035640 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2d4b130 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f581a72cd00a27f9 server_cookie=2b82e9b09170e3f in_seq=0 out_seq=0 2024-07-24T21:47:37.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x559723721300 0x559723723720 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db6523ffacabd989 server_cookie=a34a6004c2a4c37c in_seq=0 out_seq=0 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.960+0000 7f4d56ff5640 10 client.15774 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d56ff5640 1 -- 172.21.15.60:0/3472717864 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559723675900 con 0x559723721300 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d56ff5640 10 client.15774 handle_command_reply: tid=0 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 client.15774 shutdown 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 msgr2=0x559723704510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009fd0 tx=0x7f4d6c004ad0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 shutdown_connections 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x559723721300 0x559723723720 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x559723704510 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723703fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 --2- 172.21.15.60:0/3472717864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x559723703a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 >> 172.21.15.60:0/3472717864 conn(0x5597236fa500 msgr2=0x5597236ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.961+0000 7f4d56ff5640 0 client.15774 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.962+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 shutdown_connections 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.962+0000 7f4d7b513740 1 -- 172.21.15.60:0/3472717864 wait complete. 2024-07-24T21:47:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.962+0000 7f4d7b513740 20 client.15774 trim_cache size 0 max 16384 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.968+0000 7f3c617fa640 1 -- 172.21.15.60:0/2262636278 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557c2c31c70 con 0x5557c2cdecd0 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c617fa640 10 client.15777 handle_command_reply: tid=0 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 client.15777 shutdown 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 msgr2=0x5557c2ce6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2ce6eb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3c740abe30 tx=0x7f3c7400c3f0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 shutdown_connections 2024-07-24T21:47:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5557c2cdecd0 0x5557c2d4b130 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5557c2ccf580 0x5557c2cd7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c2cbf1e0 0x5557c2cd7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 --2- 172.21.15.60:0/2262636278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5557c2cbe8c0 0x5557c2ce6eb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 >> 172.21.15.60:0/2262636278 conn(0x5557c2cbc6f0 msgr2=0x5557c2ccaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c617fa640 0 client.15777 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.969+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 shutdown_connections 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.970+0000 7f3c827f9740 1 -- 172.21.15.60:0/2262636278 wait complete. 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.970+0000 7f3c827f9740 20 client.15777 trim_cache size 0 max 16384 2024-07-24T21:47:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:47:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:47:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:47:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:47:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.970+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 msgr2=0x7f3c640b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.970+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 0x7f3c640b6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005f40 tx=0x7f3c700096a0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 msgr2=0x7f3c78196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 0x7f3c78196a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c760 tx=0x7f3c7400cc20 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 shutdown_connections 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3c640b3dc0 0x7f3c640b6280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c78107910 0x7f3c7819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c78106f90 0x7f3c78196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 --2- 172.21.15.60:0/4274138577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c78101590 0x7f3c78196a20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 >> 172.21.15.60:0/4274138577 conn(0x7f3c780fd010 msgr2=0x7f3c780ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 shutdown_connections 2024-07-24T21:47:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:37.971+0000 7f3c7fa9a640 1 -- 172.21.15.60:0/4274138577 wait complete. 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 Processor -- start 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 -- start start 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597237029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723713910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x5597236ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.990+0000 7f4d7b513740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741ae8a0 con 0x559723711630 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7b513740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741ac170 con 0x559723706a30 2024-07-24T21:47:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7b513740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741a9fc0 con 0x5597236fc6d0 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x5597236ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x5597236ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40752/0 (socket says 172.21.15.60:40752) 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7359e640 1 -- 172.21.15.60:0/924453970 learned_addr learned my addr 172.21.15.60:0/924453970 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723713910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597237029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 -- 172.21.15.60:0/924453970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 msgr2=0x5597237029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597237029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 -- 172.21.15.60:0/924453970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 msgr2=0x5597236ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x5597236ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 -- 172.21.15.60:0/924453970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55972371ff50 con 0x559723706a30 2024-07-24T21:47:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.991+0000 7f4d7259c640 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723713910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800faa0 tx=0x7f4d680abee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad2ce29a06f734f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:37.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.992+0000 7f4d58ff9640 1 -- 172.21.15.60:0/924453970 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680ace70 con 0x559723706a30 2024-07-24T21:47:37.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.992+0000 7f4d58ff9640 1 -- 172.21.15.60:0/924453970 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d68009de0 con 0x559723706a30 2024-07-24T21:47:37.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.992+0000 7f4d58ff9640 1 -- 172.21.15.60:0/924453970 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680cf0d0 con 0x559723706a30 2024-07-24T21:47:37.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- 172.21.15.60:0/924453970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 msgr2=0x559723713910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:37.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723713910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800faa0 tx=0x7f4d680abee0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- 172.21.15.60:0/924453970 shutdown_connections 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559723711630 0x5597236ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723706a30 0x559723713910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 --2- 172.21.15.60:0/924453970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597236fc6d0 0x5597237029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- 172.21.15.60:0/924453970 >> 172.21.15.60:0/924453970 conn(0x5597236fa500 msgr2=0x559723713200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- 172.21.15.60:0/924453970 shutdown_connections 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- 172.21.15.60:0/924453970 wait complete. 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 -- start start 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.993+0000 7f4d7b513740 1 Processor -- start 2024-07-24T21:47:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 0x559723719e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 0x559723728bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723729290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741aa630 con 0x5597236fc6d0 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741abaa0 con 0x559723711630 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d741aad00 con 0x559723706a30 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7b513740 10 client.0 _ll_register_callbacks cb 0x7f4d6c00a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 0x559723728bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 0x559723728bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56592/0 (socket says 172.21.15.60:56592) 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7259c640 1 -- 172.21.15.60:0/2888616633 learned_addr learned my addr 172.21.15.60:0/2888616633 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723729290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 0x559723719e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 -- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 msgr2=0x559723728bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 0x559723728bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 -- 172.21.15.60:0/2888616633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 msgr2=0x559723719e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 0x559723719e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d7359e640 1 -- 172.21.15.60:0/2888616633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972378afc0 con 0x559723711630 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.994+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 0x559723719e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d7359e640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723729290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009690 tx=0x7f4d6c015600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8330517da5ad18e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c036070 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d68009de0 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d6c020990 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d6c042050 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d7b513740 10 client.25337 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55972378c1c0 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.995+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d68009de0 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4d6c020b90 con 0x559723711630 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 10 client.25337 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 10 client.25337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:47:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 20 client.25337 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55972378c8e0 0x55972378ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 4 client.25337 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597236713a0 con 0x55972378c8e0 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55972378c8e0 0x55972378ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d56ff5640 10 client.25337 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.996+0000 7f4d72d9d640 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55972378c8e0 0x55972378ed00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c54111ee50ff539e server_cookie=910a4764af8370de in_seq=0 out_seq=0 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d56ff5640 1 -- 172.21.15.60:0/2888616633 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597236713a0 con 0x55972378c8e0 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d56ff5640 10 client.25337 handle_command_reply: tid=0 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 client.25337 shutdown 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 msgr2=0x559723729290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723729290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009690 tx=0x7f4d6c015600 comp rx=0 tx=0).stop 2024-07-24T21:47:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 shutdown_connections 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55972378c8e0 0x55972378ed00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559723711630 0x559723729290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559723706a30 0x559723728bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 --2- 172.21.15.60:0/2888616633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597236fc6d0 0x559723719e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 >> 172.21.15.60:0/2888616633 conn(0x5597236fa500 msgr2=0x5597236ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d56ff5640 0 client.25337 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 shutdown_connections 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 1 -- 172.21.15.60:0/2888616633 wait complete. 2024-07-24T21:47:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.997+0000 7f4d7b513740 20 client.25337 trim_cache size 0 max 16384 2024-07-24T21:47:38.004 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 msgr2=0x7f4d5c0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 0x7f4d5c0b63d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00a7c0 tx=0x7f4d6c00d040 comp rx=0 tx=0).stop 2024-07-24T21:47:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 msgr2=0x7f4d74198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800bea0 tx=0x7f4d6800bf90 comp rx=0 tx=0).stop 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 shutdown_connections 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d5c0b3f10 0x7f4d5c0b63d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d741079e0 0x7f4d7419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d74107060 0x7f4d74198110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 --2- 172.21.15.60:0/3098828487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d74101650 0x7f4d74197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.998+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 >> 172.21.15.60:0/3098828487 conn(0x7f4d740fd0c0 msgr2=0x7f4d740fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.999+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 shutdown_connections 2024-07-24T21:47:38.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:37.999+0000 7f4d79162640 1 -- 172.21.15.60:0/3098828487 wait complete. 2024-07-24T21:47:38.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:47:39.055 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:39.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:39 smithi060 ceph-mon[33000]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-07-24T21:47:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:39 smithi086 ceph-mon[37128]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-07-24T21:47:39.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:39 smithi123 ceph-mon[37143]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-07-24T21:47:39.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:47:41.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:41 smithi086 ceph-mon[37128]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:41.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:41 smithi060 ceph-mon[33000]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:41 smithi123 ceph-mon[37143]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T21:47:43.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:43 smithi060 ceph-mon[33000]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-07-24T21:47:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:43 smithi086 ceph-mon[37128]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-07-24T21:47:43.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:43 smithi123 ceph-mon[37143]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-07-24T21:47:43.940 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 -- 172.21.15.60:0/4207404209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 msgr2=0x7f53cc108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 --2- 172.21.15.60:0/4207404209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc108600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009900 tx=0x7f53bc02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 -- 172.21.15.60:0/4207404209 shutdown_connections 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 --2- 172.21.15.60:0/4207404209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 0x7f53cc10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 --2- 172.21.15.60:0/4207404209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc108600 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 --2- 172.21.15.60:0/4207404209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 0x7f53cc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 -- 172.21.15.60:0/4207404209 >> 172.21.15.60:0/4207404209 conn(0x7f53cc0fe200 msgr2=0x7f53cc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 -- 172.21.15.60:0/4207404209 shutdown_connections 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.149+0000 7f53d335b640 1 -- 172.21.15.60:0/4207404209 wait complete. 2024-07-24T21:47:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.150+0000 7f53d335b640 1 Processor -- start 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.150+0000 7f53d335b640 1 -- start start 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 0x7f53cc074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 0x7f53cc075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40786/0 (socket says 172.21.15.60:40786) 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 -- 172.21.15.60:0/3461967178 learned_addr learned my addr 172.21.15.60:0/3461967178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc112770 con 0x7f53cc1081c0 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1125f0 con 0x7f53cc108b40 2024-07-24T21:47:44.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1128f0 con 0x7f53cc1027b0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 0x7f53cc074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 0x7f53cc075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 -- 172.21.15.60:0/3461967178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 msgr2=0x7f53cc074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 0x7f53cc074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 -- 172.21.15.60:0/3461967178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 msgr2=0x7f53cc075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 0x7f53cc075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 -- 172.21.15.60:0/3461967178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc1a7080 con 0x7f53cc1081c0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.151+0000 7f53d08f6640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005940 tx=0x7f53bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.152+0000 7f53c27fc640 1 -- 172.21.15.60:0/3461967178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc03f030 con 0x7f53cc1081c0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.152+0000 7f53c27fc640 1 -- 172.21.15.60:0/3461967178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc003cb0 con 0x7f53cc1081c0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.152+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc1a7310 con 0x7f53cc1081c0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.152+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1a7840 con 0x7f53cc1081c0 2024-07-24T21:47:44.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.152+0000 7f53c27fc640 1 -- 172.21.15.60:0/3461967178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc035070 con 0x7f53cc1081c0 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.154+0000 7f53c27fc640 1 -- 172.21.15.60:0/3461967178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f53bc042030 con 0x7f53cc1081c0 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 Processor -- start 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 -- start start 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c97546ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 0x560c97552a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 0x560c97555370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c974bd9b0 con 0x560c97552f80 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c974aceb0 con 0x560c975466d0 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.155+0000 7f53d570c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560c974b0ef0 con 0x560c97546ff0 2024-07-24T21:47:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 0x560c97555370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 0x560c97555370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40790/0 (socket says 172.21.15.60:40790) 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d08f6640 1 -- 172.21.15.60:0/533186388 learned_addr learned my addr 172.21.15.60:0/533186388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d18f8640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c97546ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 0x560c97552a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 -- 172.21.15.60:0/533186388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 msgr2=0x560c97546ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c97546ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 -- 172.21.15.60:0/533186388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 msgr2=0x560c97555370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 0x560c97555370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 -- 172.21.15.60:0/533186388 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c97436320 con 0x560c97546ff0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d18f8640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c97546ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53d10f7640 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 0x560c97552a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53c8004630 tx=0x7f53c800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3038cddd01fbb1c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53c27fc640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 0x7f53b80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.156+0000 7f53b77fe640 1 -- 172.21.15.60:0/533186388 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003bd0 con 0x560c97546ff0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53b77fe640 1 -- 172.21.15.60:0/533186388 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c8015070 con 0x560c97546ff0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53b77fe640 1 -- 172.21.15.60:0/533186388 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8004c80 con 0x560c97546ff0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 0x7f53b80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53c27fc640 1 -- 172.21.15.60:0/3461967178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f53bc1044c0 con 0x7f53cc1081c0 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 0x7f53b80b6280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53cc076f50 tx=0x7f53c40096e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53d570c740 1 -- 172.21.15.60:0/533186388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 msgr2=0x560c97552a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53d570c740 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 0x560c97552a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f53c8004630 tx=0x7f53c800bb20 comp rx=0 tx=0).stop 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.157+0000 7f53d570c740 1 -- 172.21.15.60:0/533186388 shutdown_connections 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97552f80 0x560c97555370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97546ff0 0x560c97552a40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 --2- 172.21.15.60:0/533186388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c97546ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 -- 172.21.15.60:0/533186388 >> 172.21.15.60:0/533186388 conn(0x560c97544500 msgr2=0x560c975456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 -- 172.21.15.60:0/533186388 shutdown_connections 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 -- 172.21.15.60:0/533186388 wait complete. 2024-07-24T21:47:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 -- start start 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.158+0000 7f53d570c740 1 Processor -- start 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c97550970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c97310700 con 0x560c97546ff0 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c9744f5f0 con 0x560c975466d0 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560c97555c00 con 0x560c97552f80 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40800/0 (socket says 172.21.15.60:40800) 2024-07-24T21:47:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 -- 172.21.15.60:0/4163311040 learned_addr learned my addr 172.21.15.60:0/4163311040 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d10f7640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d18f8640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c97550970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 -- 172.21.15.60:0/4163311040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 msgr2=0x560c97550970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c97550970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 -- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 msgr2=0x560c9754fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d08f6640 1 -- 172.21.15.60:0/4163311040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c97442680 con 0x560c97546ff0 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d10f7640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 10 client.0 _ll_register_callbacks cb 0x560c9746f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.159+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c8015070 con 0x560c97546ff0 2024-07-24T21:47:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53d08f6640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f53bc101cc0 tx=0x7f53bc0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5490ff11a4710ce6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53b4ff9640 10 client.15792 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53b4ff9640 1 -- 172.21.15.60:0/4163311040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc049040 con 0x560c97546ff0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53b4ff9640 1 -- 172.21.15.60:0/4163311040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f53bc03bbf0 con 0x560c97546ff0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53b4ff9640 1 -- 172.21.15.60:0/4163311040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53bc0b2040 con 0x560c97546ff0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53d570c740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.160+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560c9720d650 con 0x560c97546ff0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53b4ff9640 1 -- 172.21.15.60:0/4163311040 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f53bc0cb330 con 0x560c97546ff0 2024-07-24T21:47:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 20 client.15792 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c9756b010 0x560c9756d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 4 client.15792 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c974bf900 con 0x560c9756b010 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.161+0000 7f53d10f7640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c9756b010 0x560c9756d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.162+0000 7f53d10f7640 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c9756b010 0x560c9756d4a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b04572fafdec6f8e server_cookie=526a4a82b6dbee01 in_seq=0 out_seq=0 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.162+0000 7f53b4ff9640 10 client.15792 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.162+0000 7f53b4ff9640 1 -- 172.21.15.60:0/4163311040 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c974bf900 con 0x560c9756b010 2024-07-24T21:47:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.162+0000 7f53b4ff9640 10 client.15792 handle_command_reply: tid=0 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.162+0000 7f53d570c740 1 client.15792 shutdown 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 msgr2=0x560c97550430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f53bc101cc0 tx=0x7f53bc0092f0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 shutdown_connections 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c9756b010 0x560c9756d4a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c97550970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97550430 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 --2- 172.21.15.60:0/4163311040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 >> 172.21.15.60:0/4163311040 conn(0x560c97544500 msgr2=0x560c9754a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53b4ff9640 0 client.15792 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 shutdown_connections 2024-07-24T21:47:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 1 -- 172.21.15.60:0/4163311040 wait complete. 2024-07-24T21:47:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.163+0000 7f53d570c740 20 client.15792 trim_cache size 0 max 16384 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.190+0000 7f53d570c740 1 Processor -- start 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.190+0000 7f53d570c740 1 -- start start 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97572d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c975732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ae720 con 0x560c97546ff0 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1abff0 con 0x560c975466d0 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d570c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1a9e40 con 0x560c97552f80 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97572d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d08f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97572d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40804/0 (socket says 172.21.15.60:40804) 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d08f6640 1 -- 172.21.15.60:0/3305008449 learned_addr learned my addr 172.21.15.60:0/3305008449 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c975732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 -- 172.21.15.60:0/3305008449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 msgr2=0x560c975732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c975732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 -- 172.21.15.60:0/3305008449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 msgr2=0x560c97572d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97572d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.191+0000 7f53d10f7640 1 -- 172.21.15.60:0/3305008449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c975451e0 con 0x560c975466d0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.192+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754a820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005cb0 tx=0x7f53c8005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c19eea284a4848 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.192+0000 7f53b6ffd640 1 -- 172.21.15.60:0/3305008449 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003e50 con 0x560c975466d0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.192+0000 7f53b6ffd640 1 -- 172.21.15.60:0/3305008449 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c800be90 con 0x560c975466d0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.192+0000 7f53b6ffd640 1 -- 172.21.15.60:0/3305008449 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8019680 con 0x560c975466d0 2024-07-24T21:47:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 -- 172.21.15.60:0/3305008449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 msgr2=0x560c9754a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754a820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005cb0 tx=0x7f53c8005c40 comp rx=0 tx=0).stop 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 -- 172.21.15.60:0/3305008449 shutdown_connections 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c975732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97572d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 --2- 172.21.15.60:0/3305008449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c9754a820 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.193+0000 7f53d570c740 1 -- 172.21.15.60:0/3305008449 >> 172.21.15.60:0/3305008449 conn(0x560c97544500 msgr2=0x560c97551d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.194+0000 7f53d570c740 1 -- 172.21.15.60:0/3305008449 shutdown_connections 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.194+0000 7f53d570c740 1 -- 172.21.15.60:0/3305008449 wait complete. 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.194+0000 7f53d570c740 1 -- start start 2024-07-24T21:47:44.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.194+0000 7f53d570c740 1 Processor -- start 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c975566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97556c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1aa4b0 con 0x560c97546ff0 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ab920 con 0x560c975466d0 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1aab80 con 0x560c97552f80 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.195+0000 7f53d570c740 10 client.0 _ll_register_callbacks cb 0x7f53bc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d10f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c975566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56622/0 (socket says 172.21.15.60:56622) 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 -- 172.21.15.60:0/3548273272 learned_addr learned my addr 172.21.15.60:0/3548273272 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d08f6640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97556c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 -- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 msgr2=0x560c975566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c975566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 -- 172.21.15.60:0/3548273272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 msgr2=0x560c97556c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97556c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 -- 172.21.15.60:0/3548273272 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9755b300 con 0x560c97552f80 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53d18f8640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53c4012ab0 tx=0x7f53c4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab63550d5f7c03 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:44.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.196+0000 7f53b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4020cc0 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c800be90 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f53c401f420 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53c4020e60 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53d570c740 10 client.25336 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c975d58e0 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.197+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c401fad0 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f53c400f040 con 0x560c97552f80 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 10 client.25336 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 10 client.25336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 20 client.25336 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c975d6080 0x560c975d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:44.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 4 client.25336 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f53c800be90 con 0x560c975d6080 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.198+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c975d6080 0x560c975d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d10f7640 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c975d6080 0x560c975d84a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c1477ec374a2553 server_cookie=698381df4b7a5bef in_seq=0 out_seq=0 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53b4ff9640 10 client.25336 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53b4ff9640 1 -- 172.21.15.60:0/3548273272 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f53c800be90 con 0x560c975d6080 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53b4ff9640 10 client.25336 handle_command_reply: tid=0 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 client.25336 shutdown 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 msgr2=0x560c9755abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53c4012ab0 tx=0x7f53c4004a40 comp rx=0 tx=0).stop 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 shutdown_connections 2024-07-24T21:47:44.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x560c975d6080 0x560c975d84a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c97552f80 0x560c9755abc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c97546ff0 0x560c97556c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 --2- 172.21.15.60:0/3548273272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c975466d0 0x560c975566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.199+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 >> 172.21.15.60:0/3548273272 conn(0x560c97544500 msgr2=0x560c9754fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.200+0000 7f53b4ff9640 0 client.25336 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.200+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 shutdown_connections 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.200+0000 7f53d570c740 1 -- 172.21.15.60:0/3548273272 wait complete. 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.200+0000 7f53d570c740 20 client.25336 trim_cache size 0 max 16384 2024-07-24T21:47:44.211 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 msgr2=0x7f53b80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 0x7f53b80b6280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53cc076f50 tx=0x7f53c40096e0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 msgr2=0x7f53cc0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005940 tx=0x7f53bc0042f0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 shutdown_connections 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53b80b3dc0 0x7f53b80b6280 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53cc108b40 0x7f53cc075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53cc1081c0 0x7f53cc0753d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 --2- 172.21.15.60:0/3461967178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53cc1027b0 0x7f53cc074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.201+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 >> 172.21.15.60:0/3461967178 conn(0x7f53cc0fe200 msgr2=0x7f53cc1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.202+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 shutdown_connections 2024-07-24T21:47:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:44.202+0000 7f53d335b640 1 -- 172.21.15.60:0/3461967178 wait complete. 2024-07-24T21:47:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:45 smithi086 ceph-mon[37128]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:45.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:45 smithi060 ceph-mon[33000]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:45.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:47:45.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:45 smithi123 ceph-mon[37143]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.788+0000 7f64227fc640 1 -- 172.21.15.60:0/1494338321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180318a0 con 0x7f6424107060 2024-07-24T21:47:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 -- 172.21.15.60:0/1494338321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424107060 msgr2=0x7f64241074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 --2- 172.21.15.60:0/1494338321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424107060 0x7f64241074a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6418009900 tx=0x7f641802a3f0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 -- 172.21.15.60:0/1494338321 shutdown_connections 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 --2- 172.21.15.60:0/1494338321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f642410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 --2- 172.21.15.60:0/1494338321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424107060 0x7f64241074a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 --2- 172.21.15.60:0/1494338321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6424101650 0x7f6424101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 -- 172.21.15.60:0/1494338321 >> 172.21.15.60:0/1494338321 conn(0x7f64240fd0c0 msgr2=0x7f64240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 -- 172.21.15.60:0/1494338321 shutdown_connections 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.789+0000 7f6429eed640 1 -- 172.21.15.60:0/1494338321 wait complete. 2024-07-24T21:47:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.790+0000 7f6429eed640 1 Processor -- start 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.790+0000 7f6429eed640 1 -- start start 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424101650 0x7f642410d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6424107060 0x7f642410db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6424112760 con 0x7f6424101650 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64241125e0 con 0x7f6424107060 2024-07-24T21:47:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6429eed640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64241128e0 con 0x7f64241079e0 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55076/0 (socket says 172.21.15.60:55076) 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 -- 172.21.15.60:0/2670190191 learned_addr learned my addr 172.21.15.60:0/2670190191 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 -- 172.21.15.60:0/2670190191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6424107060 msgr2=0x7f642410db20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6424107060 0x7f642410db20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 -- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424101650 msgr2=0x7f642410d5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424101650 0x7f642410d5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.791+0000 7f6423fff640 1 -- 172.21.15.60:0/2670190191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6424074650 con 0x7f64241079e0 2024-07-24T21:47:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.792+0000 7f6423fff640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.793+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c017070 con 0x7f64241079e0 2024-07-24T21:47:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.793+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6424074940 con 0x7f64241079e0 2024-07-24T21:47:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.793+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6424070f20 con 0x7f64241079e0 2024-07-24T21:47:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.794+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641c003e20 con 0x7f64241079e0 2024-07-24T21:47:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.795+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0106c0 con 0x7f64241079e0 2024-07-24T21:47:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.796+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6410005290 con 0x7f64241079e0 2024-07-24T21:47:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.798+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f641c003970 con 0x7f64241079e0 2024-07-24T21:47:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.800+0000 7f6420ff9640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 0x7f640c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.800+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f641c0e4030 con 0x7f64241079e0 2024-07-24T21:47:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.802+0000 7f64237fe640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 0x7f640c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.802+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f641c0047e0 con 0x7f64241079e0 2024-07-24T21:47:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:46.803+0000 7f64237fe640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 0x7f640c0b6350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f641400b990 tx=0x7f641400b370 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:47.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.199+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6410005580 con 0x7f64241079e0 2024-07-24T21:47:47.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.201+0000 7f6420ff9640 1 -- 172.21.15.60:0/2670190191 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f641c0ab440 con 0x7f64241079e0 2024-07-24T21:47:47.202 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:47:47.202 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:47:47.202 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:47:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 msgr2=0x7f640c0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 0x7f640c0b6350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f641400b990 tx=0x7f641400b370 comp rx=0 tx=0).stop 2024-07-24T21:47:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 msgr2=0x7f6424073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c00b260 comp rx=0 tx=0).stop 2024-07-24T21:47:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 shutdown_connections 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f640c0b3e90 0x7f640c0b6350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64241079e0 0x7f6424073ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6424107060 0x7f642410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 --2- 172.21.15.60:0/2670190191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6424101650 0x7f642410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 >> 172.21.15.60:0/2670190191 conn(0x7f64240fd0c0 msgr2=0x7f64241004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.204+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 shutdown_connections 2024-07-24T21:47:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:47.205+0000 7f6429eed640 1 -- 172.21.15.60:0/2670190191 wait complete. 2024-07-24T21:47:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:47 smithi086 ceph-mon[37128]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-24T21:47:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:47 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2670190191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:47.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:47 smithi060 ceph-mon[33000]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-24T21:47:47.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:47 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2670190191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:47 smithi123 ceph-mon[37143]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-24T21:47:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:47 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2670190191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:47.984 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 98c11f33 --release --await' 2024-07-24T21:47:47.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 98c11f33 --release --await 2024-07-24T21:47:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:49 smithi086 ceph-mon[37128]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-24T21:47:49.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:49 smithi060 ceph-mon[33000]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-24T21:47:49.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:49 smithi123 ceph-mon[37143]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 -- 172.21.15.60:0/1931219814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 msgr2=0x7fea981074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1931219814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea981074a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009900 tx=0x7fea8c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 -- 172.21.15.60:0/1931219814 shutdown_connections 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1931219814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea981079e0 0x7fea9810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1931219814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea981074a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1931219814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 0x7fea98101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.968+0000 7fea9d48b640 1 -- 172.21.15.60:0/1931219814 >> 172.21.15.60:0/1931219814 conn(0x7fea980fd0c0 msgr2=0x7fea980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.969+0000 7fea9d48b640 1 -- 172.21.15.60:0/1931219814 shutdown_connections 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.969+0000 7fea9d48b640 1 -- 172.21.15.60:0/1931219814 wait complete. 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea9d48b640 1 Processor -- start 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea9d48b640 1 -- start start 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea9d48b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 0x7fea98199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea9d48b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea9d48b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea981079e0 0x7fea981a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51572/0 (socket says 172.21.15.60:51572) 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.970+0000 7fea967fc640 1 -- 172.21.15.60:0/1116876561 learned_addr learned my addr 172.21.15.60:0/1116876561 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea98111630 con 0x7fea98107060 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea981114b0 con 0x7fea981079e0 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea981117b0 con 0x7fea98101650 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 0x7fea98199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 -- 172.21.15.60:0/1116876561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 msgr2=0x7fea98199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 0x7fea98199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 -- 172.21.15.60:0/1116876561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea981079e0 msgr2=0x7fea981a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea981079e0 0x7fea981a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea981a4f20 con 0x7fea98107060 2024-07-24T21:47:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.971+0000 7fea967fc640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005ab0 tx=0x7fea8c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.972+0000 7fea7ffff640 1 -- 172.21.15.60:0/1116876561 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c03f070 con 0x7fea98107060 2024-07-24T21:47:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.972+0000 7fea7ffff640 1 -- 172.21.15.60:0/1116876561 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c031b50 con 0x7fea98107060 2024-07-24T21:47:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.972+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea981a50f0 con 0x7fea98107060 2024-07-24T21:47:49.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.972+0000 7fea7ffff640 1 -- 172.21.15.60:0/1116876561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c035030 con 0x7fea98107060 2024-07-24T21:47:49.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.972+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea981a54f0 con 0x7fea98107060 2024-07-24T21:47:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 Processor -- start 2024-07-24T21:47:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 -- start start 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 0x555baf1bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555baf1349b0 con 0x555baf1ce370 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555baf123eb0 con 0x555baf1bd6b0 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.976+0000 7fea9f83c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555baf127ef0 con 0x555baf1bdfd0 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 0x555baf1bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 0x555baf1bda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51398/0 (socket says 172.21.15.60:51398) 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea977fe640 1 -- 172.21.15.60:0/149918877 learned_addr learned my addr 172.21.15.60:0/149918877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 -- 172.21.15.60:0/149918877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 msgr2=0x555baf1bda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 0x555baf1bda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 -- 172.21.15.60:0/149918877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 msgr2=0x555baf1d0760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:47:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1d0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 -- 172.21.15.60:0/149918877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baf0ad320 con 0x555baf1bdfd0 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea96ffd640 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1cde30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cbd0 tx=0x7fea8800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e17c0dbdaa16c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.977+0000 7fea7d7fa640 1 -- 172.21.15.60:0/149918877 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88004230 con 0x555baf1bdfd0 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.978+0000 7fea7ffff640 1 -- 172.21.15.60:0/1116876561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fea8c042070 con 0x7fea98107060 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.978+0000 7fea7d7fa640 1 -- 172.21.15.60:0/149918877 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea880043d0 con 0x555baf1bdfd0 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.978+0000 7fea7d7fa640 1 -- 172.21.15.60:0/149918877 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88004d10 con 0x555baf1bdfd0 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 -- 172.21.15.60:0/149918877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 msgr2=0x555baf1cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1cde30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cbd0 tx=0x7fea8800e720 comp rx=0 tx=0).stop 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 -- 172.21.15.60:0/149918877 shutdown_connections 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1cde30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 --2- 172.21.15.60:0/149918877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bd6b0 0x555baf1bda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 -- 172.21.15.60:0/149918877 >> 172.21.15.60:0/149918877 conn(0x555baf1bb4e0 msgr2=0x555baf1bc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea7ffff640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 0x7fea800b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea9f83c740 1 -- 172.21.15.60:0/149918877 shutdown_connections 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea7ffff640 1 -- 172.21.15.60:0/1116876561 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fea8c104570 con 0x7fea98107060 2024-07-24T21:47:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.979+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 0x7fea800b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 -- 172.21.15.60:0/149918877 wait complete. 2024-07-24T21:47:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 -- start start 2024-07-24T21:47:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 Processor -- start 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 0x7fea800b6330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cba0 tx=0x7fea8800ced0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bdfd0 0x555baf1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.980+0000 7fea9f83c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555baef87700 con 0x555baf1ce370 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea9f83c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555baf0c65f0 con 0x555baf1bdfd0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea9f83c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555baf1d1020 con 0x555baf1bd6b0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea9f83c740 10 client.0 _ll_register_callbacks cb 0x555baf0e6960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55124/0 (socket says 172.21.15.60:55124) 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 -- 172.21.15.60:0/1150693401 learned_addr learned my addr 172.21.15.60:0/1150693401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea967fc640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bdfd0 0x555baf1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea977fe640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 -- 172.21.15.60:0/1150693401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bdfd0 msgr2=0x555baf1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bdfd0 0x555baf1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 -- 172.21.15.60:0/1150693401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 msgr2=0x555baf1ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea96ffd640 1 -- 172.21.15.60:0/1150693401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baf0b9680 con 0x555baf1bd6b0 2024-07-24T21:47:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.981+0000 7fea977fe640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea88017000 tx=0x7fea8800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23edb5db06f058d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88010700 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea880043d0 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fea880108a0 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea88010aa0 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea9f83c740 10 client.25351 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555baee84650 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.982+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea880043d0 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fea88033480 con 0x555baf1bd6b0 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 10 client.25351 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 10 client.25351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 20 client.25351 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf1e2300 0x555baf1e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 4 client.25351 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:49.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555baf136900 con 0x555baf1e2300 2024-07-24T21:47:49.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.983+0000 7fea967fc640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf1e2300 0x555baf1e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:49.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.984+0000 7fea967fc640 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf1e2300 0x555baf1e46f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845e14d8e7df2366 server_cookie=60f3aa10b679d68 in_seq=0 out_seq=0 2024-07-24T21:47:49.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.984+0000 7fea7aff5640 10 client.25351 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.984+0000 7fea7aff5640 1 -- 172.21.15.60:0/1150693401 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555baf136900 con 0x555baf1e2300 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.984+0000 7fea7aff5640 10 client.25351 handle_command_reply: tid=0 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.984+0000 7fea9f83c740 1 client.25351 shutdown 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 msgr2=0x555baf1cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea88017000 tx=0x7fea8800e720 comp rx=0 tx=0).stop 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 shutdown_connections 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf1e2300 0x555baf1e46f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1ce370 0x555baf1ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1bdfd0 0x555baf1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1150693401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bd6b0 0x555baf1cdbd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 >> 172.21.15.60:0/1150693401 conn(0x555baf1bb4e0 msgr2=0x555baf1c1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea7aff5640 0 client.25351 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 shutdown_connections 2024-07-24T21:47:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 1 -- 172.21.15.60:0/1150693401 wait complete. 2024-07-24T21:47:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:49.985+0000 7fea9f83c740 20 client.25351 trim_cache size 0 max 16384 2024-07-24T21:47:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.013+0000 7fea9f83c740 1 Processor -- start 2024-07-24T21:47:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.013+0000 7fea9f83c740 1 -- start start 2024-07-24T21:47:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 0x555baf1c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1bc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea981ae760 con 0x555baf1bdfd0 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea981ac030 con 0x555baf1e4a20 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea9f83c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea981a9ea0 con 0x555baf1ce370 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea967fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea967fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55132/0 (socket says 172.21.15.60:55132) 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea967fc640 1 -- 172.21.15.60:0/2496621573 learned_addr learned my addr 172.21.15.60:0/2496621573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.014+0000 7fea977fe640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 -- 172.21.15.60:0/2496621573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 msgr2=0x555baf1bc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1bc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 -- 172.21.15.60:0/2496621573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 msgr2=0x555baf1c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea96ffd640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 0x555baf1c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 0x555baf1c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 -- 172.21.15.60:0/2496621573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baf1bde00 con 0x555baf1ce370 2024-07-24T21:47:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea96ffd640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 0x555baf1c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.015+0000 7fea967fc640 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fea8c004650 tx=0x7fea8c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1674f11255657ab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.016+0000 7fea7cff9640 1 -- 172.21.15.60:0/2496621573 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c049050 con 0x555baf1ce370 2024-07-24T21:47:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.016+0000 7fea7cff9640 1 -- 172.21.15.60:0/2496621573 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c0cf720 con 0x555baf1ce370 2024-07-24T21:47:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.016+0000 7fea7cff9640 1 -- 172.21.15.60:0/2496621573 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c101ee0 con 0x555baf1ce370 2024-07-24T21:47:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 -- 172.21.15.60:0/2496621573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 msgr2=0x555baf1bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fea8c004650 tx=0x7fea8c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 -- 172.21.15.60:0/2496621573 shutdown_connections 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1bc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1ce370 0x555baf1bc1c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 --2- 172.21.15.60:0/2496621573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1bdfd0 0x555baf1c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 -- 172.21.15.60:0/2496621573 >> 172.21.15.60:0/2496621573 conn(0x555baf1bb4e0 msgr2=0x555baf1e4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 -- 172.21.15.60:0/2496621573 shutdown_connections 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.017+0000 7fea9f83c740 1 -- 172.21.15.60:0/2496621573 wait complete. 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 -- start start 2024-07-24T21:47:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 Processor -- start 2024-07-24T21:47:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea981aa4f0 con 0x555baf1df5c0 2024-07-24T21:47:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea981ab960 con 0x555baf1e4a20 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea981aabc0 con 0x555baf1bdfd0 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea9f83c740 10 client.0 _ll_register_callbacks cb 0x7fea88014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51596/0 (socket says 172.21.15.60:51596) 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.018+0000 7fea977fe640 1 -- 172.21.15.60:0/1715667005 learned_addr learned my addr 172.21.15.60:0/1715667005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 -- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 msgr2=0x555baf1deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 -- 172.21.15.60:0/1715667005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 msgr2=0x555baf1df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 -- 172.21.15.60:0/1715667005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baf1dc050 con 0x555baf1df5c0 2024-07-24T21:47:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea977fe640 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fea90010ce0 tx=0x7fea9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6962736ddb037d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.019+0000 7fea7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90004d20 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fea9001c3f0 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c0cf720 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea9001c630 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea9f83c740 10 client.15813 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555baf24d5f0 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.020+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9001cc80 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.021+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fea9001b070 con 0x555baf1df5c0 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.021+0000 7fea9f83c740 10 client.15813 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.021+0000 7fea9f83c740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.021+0000 7fea9f83c740 20 client.15813 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.021+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf24dda0 0x555baf2501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.022+0000 7fea9f83c740 4 client.15813 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:47:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.022+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fea8c0cf720 con 0x555baf24dda0 2024-07-24T21:47:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.022+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf24dda0 0x555baf2501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.022+0000 7fea7aff5640 10 client.15813 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.022+0000 7fea96ffd640 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf24dda0 0x555baf2501e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0ee2ba7a55c780 server_cookie=1c53d2d2151d070f in_seq=0 out_seq=0 2024-07-24T21:47:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea7aff5640 1 -- 172.21.15.60:0/1715667005 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fea8c0cf720 con 0x555baf24dda0 2024-07-24T21:47:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea7aff5640 10 client.15813 handle_command_reply: tid=0 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 client.15813 shutdown 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 msgr2=0x555baf1dbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fea90010ce0 tx=0x7fea9000f790 comp rx=0 tx=0).stop 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 shutdown_connections 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555baf24dda0 0x555baf2501e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555baf1df5c0 0x555baf1dbb10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555baf1e4a20 0x555baf1df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 --2- 172.21.15.60:0/1715667005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555baf1bdfd0 0x555baf1deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 >> 172.21.15.60:0/1715667005 conn(0x555baf1bb4e0 msgr2=0x555baf1b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea7aff5640 0 client.15813 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 shutdown_connections 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 1 -- 172.21.15.60:0/1715667005 wait complete. 2024-07-24T21:47:50.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.023+0000 7fea9f83c740 20 client.15813 trim_cache size 0 max 16384 2024-07-24T21:47:50.029 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:47:50.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 msgr2=0x7fea800b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 0x7fea800b6330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cba0 tx=0x7fea8800ced0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 msgr2=0x7fea98199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005ab0 tx=0x7fea8c003c20 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 shutdown_connections 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fea800b3e70 0x7fea800b6330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea981079e0 0x7fea981a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea98107060 0x7fea98199ea0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 --2- 172.21.15.60:0/1116876561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea98101650 0x7fea98199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.024+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 >> 172.21.15.60:0/1116876561 conn(0x7fea980fd0c0 msgr2=0x7fea980ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.025+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 shutdown_connections 2024-07-24T21:47:50.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:50.025+0000 7fea9d48b640 1 -- 172.21.15.60:0/1116876561 wait complete. 2024-07-24T21:47:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 -- 172.21.15.60:0/2773369798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c741015e0 msgr2=0x7f5c741019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/2773369798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c741015e0 0x7f5c741019c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009900 tx=0x7f5c6402a410 comp rx=0 tx=0).stop 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 -- 172.21.15.60:0/2773369798 shutdown_connections 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/2773369798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74107910 0x7f5c7410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/2773369798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c74106f90 0x7f5c741073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/2773369798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c741015e0 0x7f5c741019c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 -- 172.21.15.60:0/2773369798 >> 172.21.15.60:0/2773369798 conn(0x7f5c740782e0 msgr2=0x7f5c740ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 -- 172.21.15.60:0/2773369798 shutdown_connections 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.111+0000 7f5c7a10c640 1 -- 172.21.15.60:0/2773369798 wait complete. 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 Processor -- start 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 -- start start 2024-07-24T21:47:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741015e0 0x7f5c7410ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74106f90 0x7f5c7410f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741126f0 con 0x7f5c74107910 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74112570 con 0x7f5c741015e0 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c7a10c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74112870 con 0x7f5c74106f90 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51614/0 (socket says 172.21.15.60:51614) 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c73fff640 1 -- 172.21.15.60:0/965689872 learned_addr learned my addr 172.21.15.60:0/965689872 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.112+0000 7f5c737fe640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741015e0 0x7f5c7410ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74106f90 0x7f5c7410f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 -- 172.21.15.60:0/965689872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74106f90 msgr2=0x7f5c7410f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74106f90 0x7f5c7410f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 -- 172.21.15.60:0/965689872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741015e0 msgr2=0x7f5c7410ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741015e0 0x7f5c7410ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 -- 172.21.15.60:0/965689872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c741a6fc0 con 0x7f5c74107910 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c73fff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00eb10 tx=0x7f5c6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c717fa640 1 -- 172.21.15.60:0/965689872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004b20 con 0x7f5c74107910 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c717fa640 1 -- 172.21.15.60:0/965689872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c004cc0 con 0x7f5c74107910 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.113+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c741a72b0 con 0x7f5c74107910 2024-07-24T21:47:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.114+0000 7f5c717fa640 1 -- 172.21.15.60:0/965689872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c010670 con 0x7f5c74107910 2024-07-24T21:47:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.114+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741a77f0 con 0x7f5c74107910 2024-07-24T21:47:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.117+0000 7f5c7ce67740 1 Processor -- start 2024-07-24T21:47:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.118+0000 7f5c717fa640 1 -- 172.21.15.60:0/965689872 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5c6c005020 con 0x7f5c74107910 2024-07-24T21:47:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.118+0000 7f5c7ce67740 1 -- start start 2024-07-24T21:47:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.118+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.118+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a2150c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.118+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a2153560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51626/0 (socket says 172.21.15.60:51626) 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c737fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a2150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c717fa640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 0x7f5c600b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c737fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a2150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55158/0 (socket says 172.21.15.60:55158) 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 -- 172.21.15.60:0/1593331236 learned_addr learned my addr 172.21.15.60:0/1593331236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5555a20ba280 con 0x5555a21448c0 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555a20a9780 con 0x5555a2151170 2024-07-24T21:47:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555a20ad7c0 con 0x5555a21451e0 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c717fa640 1 -- 172.21.15.60:0/965689872 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5c6c0e44d0 con 0x7f5c74107910 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 -- 172.21.15.60:0/1593331236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 msgr2=0x5555a2150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a2150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 -- 172.21.15.60:0/1593331236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 msgr2=0x5555a2153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a2153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 -- 172.21.15.60:0/1593331236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a1e0d790 con 0x5555a21448c0 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c73fff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 0x7f5c600b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.119+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009560 tx=0x7f5c5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c0b413a5ddd65a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.120+0000 7f5c69ffb640 1 -- 172.21.15.60:0/1593331236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01e070 con 0x5555a21448c0 2024-07-24T21:47:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.120+0000 7f5c69ffb640 1 -- 172.21.15.60:0/1593331236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c004250 con 0x5555a21448c0 2024-07-24T21:47:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.120+0000 7f5c69ffb640 1 -- 172.21.15.60:0/1593331236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c019690 con 0x5555a21448c0 2024-07-24T21:47:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 msgr2=0x5555a2144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009560 tx=0x7f5c5c011810 comp rx=0 tx=0).stop 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c73fff640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 0x7f5c600b6280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0e1cd0 tx=0x7f5c6c00cf70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 shutdown_connections 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a2153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a2150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/1593331236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a2144ca0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 >> 172.21.15.60:0/1593331236 conn(0x5555a21426f0 msgr2=0x5555a2143890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 shutdown_connections 2024-07-24T21:47:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.121+0000 7f5c7ce67740 1 -- 172.21.15.60:0/1593331236 wait complete. 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.122+0000 7f5c7ce67740 1 -- start start 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.122+0000 7f5c7ce67740 1 Processor -- start 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a214de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51638/0 (socket says 172.21.15.60:51638) 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 -- 172.21.15.60:0/398541198 learned_addr learned my addr 172.21.15.60:0/398541198 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a214e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5555a1fe3ab0 con 0x5555a21448c0 2024-07-24T21:47:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555a20319e0 con 0x5555a2151170 2024-07-24T21:47:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5555a2153e50 con 0x5555a21451e0 2024-07-24T21:47:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c737fe640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a214de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 -- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 msgr2=0x5555a214de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a214de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 -- 172.21.15.60:0/398541198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 msgr2=0x5555a214e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a214e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.123+0000 7f5c6bfff640 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a1e86620 con 0x5555a21448c0 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005db0 tx=0x7f5c5c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24f4dfc3fd7d6a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c7ce67740 10 client.0 _ll_register_callbacks cb 0x5555a1db9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c52ffd640 1 -- 172.21.15.60:0/398541198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01e040 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c52ffd640 1 -- 172.21.15.60:0/398541198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5c5c0052f0 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555a2058ce0 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c52ffd640 1 -- 172.21.15.60:0/398541198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c031050 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c7ce67740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.124+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5555a215f2d0 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c52ffd640 1 -- 172.21.15.60:0/398541198 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5c5c0035f0 con 0x5555a21448c0 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 20 client.15825 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21606c0 0x5555a2162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 4 client.15825 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555a20bc1d0 con 0x5555a21606c0 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.125+0000 7f5c737fe640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21606c0 0x5555a2162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.126+0000 7f5c737fe640 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21606c0 0x5555a2162ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a127fd620d552d server_cookie=7eeffd0dffdf1a7f in_seq=0 out_seq=0 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.126+0000 7f5c52ffd640 10 client.15825 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.126+0000 7f5c52ffd640 1 -- 172.21.15.60:0/398541198 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555a20bc1d0 con 0x5555a21606c0 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.126+0000 7f5c52ffd640 10 client.15825 handle_command_reply: tid=0 2024-07-24T21:47:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.126+0000 7f5c7ce67740 1 client.15825 shutdown 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 msgr2=0x5555a21509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005db0 tx=0x7f5c5c0045f0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 shutdown_connections 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21606c0 0x5555a2162ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a2151170 0x5555a214e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c52ffd640 0 client.15825 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a21451e0 0x5555a214de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/398541198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a21509a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 >> 172.21.15.60:0/398541198 conn(0x5555a21426f0 msgr2=0x5555a2148220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 shutdown_connections 2024-07-24T21:47:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.127+0000 7f5c7ce67740 1 -- 172.21.15.60:0/398541198 wait complete. 2024-07-24T21:47:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.128+0000 7f5c7ce67740 20 client.15825 trim_cache size 0 max 16384 2024-07-24T21:47:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 Processor -- start 2024-07-24T21:47:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 -- start start 2024-07-24T21:47:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 0x5555a2168340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a2168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b2a20 con 0x5555a21448c0 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b02f0 con 0x5555a21451e0 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c7ce67740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741ae140 con 0x5555a2151170 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51646/0 (socket says 172.21.15.60:51646) 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.159+0000 7f5c6bfff640 1 -- 172.21.15.60:0/3794026608 learned_addr learned my addr 172.21.15.60:0/3794026608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c737fe640 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 0x5555a2168340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 -- 172.21.15.60:0/3794026608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 msgr2=0x5555a2168880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a2168880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 -- 172.21.15.60:0/3794026608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 msgr2=0x5555a2168340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 0x5555a2168340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 -- 172.21.15.60:0/3794026608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a21455c0 con 0x5555a21448c0 2024-07-24T21:47:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c737fe640 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 0x5555a2168340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00a010 tx=0x7f5c5c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353b6772c91c7a25 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.160+0000 7f5c697fa640 1 -- 172.21.15.60:0/3794026608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c009bd0 con 0x5555a21448c0 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.161+0000 7f5c697fa640 1 -- 172.21.15.60:0/3794026608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5555a2058ce0 con 0x5555a21448c0 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.161+0000 7f5c697fa640 1 -- 172.21.15.60:0/3794026608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c004b70 con 0x5555a21448c0 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.161+0000 7f5c7ce67740 1 -- 172.21.15.60:0/3794026608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 msgr2=0x5555a214e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.161+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00a010 tx=0x7f5c5c023000 comp rx=0 tx=0).stop 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 -- 172.21.15.60:0/3794026608 shutdown_connections 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a2168880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21451e0 0x5555a2168340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/3794026608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21448c0 0x5555a214e360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 -- 172.21.15.60:0/3794026608 >> 172.21.15.60:0/3794026608 conn(0x5555a21426f0 msgr2=0x5555a214fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 -- 172.21.15.60:0/3794026608 shutdown_connections 2024-07-24T21:47:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 -- 172.21.15.60:0/3794026608 wait complete. 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 -- start start 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.162+0000 7f5c7ce67740 1 Processor -- start 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a21563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51660/0 (socket says 172.21.15.60:51660) 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 -- 172.21.15.60:0/521216811 learned_addr learned my addr 172.21.15.60:0/521216811 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21606c0 0x5555a21568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741ae7b0 con 0x5555a21451e0 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741afc20 con 0x5555a21606c0 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741aee80 con 0x5555a2151170 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c7ce67740 10 client.0 _ll_register_callbacks cb 0x7f5c5c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c737fe640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a21563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21606c0 0x5555a21568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 -- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 msgr2=0x5555a21563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a21563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 -- 172.21.15.60:0/521216811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21606c0 msgr2=0x5555a21568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21606c0 0x5555a21568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.163+0000 7f5c6bfff640 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a21c9340 con 0x5555a21451e0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c6bfff640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003a70 tx=0x7f5c5c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7ae30491b34a7e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c009bd0 con 0x5555a21451e0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5c5c005300 con 0x5555a21451e0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555a2058ce0 con 0x5555a21451e0 2024-07-24T21:47:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c053050 con 0x5555a21451e0 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c7ce67740 10 client.15831 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5555a21ca610 con 0x5555a21451e0 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c028070 con 0x5555a21451e0 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.164+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5c5c0366f0 con 0x5555a21451e0 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 20 client.15831 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21cac20 0x5555a21cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 4 client.15831 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:47:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555a2058ce0 con 0x5555a21cac20 2024-07-24T21:47:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c737fe640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21cac20 0x5555a21cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c52ffd640 10 client.15831 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.165+0000 7f5c737fe640 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21cac20 0x5555a21cd010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a38cbf9dccbfb0fa server_cookie=e2a3d4eb595345f in_seq=0 out_seq=0 2024-07-24T21:47:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c52ffd640 1 -- 172.21.15.60:0/521216811 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555a2058ce0 con 0x5555a21cac20 2024-07-24T21:47:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c52ffd640 10 client.15831 handle_command_reply: tid=0 2024-07-24T21:47:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 client.15831 shutdown 2024-07-24T21:47:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 msgr2=0x5555a2155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003a70 tx=0x7f5c5c005f90 comp rx=0 tx=0).stop 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 shutdown_connections 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5555a21cac20 0x5555a21cd010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555a21606c0 0x5555a21568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c52ffd640 0 client.15831 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555a2151170 0x5555a21563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 --2- 172.21.15.60:0/521216811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5555a21451e0 0x5555a2155e60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.169+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 >> 172.21.15.60:0/521216811 conn(0x5555a21426f0 msgr2=0x5555a21433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 shutdown_connections 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7ce67740 1 -- 172.21.15.60:0/521216811 wait complete. 2024-07-24T21:47:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7ce67740 20 client.15831 trim_cache size 0 max 16384 2024-07-24T21:47:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:47:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:47:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:47:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 msgr2=0x7f5c600b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 0x7f5c600b6280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0e1cd0 tx=0x7f5c6c00cf70 comp rx=0 tx=0).stop 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 msgr2=0x7f5c7410f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.170+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00eb10 tx=0x7f5c6c00c2d0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 shutdown_connections 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c600b3dc0 0x7f5c600b6280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c74107910 0x7f5c7410f6a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c74106f90 0x7f5c7410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 --2- 172.21.15.60:0/965689872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741015e0 0x7f5c7410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 >> 172.21.15.60:0/965689872 conn(0x7f5c740782e0 msgr2=0x7f5c741038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 shutdown_connections 2024-07-24T21:47:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:47:51.171+0000 7f5c7a10c640 1 -- 172.21.15.60:0/965689872 wait complete. 2024-07-24T21:47:51.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:51 smithi060 ceph-mon[33000]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:51.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:51 smithi086 ceph-mon[37128]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:51.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:51 smithi123 ceph-mon[37143]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-24T21:47:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 -- 172.21.15.60:0/1284929374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c102780 msgr2=0x7f580c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 --2- 172.21.15.60:0/1284929374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c102780 0x7f580c102b60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f580002a410 comp rx=0 tx=0).stop 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 -- 172.21.15.60:0/1284929374 shutdown_connections 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 --2- 172.21.15.60:0/1284929374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 --2- 172.21.15.60:0/1284929374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c108190 0x7f580c1085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 --2- 172.21.15.60:0/1284929374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c102780 0x7f580c102b60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 -- 172.21.15.60:0/1284929374 >> 172.21.15.60:0/1284929374 conn(0x7f580c0fe1d0 msgr2=0x7f580c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 -- 172.21.15.60:0/1284929374 shutdown_connections 2024-07-24T21:47:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.774+0000 7f5813060640 1 -- 172.21.15.60:0/1284929374 wait complete. 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 Processor -- start 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 -- start start 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 0x7f580c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c108190 0x7f580c06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f580c112740 con 0x7f580c108190 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1125c0 con 0x7f580c102780 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.775+0000 7f5813060640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1128c0 con 0x7f580c108b10 2024-07-24T21:47:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 0x7f580c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55204/0 (socket says 172.21.15.60:55204) 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 -- 172.21.15.60:0/178411075 learned_addr learned my addr 172.21.15.60:0/178411075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 -- 172.21.15.60:0/178411075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 msgr2=0x7f580c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 0x7f580c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 -- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c108190 msgr2=0x7f580c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c108190 0x7f580c06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 -- 172.21.15.60:0/178411075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c076780 con 0x7f580c108b10 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f5810dfc640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 0x7f580c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.776+0000 7f58115fd640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f580400eb10 tx=0x7f580400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.777+0000 7f5809ffb640 1 -- 172.21.15.60:0/178411075 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004b20 con 0x7f580c108b10 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.777+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c076a70 con 0x7f580c108b10 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.777+0000 7f5809ffb640 1 -- 172.21.15.60:0/178411075 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5804004cc0 con 0x7f580c108b10 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.778+0000 7f5809ffb640 1 -- 172.21.15.60:0/178411075 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804010690 con 0x7f580c108b10 2024-07-24T21:47:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.778+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c073050 con 0x7f580c108b10 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5809ffb640 1 -- 172.21.15.60:0/178411075 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5804005000 con 0x7f580c108b10 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 Processor -- start 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 -- start start 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e89846d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e89847bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e8983e3dd0 con 0x55e898479780 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8983d32d0 con 0x55e89846ced0 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.781+0000 7f5815411740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8983d7310 con 0x55e89846d7f0 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f580bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e89846d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55210/0 (socket says 172.21.15.60:55210) 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 -- 172.21.15.60:0/616523835 learned_addr learned my addr 172.21.15.60:0/616523835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f580bfff640 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e89847bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 -- 172.21.15.60:0/616523835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 msgr2=0x55e89846d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e89846d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 -- 172.21.15.60:0/616523835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 msgr2=0x55e89847bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e89847bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 -- 172.21.15.60:0/616523835 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e898133b00 con 0x55e89846d7f0 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5809ffb640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 0x7f57fc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5810dfc640 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa4c04c82081f30 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.782+0000 7f5809ffb640 1 -- 172.21.15.60:0/178411075 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f58040e45b0 con 0x7f580c108b10 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f57f6ffd640 1 -- 172.21.15.60:0/616523835 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800040070 con 0x55e89846d7f0 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f580bfff640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 0x7f57fc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f57f6ffd640 1 -- 172.21.15.60:0/616523835 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58000318e0 con 0x55e89846d7f0 2024-07-24T21:47:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f57f6ffd640 1 -- 172.21.15.60:0/616523835 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800035040 con 0x55e89846d7f0 2024-07-24T21:47:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f5815411740 1 -- 172.21.15.60:0/616523835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 msgr2=0x55e898479240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.783+0000 7f5815411740 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800005e40 comp rx=0 tx=0).stop 2024-07-24T21:47:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f580bfff640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 0x7f57fc0b6280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f80058c0 tx=0x7f57f80046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 -- 172.21.15.60:0/616523835 shutdown_connections 2024-07-24T21:47:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e89847bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846d7f0 0x55e898479240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 --2- 172.21.15.60:0/616523835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e89846d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 -- 172.21.15.60:0/616523835 >> 172.21.15.60:0/616523835 conn(0x55e89846ad00 msgr2=0x55e89846bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 -- 172.21.15.60:0/616523835 shutdown_connections 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 -- 172.21.15.60:0/616523835 wait complete. 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 -- start start 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.784+0000 7f5815411740 1 Processor -- start 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 0x55e898476be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e898477120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e898236bb0 con 0x55e89846d7f0 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e898375900 con 0x55e89846ced0 2024-07-24T21:47:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e89847c3a0 con 0x55e898479780 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5815411740 10 client.0 _ll_register_callbacks cb 0x55e8983a1390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51504/0 (socket says 172.21.15.60:51504) 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 -- 172.21.15.60:0/3167284688 learned_addr learned my addr 172.21.15.60:0/3167284688 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f580bfff640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 0x55e898476be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 -- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 msgr2=0x55e898477120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e898477120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 -- 172.21.15.60:0/3167284688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 msgr2=0x55e898476be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 0x55e898476be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f5810dfc640 1 -- 172.21.15.60:0/3167284688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8981aa960 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.785+0000 7f580bfff640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 0x55e898476be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f5810dfc640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f580003d980 tx=0x7f580003d9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c828fb06544fff89 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f57ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580004a070 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f58000055f0 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58000318e0 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5800035040 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f5815411740 10 client.25391 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e898487ee0 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.786+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58000318e0 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5800044400 con 0x55e89846ced0 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 10 client.25391 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 10 client.25391 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 20 client.25391 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e898489230 0x55e89848b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 4 client.25391 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8983e5d20 con 0x55e898489230 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.787+0000 7f580bfff640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e898489230 0x55e89848b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.788+0000 7f57ebfff640 10 client.25391 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.788+0000 7f580bfff640 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e898489230 0x55e89848b620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9504a5427459fa server_cookie=612c0d73a19fa9ce in_seq=0 out_seq=0 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f57ebfff640 1 -- 172.21.15.60:0/3167284688 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8983e5d20 con 0x55e898489230 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f57ebfff640 10 client.25391 handle_command_reply: tid=0 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 client.25391 shutdown 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 msgr2=0x55e8984766a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f580003d980 tx=0x7f580003d9b0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 shutdown_connections 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e898489230 0x55e89848b620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e898477120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846d7f0 0x55e898476be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 --2- 172.21.15.60:0/3167284688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846ced0 0x55e8984766a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 >> 172.21.15.60:0/3167284688 conn(0x55e89846ad00 msgr2=0x55e8984708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.789+0000 7f57ebfff640 0 client.25391 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.790+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 shutdown_connections 2024-07-24T21:47:51.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.790+0000 7f5815411740 1 -- 172.21.15.60:0/3167284688 wait complete. 2024-07-24T21:47:51.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.790+0000 7f5815411740 20 client.25391 trim_cache size 0 max 16384 2024-07-24T21:47:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 Processor -- start 2024-07-24T21:47:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 -- start start 2024-07-24T21:47:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e898471020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e89848e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1bf7f0 con 0x55e89846ced0 2024-07-24T21:47:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1bd0c0 con 0x55e89846d7f0 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.808+0000 7f5815411740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1baf10 con 0x55e898479780 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51718/0 (socket says 172.21.15.60:51718) 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 -- 172.21.15.60:0/1772866854 learned_addr learned my addr 172.21.15.60:0/1772866854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f58115fd640 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e89848e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 -- 172.21.15.60:0/1772866854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 msgr2=0x55e89848e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e89848e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 -- 172.21.15.60:0/1772866854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 msgr2=0x55e898471020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f580bfff640 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e898471020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e898471020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.809+0000 7f5810dfc640 1 -- 172.21.15.60:0/1772866854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e898474ba0 con 0x55e89846ced0 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.810+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f580002a930 tx=0x7f5800000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e12d8a816659dd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.810+0000 7f57f67fc640 1 -- 172.21.15.60:0/1772866854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580005c020 con 0x55e89846ced0 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.810+0000 7f57f67fc640 1 -- 172.21.15.60:0/1772866854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800009520 con 0x55e89846ced0 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.810+0000 7f57f67fc640 1 -- 172.21.15.60:0/1772866854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800040040 con 0x55e89846ced0 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 -- 172.21.15.60:0/1772866854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 msgr2=0x55e898475590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f580002a930 tx=0x7f5800000bc0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 -- 172.21.15.60:0/1772866854 shutdown_connections 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e898479780 0x55e89848e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e898471020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 --2- 172.21.15.60:0/1772866854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e89846ced0 0x55e898475590 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 -- 172.21.15.60:0/1772866854 >> 172.21.15.60:0/1772866854 conn(0x55e89846ad00 msgr2=0x55e8984784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 -- 172.21.15.60:0/1772866854 shutdown_connections 2024-07-24T21:47:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.811+0000 7f5815411740 1 -- 172.21.15.60:0/1772866854 wait complete. 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.812+0000 7f5815411740 1 -- start start 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.812+0000 7f5815411740 1 Processor -- start 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.812+0000 7f5815411740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 0x55e89848c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.812+0000 7f5815411740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e89848ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.812+0000 7f5815411740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e8984f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5815411740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1bb580 con 0x55e898479780 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5815411740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1bc9f0 con 0x55e89846d7f0 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5815411740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1bbc50 con 0x55e89846ced0 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5815411740 10 client.0 _ll_register_callbacks cb 0x7f57f800d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f58115fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e8984f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f58115fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e8984f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51734/0 (socket says 172.21.15.60:51734) 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f58115fd640 1 -- 172.21.15.60:0/1867262188 learned_addr learned my addr 172.21.15.60:0/1867262188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 0x55e89848c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f580bfff640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e89848ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 -- 172.21.15.60:0/1867262188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 msgr2=0x55e89848ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e89848ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 -- 172.21.15.60:0/1867262188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 msgr2=0x55e8984f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e8984f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 -- 172.21.15.60:0/1867262188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8984f30b0 con 0x55e89846ced0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f5810dfc640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 0x55e89848c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7bd561ddfcfbcde server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.813+0000 7f57ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800040070 con 0x55e89846ced0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5800009520 con 0x55e89846ced0 2024-07-24T21:47:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5800005450 con 0x55e89846ced0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5800065050 con 0x55e89846ced0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 10 client.25384 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8984f4370 con 0x55e89846ced0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800009520 con 0x55e89846ced0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5800068020 con 0x55e89846ced0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 10 client.25384 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 10 client.25384 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 20 client.25384 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.814+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8984f49d0 0x55e8984f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.815+0000 7f5815411740 4 client.25384 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "98c11f33", "release": true, "await": true}] 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.815+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "98c11f33", "release": true, "await": true}) -- 0x55e8983e17c0 con 0x55e8984f49d0 2024-07-24T21:47:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.815+0000 7f580bfff640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8984f49d0 0x55e8984f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.815+0000 7f57ebfff640 10 client.25384 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.815+0000 7f580bfff640 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8984f49d0 0x55e8984f6dc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1673a427ab07e0 server_cookie=a0292b4d5d119cbb in_seq=0 out_seq=0 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f57ebfff640 1 -- 172.21.15.60:0/1867262188 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e8983e17c0 con 0x55e8984f49d0 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f57ebfff640 10 client.25384 handle_command_reply: tid=0 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 client.25384 shutdown 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 msgr2=0x55e89848c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 0x55e89848c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5800009900 tx=0x7f5800031400 comp rx=0 tx=0).stop 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 shutdown_connections 2024-07-24T21:47:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8984f49d0 0x55e8984f6dc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f57ebfff640 0 client.25384 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e898479780 0x55e8984f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e89846d7f0 0x55e89848ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 --2- 172.21.15.60:0/1867262188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e89846ced0 0x55e89848c550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 >> 172.21.15.60:0/1867262188 conn(0x55e89846ad00 msgr2=0x55e898484660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 shutdown_connections 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 1 -- 172.21.15.60:0/1867262188 wait complete. 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.817+0000 7f5815411740 20 client.25384 trim_cache size 0 max 16384 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:51.821 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 26, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 4, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "98c11f33": { 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 4, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:47:51.822 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.3, 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:51.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:47:51.824 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 msgr2=0x7f57fc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 0x7f57fc0b6280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f80058c0 tx=0x7f57f80046c0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 msgr2=0x7f580c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f580400eb10 tx=0x7f580400c2d0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 shutdown_connections 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57fc0b3dc0 0x7f57fc0b6280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f580c108b10 0x7f580c076000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f580c108190 0x7f580c06dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 --2- 172.21.15.60:0/178411075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c102780 0x7f580c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 >> 172.21.15.60:0/178411075 conn(0x7f580c0fe1d0 msgr2=0x7f580c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 shutdown_connections 2024-07-24T21:47:51.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:51.820+0000 7f5813060640 1 -- 172.21.15.60:0/178411075 wait complete. 2024-07-24T21:47:52.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:47:52.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:47:52.532 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '98c11f33', total seconds elapsed: 32.9 2024-07-24T21:47:52.532 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-07-24T21:47:52.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:47:52.669 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:53.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:53 smithi060 ceph-mon[33000]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:53 smithi086 ceph-mon[37128]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:53.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:53 smithi123 ceph-mon[37143]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-24T21:47:54.301 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 -- 172.21.15.60:0/2376874977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 msgr2=0x7f19f81074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 --2- 172.21.15.60:0/2376874977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f81074a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009900 tx=0x7f19ec02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 -- 172.21.15.60:0/2376874977 shutdown_connections 2024-07-24T21:47:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 --2- 172.21.15.60:0/2376874977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 0x7f19f810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 --2- 172.21.15.60:0/2376874977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f81074a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 --2- 172.21.15.60:0/2376874977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 0x7f19f8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 -- 172.21.15.60:0/2376874977 >> 172.21.15.60:0/2376874977 conn(0x7f19f80fd0c0 msgr2=0x7f19f80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 -- 172.21.15.60:0/2376874977 shutdown_connections 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.814+0000 7f19fe849640 1 -- 172.21.15.60:0/2376874977 wait complete. 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 Processor -- start 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 -- start start 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 0x7f19f8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 0x7f19f819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8111600 con 0x7f19f8107060 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8111480 con 0x7f19f81079e0 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19fe849640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8111780 con 0x7f19f8101650 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.815+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51754/0 (socket says 172.21.15.60:51754) 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 -- 172.21.15.60:0/3094593941 learned_addr learned my addr 172.21.15.60:0/3094593941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19fcde6640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 0x7f19f819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f7fff640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 0x7f19f8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 -- 172.21.15.60:0/3094593941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 msgr2=0x7f19f8198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 0x7f19f8198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 -- 172.21.15.60:0/3094593941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 msgr2=0x7f19f819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 0x7f19f819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 -- 172.21.15.60:0/3094593941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f819eca0 con 0x7f19f8107060 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f77fe640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f19ec005940 tx=0x7f19ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f57fa640 1 -- 172.21.15.60:0/3094593941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec03f030 con 0x7f19f8107060 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f819ef30 con 0x7f19f8107060 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f57fa640 1 -- 172.21.15.60:0/3094593941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec003cb0 con 0x7f19f8107060 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19f57fa640 1 -- 172.21.15.60:0/3094593941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec035070 con 0x7f19f8107060 2024-07-24T21:47:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.816+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a5650 con 0x7f19f8107060 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.819+0000 7f1a00bfa740 1 Processor -- start 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.819+0000 7f1a00bfa740 1 -- start start 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa1193d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11949290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa118b49f0 con 0x55aa119497d0 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa118a3ef0 con 0x55aa1193d840 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa118a7f30 con 0x55aa1193cf20 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f57fa640 1 -- 172.21.15.60:0/3094593941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f19ec042030 con 0x7f19f8107060 2024-07-24T21:47:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51770/0 (socket says 172.21.15.60:51770) 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19fcde6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa1193d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 -- 172.21.15.60:0/2046098813 learned_addr learned my addr 172.21.15.60:0/2046098813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f7fff640 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11949290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 -- 172.21.15.60:0/2046098813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 msgr2=0x55aa1193d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa1193d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 -- 172.21.15.60:0/2046098813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 msgr2=0x55aa11949290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11949290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19f77fe640 1 -- 172.21.15.60:0/2046098813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa11604930 con 0x55aa119497d0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.820+0000 7f19fcde6640 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa1193d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f19ec101ee0 tx=0x7f19ec10acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b478855dd771ac server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19de7fc640 1 -- 172.21.15.60:0/2046098813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec10e670 con 0x55aa119497d0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19de7fc640 1 -- 172.21.15.60:0/2046098813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec114020 con 0x55aa119497d0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19de7fc640 1 -- 172.21.15.60:0/2046098813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec10fcb0 con 0x55aa119497d0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19f57fa640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 0x7f19e00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19fcde6640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 0x7f19e00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.821+0000 7f19f57fa640 1 -- 172.21.15.60:0/3094593941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f19ec104820 con 0x7f19f8107060 2024-07-24T21:47:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2046098813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 msgr2=0x55aa1194bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f19ec101ee0 tx=0x7f19ec10acd0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f19fcde6640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 0x7f19e00b6200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010620 tx=0x7f19f0007520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2046098813 shutdown_connections 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa1194bbc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11949290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2046098813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa1193d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2046098813 >> 172.21.15.60:0/2046098813 conn(0x55aa1193ad50 msgr2=0x55aa1193bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2046098813 shutdown_connections 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2046098813 wait complete. 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.822+0000 7f1a00bfa740 1 -- start start 2024-07-24T21:47:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 Processor -- start 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11949280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11946710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa11707890 con 0x55aa119497d0 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa11846390 con 0x55aa1193d840 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa118b49f0 con 0x55aa1193cf20 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f1a00bfa740 10 client.0 _ll_register_callbacks cb 0x55aa11850500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51784/0 (socket says 172.21.15.60:51784) 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 -- 172.21.15.60:0/609212810 learned_addr learned my addr 172.21.15.60:0/609212810 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19f77fe640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11946710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19f7fff640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11949280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 -- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 msgr2=0x55aa11949280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11949280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 -- 172.21.15.60:0/609212810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 msgr2=0x55aa11946710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11946710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 -- 172.21.15.60:0/609212810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa11840d60 con 0x55aa119497d0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.823+0000 7f19fcde6640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19f000fdb0 tx=0x7f19f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a139c4e5aa770d68 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0027440 con 0x55aa119497d0 2024-07-24T21:47:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f19f001b3f0 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ec114020 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f001b5f0 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa11957f10 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec114020 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f19f0025070 con 0x55aa119497d0 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.824+0000 7f1a00bfa740 20 client.15858 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa11959470 0x55aa1195b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f1a00bfa740 4 client.15858 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa118b6940 con 0x55aa11959470 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f19f7fff640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa11959470 0x55aa1195b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f19d77fe640 10 client.15858 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.825+0000 7f19f7fff640 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa11959470 0x55aa1195b860 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6246a85e9301c0 server_cookie=fee15cafff96b274 in_seq=0 out_seq=0 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f19d77fe640 1 -- 172.21.15.60:0/609212810 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa118b6940 con 0x55aa11959470 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f19d77fe640 10 client.15858 handle_command_reply: tid=0 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 client.15858 shutdown 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 msgr2=0x55aa11946ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19f000fdb0 tx=0x7f19f0004ab0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 shutdown_connections 2024-07-24T21:47:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa11959470 0x55aa1195b860 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa119497d0 0x55aa11946ea0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193d840 0x55aa11946710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/609212810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11949280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f19d77fe640 0 client.15858 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 >> 172.21.15.60:0/609212810 conn(0x55aa1193ad50 msgr2=0x55aa119408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 shutdown_connections 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 1 -- 172.21.15.60:0/609212810 wait complete. 2024-07-24T21:47:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.826+0000 7f1a00bfa740 20 client.15858 trim_cache size 0 max 16384 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 Processor -- start 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 -- start start 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11961130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 0x55aa119438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81b0950 con 0x55aa1193d840 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ae220 con 0x55aa119497d0 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.845+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ac070 con 0x55aa1193cf20 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51790/0 (socket says 172.21.15.60:51790) 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 -- 172.21.15.60:0/2258636604 learned_addr learned my addr 172.21.15.60:0/2258636604 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19fcde6640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 0x55aa119438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f7fff640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11961130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 -- 172.21.15.60:0/2258636604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 msgr2=0x55aa11961130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11961130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 -- 172.21.15.60:0/2258636604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 msgr2=0x55aa119438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 0x55aa119438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 -- 172.21.15.60:0/2258636604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1193bc60 con 0x55aa1193d840 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19fcde6640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 0x55aa119438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f7fff640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11961130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19f77fe640 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f19ec10f7e0 tx=0x7f19ec10f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270060f077b7c253 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19ddffb640 1 -- 172.21.15.60:0/2258636604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec049050 con 0x55aa1193d840 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19ddffb640 1 -- 172.21.15.60:0/2258636604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec10a320 con 0x55aa1193d840 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.846+0000 7f19ddffb640 1 -- 172.21.15.60:0/2258636604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0cae30 con 0x55aa1193d840 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2258636604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 msgr2=0x55aa11961670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f19ec10f7e0 tx=0x7f19ec10f770 comp rx=0 tx=0).stop 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2258636604 shutdown_connections 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa119497d0 0x55aa119438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa11961670 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/2258636604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa1193cf20 0x55aa11961130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2258636604 >> 172.21.15.60:0/2258636604 conn(0x55aa1193ad50 msgr2=0x55aa11948520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2258636604 shutdown_connections 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.847+0000 7f1a00bfa740 1 -- 172.21.15.60:0/2258636604 wait complete. 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 -- start start 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 Processor -- start 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193cf20 0x55aa1194d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa119497d0 0x55aa119c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ac6e0 con 0x55aa1193d840 2024-07-24T21:47:54.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81adb50 con 0x55aa1193cf20 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81acdb0 con 0x55aa119497d0 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f1a00bfa740 10 client.0 _ll_register_callbacks cb 0x7f19f000fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f19f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51804/0 (socket says 172.21.15.60:51804) 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.848+0000 7f19f77fe640 1 -- 172.21.15.60:0/4003971206 learned_addr learned my addr 172.21.15.60:0/4003971206 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f7fff640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193cf20 0x55aa1194d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19fcde6640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa119497d0 0x55aa119c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 -- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa119497d0 msgr2=0x55aa119c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa119497d0 0x55aa119c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 -- 172.21.15.60:0/4003971206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193cf20 msgr2=0x55aa1194d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:54.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193cf20 0x55aa1194d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 -- 172.21.15.60:0/4003971206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa119c7bb0 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19f77fe640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f19ec004650 tx=0x7f19ec0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4b428e84d0ebe3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec113070 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f19ec0c7ad0 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ec10a320 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.849+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19ec049030 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa119c8ef0 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ec10a320 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f19ec10f440 con 0x55aa1193d840 2024-07-24T21:47:54.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 20 client.15864 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa119c9370 0x55aa119cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 4 client.15864 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aa118b23e0 con 0x55aa119c9370 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.850+0000 7f19f7fff640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa119c9370 0x55aa119cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.851+0000 7f19d77fe640 10 client.15864 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.851+0000 7f19f7fff640 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa119c9370 0x55aa119cb760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33bab2627d40b055 server_cookie=43f0b9a7d29b02b in_seq=0 out_seq=0 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f19d77fe640 1 -- 172.21.15.60:0/4003971206 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aa118b23e0 con 0x55aa119c9370 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f19d77fe640 10 client.15864 handle_command_reply: tid=0 2024-07-24T21:47:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 client.15864 shutdown 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 msgr2=0x55aa1194deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f19ec004650 tx=0x7f19ec0045a0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 shutdown_connections 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa119c9370 0x55aa119cb760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa119497d0 0x55aa119c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa1193d840 0x55aa1194deb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 --2- 172.21.15.60:0/4003971206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa1193cf20 0x55aa1194d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 >> 172.21.15.60:0/4003971206 conn(0x55aa1193ad50 msgr2=0x55aa11954160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f19d77fe640 0 client.15864 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 shutdown_connections 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 1 -- 172.21.15.60:0/4003971206 wait complete. 2024-07-24T21:47:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.852+0000 7f1a00bfa740 20 client.15864 trim_cache size 0 max 16384 2024-07-24T21:47:54.862 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:54.862 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:47:54.862 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b204a749-e5b3-4e78-aaa4-6883144430a1", 2024-07-24T21:47:54.862 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:47:54.862 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 msgr2=0x7f19e00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 0x7f19e00b6200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010620 tx=0x7f19f0007520 comp rx=0 tx=0).stop 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 msgr2=0x7f19f8199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f19ec005940 tx=0x7f19ec0042f0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 shutdown_connections 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19e00b3d40 0x7f19e00b6200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f81079e0 0x7f19f819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8107060 0x7f19f8199130 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 --2- 172.21.15.60:0/3094593941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f8101650 0x7f19f8198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.853+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 >> 172.21.15.60:0/3094593941 conn(0x7f19f80fd0c0 msgr2=0x7f19f80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.854+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 shutdown_connections 2024-07-24T21:47:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:54.854+0000 7f19fe849640 1 -- 172.21.15.60:0/3094593941 wait complete. 2024-07-24T21:47:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:55 smithi086 ceph-mon[37128]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-07-24T21:47:55.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:55 smithi060 ceph-mon[33000]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-07-24T21:47:55.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:47:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:55 smithi123 ceph-mon[37143]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-07-24T21:47:55.881 INFO:tasks.workunit.client.0.smithi060.stderr:Updating files: 85% (10858/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:Note: switching to '0325fd3c11a225c0fd9eb38fab88aad1370ad779'. 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:state without impacting any branches by switching back to a branch. 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-24T21:47:56.516 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: git switch -c 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr:Or undo this operation with: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: git switch - 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:56.517 INFO:tasks.workunit.client.0.smithi060.stderr:HEAD is now at 0325fd3c11a Merge pull request #58530 from zdover23/wip-doc-2024-07-11-radosgw-s3-authentication 2024-07-24T21:47:56.519 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-24T21:47:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:56 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:47:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:56 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:47:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:56 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:47:56.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:56 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:47:56.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.620 INFO:tasks.workunit.client.0.smithi060.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-24T21:47:56.623 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-24T21:47:56.623 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-24T21:47:56.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:56 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:47:56.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:56 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:47:56.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:47:56.988 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:47:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:57 smithi086 ceph-mon[37128]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-07-24T21:47:57.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:57 smithi060 ceph-mon[33000]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-07-24T21:47:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:57 smithi123 ceph-mon[37143]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 -- 172.21.15.60:0/3151593509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00102790 msgr2=0x7fad00102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 --2- 172.21.15.60:0/3151593509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00102790 0x7fad00102b70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7facfc009ca0 tx=0x7facfc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 -- 172.21.15.60:0/3151593509 shutdown_connections 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 --2- 172.21.15.60:0/3151593509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00108b20 0x7fad0010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 --2- 172.21.15.60:0/3151593509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 0x7fad001085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 --2- 172.21.15.60:0/3151593509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00102790 0x7fad00102b70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 -- 172.21.15.60:0/3151593509 >> 172.21.15.60:0/3151593509 conn(0x7fad000fe200 msgr2=0x7fad00100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 -- 172.21.15.60:0/3151593509 shutdown_connections 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.410+0000 7fad076f6640 1 -- 172.21.15.60:0/3151593509 wait complete. 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 Processor -- start 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 -- start start 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 0x7fad00198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 0x7fad001991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fad00112740 con 0x7fad00108b20 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad001125c0 con 0x7fad00102790 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad076f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad001128c0 con 0x7fad001081a0 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42836/0 (socket says 172.21.15.60:42836) 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 -- 172.21.15.60:0/1612497746 learned_addr learned my addr 172.21.15.60:0/1612497746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:58.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad04c91640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 0x7fad001991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05492640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 0x7fad00198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 -- 172.21.15.60:0/1612497746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 msgr2=0x7fad001991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 0x7fad001991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 -- 172.21.15.60:0/1612497746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 msgr2=0x7fad00198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 0x7fad00198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05c93640 1 -- 172.21.15.60:0/1612497746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0019ed40 con 0x7fad00108b20 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.411+0000 7fad05492640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 0x7fad00198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7fad05c93640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7facf800c970 tx=0x7facf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8007bb0 con 0x7fad00108b20 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf8007d50 con 0x7fad00108b20 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0019f030 con 0x7fad00108b20 2024-07-24T21:47:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8004e60 con 0x7fad00108b20 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.412+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad001a56f0 con 0x7fad00108b20 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.413+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad00102b70 con 0x7fad00108b20 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.414+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7facf8005000 con 0x7fad00108b20 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.415+0000 7facf67fc640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 0x7facec0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.416+0000 7fad05492640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 0x7facec0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.416+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7facf80e4140 con 0x7fad00108b20 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.416+0000 7fad05492640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 0x7facec0b6200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7facfc000f80 tx=0x7facfc0098e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.419+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7facf8014030 con 0x7fad00108b20 2024-07-24T21:47:58.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.655+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad00199dc0 con 0x7fad00108b20 2024-07-24T21:47:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.656+0000 7facf67fc640 1 -- 172.21.15.60:0/1612497746 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fad00199dc0 con 0x7fad00108b20 2024-07-24T21:47:58.657 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:47:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:47:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:47:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 msgr2=0x7facec0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 0x7facec0b6200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7facfc000f80 tx=0x7facfc0098e0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 msgr2=0x7fad0019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7facf800c970 tx=0x7facf800ce30 comp rx=0 tx=0).stop 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 shutdown_connections 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7facec0b3d40 0x7facec0b6200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad00108b20 0x7fad0019e570 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad001081a0 0x7fad001991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 --2- 172.21.15.60:0/1612497746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad00102790 0x7fad00198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 >> 172.21.15.60:0/1612497746 conn(0x7fad000fe200 msgr2=0x7fad001005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 shutdown_connections 2024-07-24T21:47:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:58.658+0000 7fad076f6640 1 -- 172.21.15.60:0/1612497746 wait complete. 2024-07-24T21:47:58.800 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-24T21:47:58.903 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-24T21:47:58.944 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-24T21:47:58.945 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-24T21:47:58.945 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-24T21:47:59.008 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-24T21:47:59.012 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T21:47:59.012 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-24T21:47:59.070 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-07-24T21:47:59.072 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-07-24T21:47:59.072 DEBUG:teuthology.orchestra.run.smithi060:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0325fd3c11a225c0fd9eb38fab88aad1370ad779 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-07-24T21:47:59.134 INFO:tasks.workunit.client.0.smithi060.stdout:getting blogbench 2024-07-24T21:47:59.135 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'getting blogbench' 2024-07-24T21:47:59.135 INFO:tasks.workunit.client.0.smithi060.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-07-24T21:47:59.242 INFO:tasks.workunit.client.0.smithi060.stderr:--2024-07-24 21:47:59-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-07-24T21:47:59.243 INFO:tasks.workunit.client.0.smithi060.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-07-24T21:47:59.268 INFO:tasks.workunit.client.0.smithi060.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-07-24T21:47:59.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:59 smithi060 ceph-mon[33000]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-07-24T21:47:59.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:59 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1612497746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:59.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:47:59 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:47:59.298 INFO:tasks.workunit.client.0.smithi060.stderr:HTTP request sent, awaiting response... 200 OK 2024-07-24T21:47:59.298 INFO:tasks.workunit.client.0.smithi060.stderr:Length: 103355 (101K) [application/octet-stream] 2024-07-24T21:47:59.300 INFO:tasks.workunit.client.0.smithi060.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-07-24T21:47:59.302 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:59.339 INFO:tasks.workunit.client.0.smithi060.stderr: 0K .......... .......... .......... .......... .......... 49% 1.26M 0s 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr: 50K .......... .......... .......... .......... .......... 99% 1.65M 0s 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr: 100K 100% 1.74T=0.07s 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr:2024-07-24 21:47:59 (1.44 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-07-24T21:47:59.369 INFO:tasks.workunit.client.0.smithi060.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-07-24T21:47:59.481 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/ 2024-07-24T21:47:59.483 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/m4/ 2024-07-24T21:47:59.484 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-07-24T21:47:59.485 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/man/ 2024-07-24T21:47:59.486 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/man/blogbench.8 2024-07-24T21:47:59.488 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/man/Makefile.am 2024-07-24T21:47:59.489 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/man/Makefile.in 2024-07-24T21:47:59.490 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/ 2024-07-24T21:47:59.491 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/mysnprintf.c 2024-07-24T21:47:59.492 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/mysnprintf.h 2024-07-24T21:47:59.493 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/blogbench.c 2024-07-24T21:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/blogbench.h 2024-07-24T21:47:59.495 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/fakesnprintf.c 2024-07-24T21:47:59.496 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/fakesnprintf.h 2024-07-24T21:47:59.497 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/blogbench_p.h 2024-07-24T21:47:59.498 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-07-24T21:47:59.498 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-07-24T21:47:59.500 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/Makefile.am 2024-07-24T21:47:59.500 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/Makefile.in 2024-07-24T21:47:59.501 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/process.c 2024-07-24T21:47:59.502 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/globals.h 2024-07-24T21:47:59.503 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/commenter.c 2024-07-24T21:47:59.503 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/rewriter.c 2024-07-24T21:47:59.504 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/reader.c 2024-07-24T21:47:59.505 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/writer.c 2024-07-24T21:47:59.506 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/src/helpers.c 2024-07-24T21:47:59.507 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/NEWS 2024-07-24T21:47:59.508 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/compile 2024-07-24T21:47:59.509 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/depcomp 2024-07-24T21:47:59.509 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/aclocal.m4 2024-07-24T21:47:59.510 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/README 2024-07-24T21:47:59.511 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/configure 2024-07-24T21:47:59.515 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/configure.ac 2024-07-24T21:47:59.516 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/config.guess 2024-07-24T21:47:59.516 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/install-sh 2024-07-24T21:47:59.517 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/config.sub 2024-07-24T21:47:59.518 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/missing 2024-07-24T21:47:59.519 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/mkinstalldirs 2024-07-24T21:47:59.520 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/Makefile.am 2024-07-24T21:47:59.520 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/Makefile.in 2024-07-24T21:47:59.521 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/config.h.in 2024-07-24T21:47:59.522 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/AUTHORS 2024-07-24T21:47:59.523 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/INSTALL 2024-07-24T21:47:59.524 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/ChangeLog 2024-07-24T21:47:59.525 INFO:tasks.workunit.client.0.smithi060.stdout:blogbench-1.0/COPYING 2024-07-24T21:47:59.526 INFO:tasks.workunit.client.0.smithi060.stderr:+ cd blogbench-1.0/ 2024-07-24T21:47:59.526 INFO:tasks.workunit.client.0.smithi060.stdout:making blogbench 2024-07-24T21:47:59.527 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'making blogbench' 2024-07-24T21:47:59.527 INFO:tasks.workunit.client.0.smithi060.stderr:+ ./configure 2024-07-24T21:47:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:59 smithi086 ceph-mon[37128]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-07-24T21:47:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:59 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1612497746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:47:59 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:47:59.657 INFO:tasks.workunit.client.0.smithi060.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-07-24T21:47:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:59 smithi123 ceph-mon[37143]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-07-24T21:47:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:59 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1612497746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:47:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:47:59 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:47:59.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 -- 172.21.15.60:0/2723203707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a41079e0 msgr2=0x7f22a410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/2723203707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a41079e0 0x7f22a410e1e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c02a410 comp rx=0 tx=0).stop 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 -- 172.21.15.60:0/2723203707 shutdown_connections 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/2723203707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a41079e0 0x7f22a410e1e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/2723203707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4107060 0x7f22a41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/2723203707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4101650 0x7f22a4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 -- 172.21.15.60:0/2723203707 >> 172.21.15.60:0/2723203707 conn(0x7f22a40fd0c0 msgr2=0x7f22a40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 -- 172.21.15.60:0/2723203707 shutdown_connections 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.767+0000 7f22ac4cf640 1 -- 172.21.15.60:0/2723203707 wait complete. 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 Processor -- start 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 -- start start 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4107060 0x7f22a419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a41079e0 0x7f22a41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112780 con 0x7f22a4101650 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112600 con 0x7f22a4107060 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22ac4cf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4112900 con 0x7f22a41079e0 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42860/0 (socket says 172.21.15.60:42860) 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 -- 172.21.15.60:0/624201764 learned_addr learned my addr 172.21.15.60:0/624201764 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4107060 0x7f22a419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a41079e0 0x7f22a41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 -- 172.21.15.60:0/624201764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a41079e0 msgr2=0x7f22a41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a41079e0 0x7f22a41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 -- 172.21.15.60:0/624201764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4107060 msgr2=0x7f22a419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4107060 0x7f22a419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.768+0000 7f22aa26b640 1 -- 172.21.15.60:0/624201764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41a0ed0 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22aa26b640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f229400ebc0 tx=0x7f229400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22a37fe640 1 -- 172.21.15.60:0/624201764 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294010630 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41a11c0 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22a37fe640 1 -- 172.21.15.60:0/624201764 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2294010c80 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22a37fe640 1 -- 172.21.15.60:0/624201764 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294018b60 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.769+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a7a50 con 0x7f22a4101650 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 Processor -- start 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 -- start start 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcbca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcc89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 0x5595afccb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5595afc33960 con 0x5595afcc8f30 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595afc22e60 con 0x5595afcbc680 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22ae880740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595afc26ea0 con 0x5595afcbcfa0 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22a37fe640 1 -- 172.21.15.60:0/624201764 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2294018d00 con 0x7f22a4101650 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22a9a6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 0x5595afccb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22a9a6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 0x5595afccb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42876/0 (socket says 172.21.15.60:42876) 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22a9a6a640 1 -- 172.21.15.60:0/233152008 learned_addr learned my addr 172.21.15.60:0/233152008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.772+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcbca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aa26b640 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcc89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 -- 172.21.15.60:0/233152008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 msgr2=0x5595afcc89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcc89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 -- 172.21.15.60:0/233152008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 msgr2=0x5595afccb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 0x5595afccb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 -- 172.21.15.60:0/233152008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595afbac2d0 con 0x5595afcbc680 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcbca60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f229c005940 tx=0x7f229c032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3839542b96e3a4a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22a37fe640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 0x7f228c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22a0ff9640 1 -- 172.21.15.60:0/233152008 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c040070 con 0x5595afcbc680 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22a0ff9640 1 -- 172.21.15.60:0/233152008 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c004030 con 0x5595afcbc680 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22aa26b640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 0x7f228c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22a0ff9640 1 -- 172.21.15.60:0/233152008 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c035040 con 0x5595afcbc680 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.773+0000 7f22a37fe640 1 -- 172.21.15.60:0/624201764 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f22940e4620 con 0x7f22a4101650 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22aa26b640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 0x7f228c0b6200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5595afcbb390 tx=0x7f229400cf90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 -- 172.21.15.60:0/233152008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 msgr2=0x5595afcbca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcbca60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f229c005940 tx=0x7f229c032b20 comp rx=0 tx=0).stop 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 -- 172.21.15.60:0/233152008 shutdown_connections 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcc8f30 0x5595afccb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcc89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 --2- 172.21.15.60:0/233152008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcbca60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 -- 172.21.15.60:0/233152008 >> 172.21.15.60:0/233152008 conn(0x5595afcba4b0 msgr2=0x5595afcbb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 -- 172.21.15.60:0/233152008 shutdown_connections 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.774+0000 7f22ae880740 1 -- 172.21.15.60:0/233152008 wait complete. 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 -- start start 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 Processor -- start 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcc5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afcc6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5595afa86700 con 0x5595afcbcfa0 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595afbc55a0 con 0x5595afcbc680 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595afccbb50 con 0x5595afcc8f30 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22ae880740 10 client.0 _ll_register_callbacks cb 0x5595afbe5910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22a9a6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22a9a6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42886/0 (socket says 172.21.15.60:42886) 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22a9a6a640 1 -- 172.21.15.60:0/1174210683 learned_addr learned my addr 172.21.15.60:0/1174210683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcc5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.775+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afcc6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 -- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 msgr2=0x5595afcc6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afcc6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 -- 172.21.15.60:0/1174210683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 msgr2=0x5595afcc5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcc5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 -- 172.21.15.60:0/1174210683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595afbb8630 con 0x5595afcbcfa0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcc5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2298012840 tx=0x7f2298012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4370b4ee2b2b62a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f2289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229800f8c0 con 0x5595afcbcfa0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f229800fa60 con 0x5595afcbcfa0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c004030 con 0x5595afcbcfa0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f229800fc60 con 0x5595afcbcfa0 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22ae880740 10 client.15882 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595af983650 con 0x5595afcbcfa0 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.776+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c004030 con 0x5595afcbcfa0 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f229801c420 con 0x5595afcbcfa0 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 20 client.15882 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afcdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 4 client.15882 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595afc358b0 con 0x5595afcd8a10 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afcdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afcdae70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1814ef7dac9b0ec7 server_cookie=e43f5aeee5dc10b3 in_seq=0 out_seq=0 2024-07-24T21:47:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.777+0000 7f2289ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f2289ffb640 1 -- 172.21.15.60:0/1174210683 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595afc358b0 con 0x5595afcd8a10 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f2289ffb640 10 client.15882 handle_command_reply: tid=0 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 client.15882 shutdown 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 msgr2=0x5595afcc63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2298012840 tx=0x7f2298012d00 comp rx=0 tx=0).stop 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 shutdown_connections 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afcdae70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f2289ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afcc6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbcfa0 0x5595afcc63f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 --2- 172.21.15.60:0/1174210683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbc680 0x5595afcc5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.778+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 >> 172.21.15.60:0/1174210683 conn(0x5595afcba4b0 msgr2=0x5595afcc0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.779+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 shutdown_connections 2024-07-24T21:47:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.779+0000 7f22ae880740 1 -- 172.21.15.60:0/1174210683 wait complete. 2024-07-24T21:47:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.779+0000 7f22ae880740 20 client.15882 trim_cache size 0 max 16384 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.797+0000 7f22ae880740 1 Processor -- start 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 -- start start 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbcfa0 0x5595afce0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afce0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b2bb0 con 0x5595afcbc680 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b0480 con 0x5595afcbcfa0 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22ae880740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ae2d0 con 0x5595afcc8f30 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42898/0 (socket says 172.21.15.60:42898) 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1926318180 learned_addr learned my addr 172.21.15.60:0/1926318180 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbcfa0 0x5595afce0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1926318180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 msgr2=0x5595afce0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afce0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1926318180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbcfa0 msgr2=0x5595afce0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbcfa0 0x5595afce0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1926318180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595afcbcdd0 con 0x5595afcbc680 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.798+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f229c0050b0 tx=0x7f229c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc89b2553612bfec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f228bfff640 1 -- 172.21.15.60:0/1926318180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c035030 con 0x5595afcbc680 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f228bfff640 1 -- 172.21.15.60:0/1926318180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c005440 con 0x5595afcbc680 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f228bfff640 1 -- 172.21.15.60:0/1926318180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c040040 con 0x5595afcbc680 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 -- 172.21.15.60:0/1926318180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 msgr2=0x5595afcc7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f229c0050b0 tx=0x7f229c0058d0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 -- 172.21.15.60:0/1926318180 shutdown_connections 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcc8f30 0x5595afce0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcbcfa0 0x5595afce0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.799+0000 7f22ae880740 1 --2- 172.21.15.60:0/1926318180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcc7bf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- 172.21.15.60:0/1926318180 >> 172.21.15.60:0/1926318180 conn(0x5595afcba4b0 msgr2=0x5595afcc74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- 172.21.15.60:0/1926318180 shutdown_connections 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- 172.21.15.60:0/1926318180 wait complete. 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- start start 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 Processor -- start 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcd1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcc8f30 0x5595afccc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ae940 con 0x5595afcbc680 2024-07-24T21:47:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41afdb0 con 0x5595afcc8f30 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41af010 con 0x5595afcbcfa0 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.800+0000 7f22ae880740 10 client.0 _ll_register_callbacks cb 0x7f22980098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42908/0 (socket says 172.21.15.60:42908) 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1155001250 learned_addr learned my addr 172.21.15.60:0/1155001250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcd1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcc8f30 0x5595afccc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 msgr2=0x5595afcd1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcd1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1155001250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcc8f30 msgr2=0x5595afccc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcc8f30 0x5595afccc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 -- 172.21.15.60:0/1155001250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595afccd050 con 0x5595afcbc680 2024-07-24T21:47:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22aaa6c640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dd561ced0d15f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f2289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c035030 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f229c0049c0 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.801+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c005440 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f229c040070 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595afd433d0 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c005440 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f229c0327a0 con 0x5595afcbc680 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:47:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 20 client.15888 populate_metadata read hostname 'smithi060' 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afd455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 4 client.15888 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595afcc1a80 con 0x5595afcd8a10 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.802+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afd455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f2289ffb640 10 client.15888 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22a9a6a640 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afd455b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e9fed33d0af154 server_cookie=bbb313d2e4eb81b7 in_seq=0 out_seq=0 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f2289ffb640 1 -- 172.21.15.60:0/1155001250 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5595afcc1a80 con 0x5595afcd8a10 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f2289ffb640 10 client.15888 handle_command_reply: tid=0 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 client.15888 shutdown 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 msgr2=0x5595afcd10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c005e20 comp rx=0 tx=0).stop 2024-07-24T21:47:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 shutdown_connections 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5595afcd8a10 0x5595afd455b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595afcc8f30 0x5595afccc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595afcbcfa0 0x5595afcd1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 --2- 172.21.15.60:0/1155001250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5595afcbc680 0x5595afcd10d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 >> 172.21.15.60:0/1155001250 conn(0x5595afcba4b0 msgr2=0x5595afcb9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f2289ffb640 0 client.15888 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 shutdown_connections 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 1 -- 172.21.15.60:0/1155001250 wait complete. 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.803+0000 7f22ae880740 20 client.15888 trim_cache size 0 max 16384 2024-07-24T21:47:59.811 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:47:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:47:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:47:59.812 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:47:59.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 msgr2=0x7f228c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 0x7f228c0b6200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x5595afcbb390 tx=0x7f229400cf90 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 msgr2=0x7f22a419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f229400ebc0 tx=0x7f229400c6a0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 shutdown_connections 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f228c0b3d40 0x7f228c0b6200 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a41079e0 0x7f22a41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22a4107060 0x7f22a419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 --2- 172.21.15.60:0/624201764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22a4101650 0x7f22a419ae20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 >> 172.21.15.60:0/624201764 conn(0x7f22a40fd0c0 msgr2=0x7f22a4105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 shutdown_connections 2024-07-24T21:47:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:47:59.804+0000 7f22ac4cf640 1 -- 172.21.15.60:0/624201764 wait complete. 2024-07-24T21:48:00.573 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:48:00.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:48:00.666 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether build environment is sane... yes 2024-07-24T21:48:00.676 INFO:tasks.workunit.client.0.smithi060.stdout:checking for gawk... gawk 2024-07-24T21:48:00.694 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether make sets $(MAKE)... yes 2024-07-24T21:48:00.735 INFO:tasks.workunit.client.0.smithi060.stdout:checking for gcc... gcc 2024-07-24T21:48:00.787 INFO:tasks.workunit.client.0.smithi060.stdout:checking for C compiler default output file name... a.out 2024-07-24T21:48:00.789 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether the C compiler works... yes 2024-07-24T21:48:00.791 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether we are cross compiling... no 2024-07-24T21:48:00.834 INFO:tasks.workunit.client.0.smithi060.stdout:checking for suffix of executables... 2024-07-24T21:48:00.875 INFO:tasks.workunit.client.0.smithi060.stdout:checking for suffix of object files... o 2024-07-24T21:48:00.918 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether we are using the GNU C compiler... yes 2024-07-24T21:48:00.971 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether gcc accepts -g... yes 2024-07-24T21:48:01.009 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-07-24T21:48:01.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-07-24T21:48:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:01.062 INFO:tasks.workunit.client.0.smithi060.stdout:checking for gcc option to accept ANSI C... none needed 2024-07-24T21:48:01.121 INFO:tasks.workunit.client.0.smithi060.stdout:checking for style of include used by make... GNU 2024-07-24T21:48:01.245 INFO:tasks.workunit.client.0.smithi060.stdout:checking dependency style of gcc... gcc3 2024-07-24T21:48:01.245 INFO:tasks.workunit.client.0.smithi060.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-07-24T21:48:01.379 INFO:tasks.workunit.client.0.smithi060.stdout:checking how to run the C preprocessor... gcc -E 2024-07-24T21:48:01.420 INFO:tasks.workunit.client.0.smithi060.stdout:checking for egrep... grep -E 2024-07-24T21:48:01.428 INFO:tasks.workunit.client.0.smithi060.stdout:checking for AIX... no 2024-07-24T21:48:01.497 INFO:tasks.workunit.client.0.smithi060.stdout:checking for library containing strerror... none required 2024-07-24T21:48:01.544 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:01 smithi060 ceph-mon[33000]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:48:01.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:01 smithi086 ceph-mon[37128]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:48:01.719 INFO:tasks.workunit.client.0.smithi060.stdout:checking for ANSI C header files... yes 2024-07-24T21:48:01.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:01 smithi123 ceph-mon[37143]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T21:48:01.768 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-07-24T21:48:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:01.818 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/types.h... yes 2024-07-24T21:48:01.876 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/stat.h... yes 2024-07-24T21:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:checking for stdlib.h... yes 2024-07-24T21:48:01.972 INFO:tasks.workunit.client.0.smithi060.stdout:checking for string.h... yes 2024-07-24T21:48:02.035 INFO:tasks.workunit.client.0.smithi060.stdout:checking for memory.h... yes 2024-07-24T21:48:02.095 INFO:tasks.workunit.client.0.smithi060.stdout:checking for strings.h... yes 2024-07-24T21:48:02.153 INFO:tasks.workunit.client.0.smithi060.stdout:checking for inttypes.h... yes 2024-07-24T21:48:02.204 INFO:tasks.workunit.client.0.smithi060.stdout:checking for stdint.h... yes 2024-07-24T21:48:02.260 INFO:tasks.workunit.client.0.smithi060.stdout:checking for unistd.h... yes 2024-07-24T21:48:02.271 INFO:tasks.workunit.client.0.smithi060.stdout:checking for memory.h... (cached) yes 2024-07-24T21:48:02.277 INFO:tasks.workunit.client.0.smithi060.stdout:checking for strings.h... (cached) yes 2024-07-24T21:48:02.282 INFO:tasks.workunit.client.0.smithi060.stdout:checking for unistd.h... (cached) yes 2024-07-24T21:48:02.363 INFO:tasks.workunit.client.0.smithi060.stdout:checking locale.h usability... yes 2024-07-24T21:48:02.382 INFO:tasks.workunit.client.0.smithi060.stdout:checking locale.h presence... yes 2024-07-24T21:48:02.383 INFO:tasks.workunit.client.0.smithi060.stdout:checking for locale.h... yes 2024-07-24T21:48:02.397 INFO:tasks.workunit.client.0.smithi060.stdout:checking for stdlib.h... (cached) yes 2024-07-24T21:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:checking for string.h... (cached) yes 2024-07-24T21:48:02.498 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/socket.h usability... yes 2024-07-24T21:48:02.519 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/socket.h presence... yes 2024-07-24T21:48:02.520 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/socket.h... yes 2024-07-24T21:48:02.575 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/time.h usability... yes 2024-07-24T21:48:02.596 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/time.h presence... yes 2024-07-24T21:48:02.597 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/time.h... yes 2024-07-24T21:48:02.651 INFO:tasks.workunit.client.0.smithi060.stdout:checking fcntl.h usability... yes 2024-07-24T21:48:02.672 INFO:tasks.workunit.client.0.smithi060.stdout:checking fcntl.h presence... yes 2024-07-24T21:48:02.672 INFO:tasks.workunit.client.0.smithi060.stdout:checking for fcntl.h... yes 2024-07-24T21:48:02.764 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/fcntl.h usability... yes 2024-07-24T21:48:02.784 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/fcntl.h presence... yes 2024-07-24T21:48:02.785 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/fcntl.h... yes 2024-07-24T21:48:02.885 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/param.h usability... yes 2024-07-24T21:48:02.906 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/param.h presence... yes 2024-07-24T21:48:02.907 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/param.h... yes 2024-07-24T21:48:02.959 INFO:tasks.workunit.client.0.smithi060.stdout:checking ioctl.h usability... no 2024-07-24T21:48:02.980 INFO:tasks.workunit.client.0.smithi060.stdout:checking ioctl.h presence... no 2024-07-24T21:48:02.980 INFO:tasks.workunit.client.0.smithi060.stdout:checking for ioctl.h... no 2024-07-24T21:48:03.031 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/ioctl.h usability... yes 2024-07-24T21:48:03.051 INFO:tasks.workunit.client.0.smithi060.stdout:checking sys/ioctl.h presence... yes 2024-07-24T21:48:03.052 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sys/ioctl.h... yes 2024-07-24T21:48:03.102 INFO:tasks.workunit.client.0.smithi060.stdout:checking return type of signal handlers... void 2024-07-24T21:48:03.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.106+0000 7fe284009640 1 -- 172.21.15.60:0/3050216779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 msgr2=0x7fe27c1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:03.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.106+0000 7fe284009640 1 --2- 172.21.15.60:0/3050216779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c1015f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe278009900 tx=0x7fe27802a410 comp rx=0 tx=0).stop 2024-07-24T21:48:03.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 -- 172.21.15.60:0/3050216779 shutdown_connections 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 --2- 172.21.15.60:0/3050216779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 0x7fe27c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 --2- 172.21.15.60:0/3050216779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 --2- 172.21.15.60:0/3050216779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c1015f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 -- 172.21.15.60:0/3050216779 >> 172.21.15.60:0/3050216779 conn(0x7fe27c0fd0c0 msgr2=0x7fe27c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 -- 172.21.15.60:0/3050216779 shutdown_connections 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 -- 172.21.15.60:0/3050216779 wait complete. 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 Processor -- start 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 -- start start 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.107+0000 7fe284009640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe284009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 0x7fe27c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe284009640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c114940 con 0x7fe27c101210 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe284009640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1147c0 con 0x7fe27c10edc0 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe284009640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c114ac0 con 0x7fe27c101b30 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42924/0 (socket says 172.21.15.60:42924) 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 -- 172.21.15.60:0/4281497412 learned_addr learned my addr 172.21.15.60:0/4281497412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe2815a4640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe2825a6640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 0x7fe27c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 -- 172.21.15.60:0/4281497412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 msgr2=0x7fe27c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 -- 172.21.15.60:0/4281497412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 msgr2=0x7fe27c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 0x7fe27c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 -- 172.21.15.60:0/4281497412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c1a0f20 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe2815a4640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe281da5640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe278005940 tx=0x7fe278004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27803f030 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.108+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe278003aa0 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.109+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c1a11b0 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.109+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278035070 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.109+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1a78d0 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.110+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c1015f0 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.111+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe278042030 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.112+0000 7fe272ffd640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 0x7fe2680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.112+0000 7fe2815a4640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 0x7fe2680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.112+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe278104940 con 0x7fe27c101210 2024-07-24T21:48:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.113+0000 7fe2815a4640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 0x7fe2680b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe27c111440 tx=0x7fe26c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:03.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.119+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe2780cbc50 con 0x7fe27c101210 2024-07-24T21:48:03.150 INFO:tasks.workunit.client.0.smithi060.stdout:checking for size_t... yes 2024-07-24T21:48:03.187 INFO:tasks.workunit.client.0.smithi060.stdout:checking for dev_t... yes 2024-07-24T21:48:03.230 INFO:tasks.workunit.client.0.smithi060.stdout:checking for short... yes 2024-07-24T21:48:03.307 INFO:tasks.workunit.client.0.smithi060.stdout:checking size of short... 2 2024-07-24T21:48:03.372 INFO:tasks.workunit.client.0.smithi060.stdout:checking for int... yes 2024-07-24T21:48:03.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.372+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe27c1a7b70 con 0x7fe27c101210 2024-07-24T21:48:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.373+0000 7fe272ffd640 1 -- 172.21.15.60:0/4281497412 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fe2780cb550 con 0x7fe27c101210 2024-07-24T21:48:03.375 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:48:03.375 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:48:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 msgr2=0x7fe2680b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 0x7fe2680b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe27c111440 tx=0x7fe26c005f90 comp rx=0 tx=0).stop 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 msgr2=0x7fe27c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe278005940 tx=0x7fe278004270 comp rx=0 tx=0).stop 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 shutdown_connections 2024-07-24T21:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe2680b3d40 0x7fe2680b6200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c10edc0 0x7fe27c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.375+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27c101b30 0x7fe27c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.376+0000 7fe284009640 1 --2- 172.21.15.60:0/4281497412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe27c101210 0x7fe27c19ae70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.376+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 >> 172.21.15.60:0/4281497412 conn(0x7fe27c0fd0c0 msgr2=0x7fe27c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.376+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 shutdown_connections 2024-07-24T21:48:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:03.376+0000 7fe284009640 1 -- 172.21.15.60:0/4281497412 wait complete. 2024-07-24T21:48:03.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:03 smithi060 ceph-mon[33000]: pgmap v418: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-07-24T21:48:03.442 INFO:tasks.workunit.client.0.smithi060.stdout:checking size of int... 4 2024-07-24T21:48:03.495 INFO:tasks.workunit.client.0.smithi060.stdout:checking for long... yes 2024-07-24T21:48:03.544 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:03 smithi123 ceph-mon[37143]: pgmap v418: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-07-24T21:48:03.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:03 smithi086 ceph-mon[37128]: pgmap v418: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-07-24T21:48:03.564 INFO:tasks.workunit.client.0.smithi060.stdout:checking size of long... 8 2024-07-24T21:48:03.611 INFO:tasks.workunit.client.0.smithi060.stdout:checking for long long... yes 2024-07-24T21:48:03.681 INFO:tasks.workunit.client.0.smithi060.stdout:checking size of long long... 8 2024-07-24T21:48:03.737 INFO:tasks.workunit.client.0.smithi060.stdout:checking for ssize_t... yes 2024-07-24T21:48:03.777 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sig_atomic_t... yes 2024-07-24T21:48:03.860 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether gcc needs -traditional... no 2024-07-24T21:48:03.900 INFO:tasks.workunit.client.0.smithi060.stdout:checking for an ANSI C-conforming const... yes 2024-07-24T21:48:03.927 INFO:tasks.workunit.client.0.smithi060.stdout:checking for inline... inline 2024-07-24T21:48:03.989 INFO:tasks.workunit.client.0.smithi060.stdout:checking for working alloca.h... yes 2024-07-24T21:48:04.053 INFO:tasks.workunit.client.0.smithi060.stdout:checking for alloca... yes 2024-07-24T21:48:04.121 INFO:tasks.workunit.client.0.smithi060.stdout:checking for working memcmp... yes 2024-07-24T21:48:04.176 INFO:tasks.workunit.client.0.smithi060.stdout:checking for strftime... yes 2024-07-24T21:48:04.266 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-07-24T21:48:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.268+0000 7f47ea7ae640 1 -- 172.21.15.60:0/1065283383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e41035c0 msgr2=0x7f47e41039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.268+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/1065283383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e41035c0 0x7f47e41039a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009900 tx=0x7f47d402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 -- 172.21.15.60:0/1065283383 shutdown_connections 2024-07-24T21:48:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/1065283383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e410ca10 0x7f47e410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/1065283383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 0x7f47e410c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/1065283383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e41035c0 0x7f47e41039a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 -- 172.21.15.60:0/1065283383 >> 172.21.15.60:0/1065283383 conn(0x7f47e40fd010 msgr2=0x7f47e40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 -- 172.21.15.60:0/1065283383 shutdown_connections 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 -- 172.21.15.60:0/1065283383 wait complete. 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 Processor -- start 2024-07-24T21:48:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.269+0000 7f47ea7ae640 1 -- start start 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 0x7f47e41017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e410ca10 0x7f47e4101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41126a0 con 0x7f47e410ca10 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4112520 con 0x7f47e4103ee0 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47ea7ae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4112820 con 0x7f47e41035c0 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e410ca10 0x7f47e4101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34028/0 (socket says 172.21.15.60:34028) 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 -- 172.21.15.60:0/2710672993 learned_addr learned my addr 172.21.15.60:0/2710672993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 0x7f47e41017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 -- 172.21.15.60:0/2710672993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 msgr2=0x7f47e41017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 0x7f47e41017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 -- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e410ca10 msgr2=0x7f47e4101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e410ca10 0x7f47e4101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 -- 172.21.15.60:0/2710672993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41a7050 con 0x7f47e41035c0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.270+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005940 tx=0x7f47d40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.271+0000 7f47e17fa640 1 -- 172.21.15.60:0/2710672993 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d403f030 con 0x7f47e41035c0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.271+0000 7f47e17fa640 1 -- 172.21.15.60:0/2710672993 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d4003cb0 con 0x7f47e41035c0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.271+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41a72e0 con 0x7f47e41035c0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.271+0000 7f47e17fa640 1 -- 172.21.15.60:0/2710672993 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4035070 con 0x7f47e41035c0 2024-07-24T21:48:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.271+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41a77d0 con 0x7f47e41035c0 2024-07-24T21:48:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.273+0000 7f47e17fa640 1 -- 172.21.15.60:0/2710672993 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f47d4042030 con 0x7f47e41035c0 2024-07-24T21:48:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.273+0000 7f47ed5e4740 1 Processor -- start 2024-07-24T21:48:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.273+0000 7f47ed5e4740 1 -- start start 2024-07-24T21:48:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564c05820280 con 0x564c058ab1e0 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c0580f780 con 0x564c058b7170 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564c058137c0 con 0x564c058aa8c0 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34036/0 (socket says 172.21.15.60:34036) 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 -- 172.21.15.60:0/2624959285 learned_addr learned my addr 172.21.15.60:0/2624959285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 -- 172.21.15.60:0/2624959285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 msgr2=0x564c058b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e17fa640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 0x7f47cc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 -- 172.21.15.60:0/2624959285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 msgr2=0x564c058b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 -- 172.21.15.60:0/2624959285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c05573790 con 0x564c058aa8c0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e17fa640 1 -- 172.21.15.60:0/2710672993 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f47d4104930 con 0x7f47e41035c0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.274+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 0x7f47cc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.275+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47dc010370 tx=0x7f47dc010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db03b95e370b41e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.275+0000 7f47ca7fc640 1 -- 172.21.15.60:0/2624959285 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc022070 con 0x564c058aa8c0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.275+0000 7f47ca7fc640 1 -- 172.21.15.60:0/2624959285 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0113e0 con 0x564c058aa8c0 2024-07-24T21:48:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.275+0000 7f47ca7fc640 1 -- 172.21.15.60:0/2624959285 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc016070 con 0x564c058aa8c0 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 0x7f47cc0b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47dc01f000 tx=0x7f47dc022000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2624959285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 msgr2=0x564c058aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47dc010370 tx=0x7f47dc010830 comp rx=0 tx=0).stop 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2624959285 shutdown_connections 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2624959285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058aaca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2624959285 >> 172.21.15.60:0/2624959285 conn(0x564c058a86f0 msgr2=0x564c058a9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2624959285 shutdown_connections 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2624959285 wait complete. 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 -- start start 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.276+0000 7f47ed5e4740 1 Processor -- start 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564c05749ab0 con 0x564c058ab1e0 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c057979e0 con 0x564c058b7170 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564c058b9e50 con 0x564c058aa8c0 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47ed5e4740 10 client.0 _ll_register_callbacks cb 0x564c0551fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42956/0 (socket says 172.21.15.60:42956) 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e37fe640 1 -- 172.21.15.60:0/52512608 learned_addr learned my addr 172.21.15.60:0/52512608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 -- 172.21.15.60:0/52512608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 msgr2=0x564c058b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 -- 172.21.15.60:0/52512608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 msgr2=0x564c058b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e3fff640 1 -- 172.21.15.60:0/52512608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c055ec620 con 0x564c058aa8c0 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.277+0000 7f47e37fe640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.278+0000 7f47e3fff640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058b6910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47d4102020 tx=0x7f47d402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fb7b2600b8d1d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.278+0000 7f47c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4049070 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f47d41022a0 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c057bece0 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47d40cae30 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47ed5e4740 10 client.25438 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c058c5630 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564c057bece0 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.280+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f47d403bbf0 con 0x564c058aa8c0 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 10 client.25438 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 20 client.25438 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c058c6ae0 0x564c058c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 4 client.25438 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c058221d0 con 0x564c058c6ae0 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47e37fe640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c058c6ae0 0x564c058c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47c37fe640 10 client.25438 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.281+0000 7f47e37fe640 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c058c6ae0 0x564c058c8f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d827e97ce81c674 server_cookie=88a4eb99e43e4039 in_seq=0 out_seq=0 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47c37fe640 1 -- 172.21.15.60:0/52512608 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c058221d0 con 0x564c058c6ae0 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47c37fe640 10 client.25438 handle_command_reply: tid=0 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 client.25438 shutdown 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 msgr2=0x564c058b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058b6910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47d4102020 tx=0x7f47d402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 shutdown_connections 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c058c6ae0 0x564c058c8f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058b7170 0x564c058b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058ab1e0 0x564c058b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47c37fe640 0 client.25438 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/52512608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058aa8c0 0x564c058b6910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 >> 172.21.15.60:0/52512608 conn(0x564c058a86f0 msgr2=0x564c058ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 shutdown_connections 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 1 -- 172.21.15.60:0/52512608 wait complete. 2024-07-24T21:48:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.282+0000 7f47ed5e4740 20 client.25438 trim_cache size 0 max 16384 2024-07-24T21:48:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 Processor -- start 2024-07-24T21:48:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 -- start start 2024-07-24T21:48:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b2bb0 con 0x564c058b7170 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b0480 con 0x564c058aa8c0 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41ae2d0 con 0x564c058ab1e0 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42960/0 (socket says 172.21.15.60:42960) 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47e8d4b640 1 -- 172.21.15.60:0/3792017612 learned_addr learned my addr 172.21.15.60:0/3792017612 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47e3fff640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.298+0000 7f47e37fe640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 -- 172.21.15.60:0/3792017612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 msgr2=0x564c058b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 -- 172.21.15.60:0/3792017612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 msgr2=0x564c058c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 -- 172.21.15.60:0/3792017612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c058b22d0 con 0x564c058b7170 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e3fff640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47dc011dc0 tx=0x7f47dc027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754e580cffdd179 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47c9ffb640 1 -- 172.21.15.60:0/3792017612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc016040 con 0x564c058b7170 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47c9ffb640 1 -- 172.21.15.60:0/3792017612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0129f0 con 0x564c058b7170 2024-07-24T21:48:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.299+0000 7f47c9ffb640 1 -- 172.21.15.60:0/3792017612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0116d0 con 0x564c058b7170 2024-07-24T21:48:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- 172.21.15.60:0/3792017612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 msgr2=0x564c058b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47dc011dc0 tx=0x7f47dc027000 comp rx=0 tx=0).stop 2024-07-24T21:48:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- 172.21.15.60:0/3792017612 shutdown_connections 2024-07-24T21:48:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c058b2b90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/3792017612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- 172.21.15.60:0/3792017612 >> 172.21.15.60:0/3792017612 conn(0x564c058a86f0 msgr2=0x564c058a93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- 172.21.15.60:0/3792017612 shutdown_connections 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- 172.21.15.60:0/3792017612 wait complete. 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 -- start start 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.300+0000 7f47ed5e4740 1 Processor -- start 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058ba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41ae940 con 0x564c058b7170 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41afdb0 con 0x564c058aa8c0 2024-07-24T21:48:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41af010 con 0x564c058ab1e0 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47ed5e4740 10 client.0 _ll_register_callbacks cb 0x7f47d4005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e8d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42962/0 (socket says 172.21.15.60:42962) 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e8d4b640 1 -- 172.21.15.60:0/2090124051 learned_addr learned my addr 172.21.15.60:0/2090124051 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058ba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 -- 172.21.15.60:0/2090124051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 msgr2=0x564c058ba2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058ba2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 -- 172.21.15.60:0/2090124051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 msgr2=0x564c0592f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 -- 172.21.15.60:0/2090124051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0592fcb0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058ba2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e8d4b640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47e37fe640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058ba7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x564c058b7090 tx=0x7f47d8010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5618bd9e8c304a3a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.301+0000 7f47c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8004220 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47dc0129f0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f47d80043c0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47d80047c0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 10 client.25444 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c05930ec0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0129f0 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f47d8024650 con 0x564c058ab1e0 2024-07-24T21:48:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 10 client.25444 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 20 client.25444 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c059315f0 0x564c05933a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 4 client.25444 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.302+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c0581dc70 con 0x564c059315f0 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.303+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c059315f0 0x564c05933a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.303+0000 7f47c37fe640 10 client.25444 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.303+0000 7f47e3fff640 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c059315f0 0x564c05933a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b1eee7feb07f63 server_cookie=86efcf823883d788 in_seq=0 out_seq=0 2024-07-24T21:48:04.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:04 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4281497412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:04.349 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether stat accepts an empty string... no 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47c37fe640 1 -- 172.21.15.60:0/2090124051 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c0581dc70 con 0x564c059315f0 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47c37fe640 10 client.25444 handle_command_reply: tid=0 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 client.25444 shutdown 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 msgr2=0x564c058ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058ba7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x564c058b7090 tx=0x7f47d8010490 comp rx=0 tx=0).stop 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 shutdown_connections 2024-07-24T21:48:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564c059315f0 0x564c05933a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564c058b7170 0x564c0592f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564c058ab1e0 0x564c058ba7f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47c37fe640 0 client.25444 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 --2- 172.21.15.60:0/2090124051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c058aa8c0 0x564c058ba2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 >> 172.21.15.60:0/2090124051 conn(0x564c058a86f0 msgr2=0x564c058b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 shutdown_connections 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 1 -- 172.21.15.60:0/2090124051 wait complete. 2024-07-24T21:48:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.357+0000 7f47ed5e4740 20 client.25444 trim_cache size 0 max 16384 2024-07-24T21:48:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:48:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:48:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:48:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 msgr2=0x7f47cc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 0x7f47cc0b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47dc01f000 tx=0x7f47dc022000 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 msgr2=0x7f47e4101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005940 tx=0x7f47d40042f0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 shutdown_connections 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47cc0b3d40 0x7f47cc0b6200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47e410ca10 0x7f47e4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47e4103ee0 0x7f47e41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 --2- 172.21.15.60:0/2710672993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e41035c0 0x7f47e4101260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 >> 172.21.15.60:0/2710672993 conn(0x7f47e40fd010 msgr2=0x7f47e40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 shutdown_connections 2024-07-24T21:48:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:04.358+0000 7f47ea7ae640 1 -- 172.21.15.60:0/2710672993 wait complete. 2024-07-24T21:48:04.409 INFO:tasks.workunit.client.0.smithi060.stdout:checking for vprintf... yes 2024-07-24T21:48:04.477 INFO:tasks.workunit.client.0.smithi060.stdout:checking for _doprnt... no 2024-07-24T21:48:04.501 INFO:tasks.workunit.client.0.smithi060.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-07-24T21:48:04.513 INFO:tasks.workunit.client.0.smithi060.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-07-24T21:48:04.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:04 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4281497412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:04.631 INFO:tasks.workunit.client.0.smithi060.stdout:checking for the pthreads library -lpthreads... no 2024-07-24T21:48:04.687 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether pthreads work without any flags... yes 2024-07-24T21:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-07-24T21:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:checking if more special flags are required for pthreads... no 2024-07-24T21:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:checking for cc_r... gcc 2024-07-24T21:48:04.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:04 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4281497412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:04.822 INFO:tasks.workunit.client.0.smithi060.stdout:checking for setrlimit... yes 2024-07-24T21:48:04.883 INFO:tasks.workunit.client.0.smithi060.stdout:checking for setproctitle... no 2024-07-24T21:48:04.941 INFO:tasks.workunit.client.0.smithi060.stdout:checking for getopt_long... yes 2024-07-24T21:48:05.001 INFO:tasks.workunit.client.0.smithi060.stdout:checking for strtoull... yes 2024-07-24T21:48:05.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:48:05.067 INFO:tasks.workunit.client.0.smithi060.stdout:checking for strtoq... yes 2024-07-24T21:48:05.128 INFO:tasks.workunit.client.0.smithi060.stdout:checking for snprintf... yes 2024-07-24T21:48:05.189 INFO:tasks.workunit.client.0.smithi060.stdout:checking for vsnprintf... yes 2024-07-24T21:48:05.251 INFO:tasks.workunit.client.0.smithi060.stdout:checking for vfprintf... yes 2024-07-24T21:48:05.318 INFO:tasks.workunit.client.0.smithi060.stdout:checking for gethostname... yes 2024-07-24T21:48:05.382 INFO:tasks.workunit.client.0.smithi060.stdout:checking for setlocale... yes 2024-07-24T21:48:05.454 INFO:tasks.workunit.client.0.smithi060.stdout:checking for sync... yes 2024-07-24T21:48:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:05 smithi086 ceph-mon[37128]: pgmap v419: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-07-24T21:48:05.546 INFO:tasks.workunit.client.0.smithi060.stdout:checking whether snprintf is C99 conformant... done 2024-07-24T21:48:05.556 INFO:tasks.workunit.client.0.smithi060.stdout:configure: creating ./config.status 2024-07-24T21:48:05.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:05 smithi060 ceph-mon[33000]: pgmap v419: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-07-24T21:48:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 -- 172.21.15.60:0/3952251980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103fa0 msgr2=0x7f982410c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3952251980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103fa0 0x7f982410c490 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9814009900 tx=0x7f981402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 -- 172.21.15.60:0/3952251980 shutdown_connections 2024-07-24T21:48:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3952251980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982410cae0 0x7f982410ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3952251980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103fa0 0x7f982410c490 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3952251980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9824103680 0x7f9824103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 -- 172.21.15.60:0/3952251980 >> 172.21.15.60:0/3952251980 conn(0x7f98240fd0c0 msgr2=0x7f98240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 -- 172.21.15.60:0/3952251980 shutdown_connections 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.605+0000 7f982b4ac640 1 -- 172.21.15.60:0/3952251980 wait complete. 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 Processor -- start 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 -- start start 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 0x7f9824101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982410cae0 0x7f9824101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9824112730 con 0x7f9824103680 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98241125b0 con 0x7f9824103fa0 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f982b4ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98241128b0 con 0x7f982410cae0 2024-07-24T21:48:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f9829248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f9829248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42972/0 (socket says 172.21.15.60:42972) 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f9829248640 1 -- 172.21.15.60:0/3696757236 learned_addr learned my addr 172.21.15.60:0/3696757236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.606+0000 7f9829a49640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982410cae0 0x7f9824101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9828a47640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 0x7f9824101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 -- 172.21.15.60:0/3696757236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982410cae0 msgr2=0x7f9824101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982410cae0 0x7f9824101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 -- 172.21.15.60:0/3696757236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 msgr2=0x7f9824101870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 0x7f9824101870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 -- 172.21.15.60:0/3696757236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98241a70a0 con 0x7f9824103680 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9828a47640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 0x7f9824101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f9829248640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f982000e9f0 tx=0x7f982000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f981a7fc640 1 -- 172.21.15.60:0/3696757236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982000cd80 con 0x7f9824103680 2024-07-24T21:48:05.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98241a7390 con 0x7f9824103680 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f981a7fc640 1 -- 172.21.15.60:0/3696757236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98200054d0 con 0x7f9824103680 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f981a7fc640 1 -- 172.21.15.60:0/3696757236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820010690 con 0x7f9824103680 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.607+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98241a7850 con 0x7f9824103680 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 Processor -- start 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 -- start start 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f981a7fc640 1 -- 172.21.15.60:0/3696757236 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9820004e90 con 0x7f9824103680 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x56228779e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562287713610 con 0x56228779e630 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562287702b10 con 0x56228779dd10 2024-07-24T21:48:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f982d85d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562287706b50 con 0x5622877aa5c0 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f9829248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f9829248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42984/0 (socket says 172.21.15.60:42984) 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f9829248640 1 -- 172.21.15.60:0/2360811479 learned_addr learned my addr 172.21.15.60:0/2360811479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f9829a49640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x56228779e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.610+0000 7f9828a47640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 -- 172.21.15.60:0/2360811479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 msgr2=0x5622877ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 -- 172.21.15.60:0/2360811479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 msgr2=0x5622877aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 -- 172.21.15.60:0/2360811479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562287698320 con 0x56228779dd10 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829248640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829a49640 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x56228779e0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f981c005fd0 tx=0x7f981c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf15a2f9eeaa6f2c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f981a7fc640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 0x7f98100b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f980f7fe640 1 -- 172.21.15.60:0/2360811479 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c016070 con 0x56228779dd10 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f9829248640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 0x7f98100b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f980f7fe640 1 -- 172.21.15.60:0/2360811479 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981c004d20 con 0x56228779dd10 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f980f7fe640 1 -- 172.21.15.60:0/2360811479 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0127c0 con 0x56228779dd10 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.611+0000 7f981a7fc640 1 -- 172.21.15.60:0/3696757236 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f98200e4490 con 0x7f9824103680 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- 172.21.15.60:0/2360811479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 msgr2=0x56228779e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f9829248640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 0x7f98100b6200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98200a7120 tx=0x7f98200e9040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x56228779e0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f981c005fd0 tx=0x7f981c010740 comp rx=0 tx=0).stop 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- 172.21.15.60:0/2360811479 shutdown_connections 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 --2- 172.21.15.60:0/2360811479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x56228779e0f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- 172.21.15.60:0/2360811479 >> 172.21.15.60:0/2360811479 conn(0x56228779bb40 msgr2=0x56228779cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- 172.21.15.60:0/2360811479 shutdown_connections 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- 172.21.15.60:0/2360811479 wait complete. 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.612+0000 7f982d85d740 1 -- start start 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 Processor -- start 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x5622877a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5622875667b0 con 0x56228779e630 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622876a5130 con 0x56228779dd10 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622877ad2e0 con 0x5622877aa5c0 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f982d85d740 10 client.0 _ll_register_callbacks cb 0x5622876fb660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f9828a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f9828a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42992/0 (socket says 172.21.15.60:42992) 2024-07-24T21:48:05.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f9828a47640 1 -- 172.21.15.60:0/1406593339 learned_addr learned my addr 172.21.15.60:0/1406593339 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f9829a49640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x5622877a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.613+0000 7f9829a49640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 -- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 msgr2=0x5622877a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 -- 172.21.15.60:0/1406593339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 msgr2=0x5622877a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x5622877a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 -- 172.21.15.60:0/1406593339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622876a8e80 con 0x56228779e630 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f9828a47640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9814009900 tx=0x7f9814031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47b8cc264c0fe0b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f980cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814035040 con 0x56228779e630 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9814004040 con 0x56228779e630 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f981c004d20 con 0x56228779e630 2024-07-24T21:48:05.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9814004240 con 0x56228779e630 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f982d85d740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.614+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562287472ba0 con 0x56228779e630 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981c004d20 con 0x56228779e630 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f98140049f0 con 0x56228779e630 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 10 client.15924 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 20 client.15924 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877ba190 0x5622877bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 4 client.15924 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562287715560 con 0x5622877ba190 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f9829a49640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877ba190 0x5622877bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f9829a49640 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877ba190 0x5622877bc580 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df50662ab19f901 server_cookie=f1896df72504c3e in_seq=0 out_seq=0 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.615+0000 7f980cff9640 10 client.15924 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f980cff9640 1 -- 172.21.15.60:0/1406593339 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562287715560 con 0x5622877ba190 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f980cff9640 10 client.15924 handle_command_reply: tid=0 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 client.15924 shutdown 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 msgr2=0x5622877a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9814009900 tx=0x7f9814031750 comp rx=0 tx=0).stop 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 shutdown_connections 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877ba190 0x5622877bc580 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622877aa5c0 0x5622877a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f980cff9640 0 client.15924 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779e630 0x5622877a7a70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 --2- 172.21.15.60:0/1406593339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779dd10 0x5622877a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.616+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 >> 172.21.15.60:0/1406593339 conn(0x56228779bb40 msgr2=0x5622877a16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.617+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 shutdown_connections 2024-07-24T21:48:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.617+0000 7f982d85d740 1 -- 172.21.15.60:0/1406593339 wait complete. 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.617+0000 7f982d85d740 20 client.15924 trim_cache size 0 max 16384 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.635+0000 7f982d85d740 1 Processor -- start 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.635+0000 7f982d85d740 1 -- start start 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779e630 0x5622877b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98241b2c30 con 0x56228779bb40 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98241b0500 con 0x56228779e630 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f982d85d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98241ae350 con 0x56228779dd10 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9829a49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34092/0 (socket says 172.21.15.60:34092) 2024-07-24T21:48:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 -- 172.21.15.60:0/1981370419 learned_addr learned my addr 172.21.15.60:0/1981370419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9829a49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42996/0 (socket says 172.21.15.60:42996) 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9829a49640 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779e630 0x5622877b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 -- 172.21.15.60:0/1981370419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779e630 msgr2=0x5622877b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779e630 0x5622877b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 -- 172.21.15.60:0/1981370419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 msgr2=0x5622877c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 -- 172.21.15.60:0/1981370419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56228779d590 con 0x56228779dd10 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9829a49640 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.636+0000 7f9828a47640 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9814031440 tx=0x7f9814032fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2092b525fd1b320c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f980effd640 1 -- 172.21.15.60:0/1981370419 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814052070 con 0x56228779dd10 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f980effd640 1 -- 172.21.15.60:0/1981370419 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98140037f0 con 0x56228779dd10 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f980effd640 1 -- 172.21.15.60:0/1981370419 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814035040 con 0x56228779dd10 2024-07-24T21:48:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 -- 172.21.15.60:0/1981370419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 msgr2=0x5622877c2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9814031440 tx=0x7f9814032fb0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 -- 172.21.15.60:0/1981370419 shutdown_connections 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779e630 0x5622877b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779dd10 0x5622877c2390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 --2- 172.21.15.60:0/1981370419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779bb40 0x5622877c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 -- 172.21.15.60:0/1981370419 >> 172.21.15.60:0/1981370419 conn(0x56228779cfa0 msgr2=0x5622877a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 -- 172.21.15.60:0/1981370419 shutdown_connections 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.637+0000 7f982d85d740 1 -- 172.21.15.60:0/1981370419 wait complete. 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 -- start start 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 Processor -- start 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779bb40 0x5622877b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98241ae9c0 con 0x56228779dd10 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98241afe30 con 0x56228779bb40 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98241af090 con 0x56228779e630 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.638+0000 7f982d85d740 10 client.0 _ll_register_callbacks cb 0x7f9820005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779bb40 0x5622877b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9828a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34104/0 (socket says 172.21.15.60:34104) 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 -- 172.21.15.60:0/697915751 learned_addr learned my addr 172.21.15.60:0/697915751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9828a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43012/0 (socket says 172.21.15.60:43012) 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 -- 172.21.15.60:0/697915751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779bb40 msgr2=0x5622877b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779bb40 0x5622877b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 -- 172.21.15.60:0/697915751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 msgr2=0x5622877af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 -- 172.21.15.60:0/697915751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622877aff30 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9828a47640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f9829a49640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098b0 tx=0x7f981c009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7404416db875e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f980cff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c016040 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f981c003970 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.639+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98140037f0 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f981c012720 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 10 client.25462 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562287825eb0 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98140037f0 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f981c028040 con 0x56228779e630 2024-07-24T21:48:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 10 client.25462 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 10 client.25462 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 20 client.25462 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877aa5c0 0x562287828460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 4 client.25462 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.640+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x562287711000 con 0x5622877aa5c0 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.641+0000 7f9828a47640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877aa5c0 0x562287828460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.641+0000 7f980cff9640 10 client.25462 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:05.641+0000 7f9828a47640 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877aa5c0 0x562287828460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce9253ca9756721 server_cookie=f6be7e8acdd5846a in_seq=0 out_seq=0 2024-07-24T21:48:05.680 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:05.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:05 smithi123 ceph-mon[37143]: pgmap v419: 97 pgs: 97 active+clean; 1.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-07-24T21:48:07.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:07 smithi060 ceph-mon[33000]: pgmap v420: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 96 KiB/s wr, 48 op/s 2024-07-24T21:48:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:07 smithi086 ceph-mon[37128]: pgmap v420: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 96 KiB/s wr, 48 op/s 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- 172.21.15.60:0/3958716242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c074eb0 msgr2=0x7f6b0c075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3958716242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c075290 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc02a410 comp rx=0 tx=0).stop 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- 172.21.15.60:0/3958716242 shutdown_connections 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3958716242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c10eda0 0x7f6b0c111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3958716242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3958716242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c075290 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- 172.21.15.60:0/3958716242 >> 172.21.15.60:0/3958716242 conn(0x7f6b0c0fe200 msgr2=0x7f6b0c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- 172.21.15.60:0/3958716242 shutdown_connections 2024-07-24T21:48:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- 172.21.15.60:0/3958716242 wait complete. 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 Processor -- start 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.547+0000 7f6b1280b640 1 -- start start 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c19afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c10eda0 0x7f6b0c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c114980 con 0x7f6b0c0757d0 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c114800 con 0x7f6b0c074eb0 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b1280b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c114b00 con 0x7f6b0c10eda0 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51472/0 (socket says 172.21.15.60:51472) 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/3157399147 learned_addr learned my addr 172.21.15.60:0/3157399147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c19afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c10eda0 0x7f6b0c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/3157399147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c10eda0 msgr2=0x7f6b0c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c10eda0 0x7f6b0c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/3157399147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c074eb0 msgr2=0x7f6b0c19afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c19afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/3157399147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c1a1080 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d950 tx=0x7f6b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.548+0000 7f6b097fa640 1 -- 172.21.15.60:0/3157399147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00017070 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.549+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c1a1370 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.549+0000 7f6b097fa640 1 -- 172.21.15.60:0/3157399147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b00009c30 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.549+0000 7f6b097fa640 1 -- 172.21.15.60:0/3157399147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000f620 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.549+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c1a7a30 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b097fa640 1 -- 172.21.15.60:0/3157399147 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6b00010460 con 0x7f6b0c0757d0 2024-07-24T21:48:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 Processor -- start 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 -- start start 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 0x55b6a27abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b6a27229b0 con 0x55b6a27b7f80 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6a2711eb0 con 0x55b6a27ab6d0 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6a2715ef0 con 0x55b6a27abff0 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.551+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51480/0 (socket says 172.21.15.60:51480) 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 0x55b6a27abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/3926673487 learned_addr learned my addr 172.21.15.60:0/3926673487 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b097fa640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 0x7f6af40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 -- 172.21.15.60:0/3926673487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 msgr2=0x55b6a27b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 -- 172.21.15.60:0/3926673487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 msgr2=0x55b6a27ba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 -- 172.21.15.60:0/3926673487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a269b320 con 0x55b6a27ab6d0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b097fa640 1 -- 172.21.15.60:0/3157399147 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6b000e3b90 con 0x7f6b0c0757d0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 0x7f6af40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 0x55b6a27abab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c06bc40 tx=0x7f6b04011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc267a27d3a539df server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.552+0000 7f6af27fc640 1 -- 172.21.15.60:0/3926673487 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04004250 con 0x55b6a27ab6d0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6af27fc640 1 -- 172.21.15.60:0/3926673487 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0401be00 con 0x55b6a27ab6d0 2024-07-24T21:48:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6af27fc640 1 -- 172.21.15.60:0/3926673487 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04019a20 con 0x55b6a27ab6d0 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b10da8640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 0x7f6af40b6200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b04020000 tx=0x7f6b04021000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 -- 172.21.15.60:0/3926673487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 msgr2=0x55b6a27abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 0x55b6a27abab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c06bc40 tx=0x7f6b04011810 comp rx=0 tx=0).stop 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 -- 172.21.15.60:0/3926673487 shutdown_connections 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/3926673487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27ab6d0 0x55b6a27abab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 -- 172.21.15.60:0/3926673487 >> 172.21.15.60:0/3926673487 conn(0x55b6a27a9500 msgr2=0x55b6a27aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 -- 172.21.15.60:0/3926673487 shutdown_connections 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.553+0000 7f6b14bbc740 1 -- 172.21.15.60:0/3926673487 wait complete. 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 -- start start 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 Processor -- start 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b6a2575700 con 0x55b6a27abff0 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6a26b45f0 con 0x55b6a27b7f80 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6a27bac00 con 0x55b6a27ab6d0 2024-07-24T21:48:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b14bbc740 10 client.0 _ll_register_callbacks cb 0x55b6a26d4960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.554+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51492/0 (socket says 172.21.15.60:51492) 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/2693990411 learned_addr learned my addr 172.21.15.60:0/2693990411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b10da8640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 msgr2=0x55b6a27b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/2693990411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 msgr2=0x55b6a27b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 -- 172.21.15.60:0/2693990411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a26a7680 con 0x55b6a27abff0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b10da8640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6b000e1390 tx=0x7f6b00005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a5db3ddf236b0a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6aeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00020070 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6b000a9ab0 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0401be00 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b000a6710 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b14bbc740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.555+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6a2472650 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0401be00 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6b000ec030 con 0x55b6a27abff0 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 20 client.15942 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a27c7a50 0x55b6a27c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 4 client.15942 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a2724900 con 0x55b6a27c7a50 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.556+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a27c7a50 0x55b6a27c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.557+0000 7f6aeb7fe640 10 client.15942 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.557+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a27c7a50 0x55b6a27c9e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=babb63d95e45f61a server_cookie=92a03d3fe1abd672 in_seq=0 out_seq=0 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.557+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/2693990411 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6a2724900 con 0x55b6a27c7a50 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.557+0000 7f6aeb7fe640 10 client.15942 handle_command_reply: tid=0 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.557+0000 7f6b14bbc740 1 client.15942 shutdown 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 msgr2=0x55b6a27b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6b000e1390 tx=0x7f6b00005f70 comp rx=0 tx=0).stop 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 shutdown_connections 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a27c7a50 0x55b6a27c9e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27abff0 0x55b6a27b53e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2693990411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6aeb7fe640 0 client.15942 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:07.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 >> 172.21.15.60:0/2693990411 conn(0x55b6a27a9500 msgr2=0x55b6a27af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 shutdown_connections 2024-07-24T21:48:07.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2693990411 wait complete. 2024-07-24T21:48:07.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.558+0000 7f6b14bbc740 20 client.15942 trim_cache size 0 max 16384 2024-07-24T21:48:07.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 Processor -- start 2024-07-24T21:48:07.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 -- start start 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27abff0 0x55b6a27cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1b2d70 con 0x55b6a27b7f80 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1b0640 con 0x55b6a27abff0 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1ae490 con 0x55b6a27ab6d0 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51502/0 (socket says 172.21.15.60:51502) 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 -- 172.21.15.60:0/2014193128 learned_addr learned my addr 172.21.15.60:0/2014193128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27abff0 0x55b6a27cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 -- 172.21.15.60:0/2014193128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 msgr2=0x55b6a27b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 -- 172.21.15.60:0/2014193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27abff0 msgr2=0x55b6a27cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27abff0 0x55b6a27cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.572+0000 7f6b10da8640 1 -- 172.21.15.60:0/2014193128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a27abe20 con 0x55b6a27b7f80 2024-07-24T21:48:07.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b10da8640 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b040195b0 tx=0x7f6b04027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac35932b31d3ed9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6af1ffb640 1 -- 172.21.15.60:0/2014193128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04033030 con 0x55b6a27b7f80 2024-07-24T21:48:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6af1ffb640 1 -- 172.21.15.60:0/2014193128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0402a040 con 0x55b6a27b7f80 2024-07-24T21:48:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6af1ffb640 1 -- 172.21.15.60:0/2014193128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04029070 con 0x55b6a27b7f80 2024-07-24T21:48:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2014193128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 msgr2=0x55b6a27cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b040195b0 tx=0x7f6b04027000 comp rx=0 tx=0).stop 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2014193128 shutdown_connections 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27b7f80 0x55b6a27cfc40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27abff0 0x55b6a27cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/2014193128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27ab6d0 0x55b6a27b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2014193128 >> 172.21.15.60:0/2014193128 conn(0x55b6a27a9500 msgr2=0x55b6a27b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2014193128 shutdown_connections 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.573+0000 7f6b14bbc740 1 -- 172.21.15.60:0/2014193128 wait complete. 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 -- start start 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 Processor -- start 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1aeb00 con 0x55b6a27ab6d0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1aff70 con 0x55b6a27b7f80 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1af1d0 con 0x55b6a27abff0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b14bbc740 10 client.0 _ll_register_callbacks cb 0x7f6b000e1770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51506/0 (socket says 172.21.15.60:51506) 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b0bfff640 1 -- 172.21.15.60:0/383639023 learned_addr learned my addr 172.21.15.60:0/383639023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.574+0000 7f6b10da8640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 -- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 msgr2=0x55b6a27b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 -- 172.21.15.60:0/383639023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 msgr2=0x55b6a27b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 -- 172.21.15.60:0/383639023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a27b2900 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b0bfff640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af84a1d31b288138 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6aeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc049020 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6afc005040 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0402a040 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6afc054050 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b14bbc740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.575+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6a2832790 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0402a040 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6afc0035f0 con 0x55b6a27ab6d0 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:48:07.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 20 client.15948 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a2832fc0 0x55b6a28353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 4 client.15948 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6a27203a0 con 0x55b6a2832fc0 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a2832fc0 0x55b6a28353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6b0b7fe640 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a2832fc0 0x55b6a28353e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=358a7ce857f3ceed server_cookie=7d085d3eec23f9b5 in_seq=0 out_seq=0 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.576+0000 7f6aeb7fe640 10 client.15948 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6aeb7fe640 1 -- 172.21.15.60:0/383639023 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6a27203a0 con 0x55b6a2832fc0 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6aeb7fe640 10 client.15948 handle_command_reply: tid=0 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 client.15948 shutdown 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 msgr2=0x55b6a27c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc009680 comp rx=0 tx=0).stop 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 shutdown_connections 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b6a2832fc0 0x55b6a28353e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6a27b7f80 0x55b6a27b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6a27abff0 0x55b6a27b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 --2- 172.21.15.60:0/383639023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6a27ab6d0 0x55b6a27c04d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6aeb7fe640 0 client.15948 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 >> 172.21.15.60:0/383639023 conn(0x55b6a27a9500 msgr2=0x55b6a27b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 shutdown_connections 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 1 -- 172.21.15.60:0/383639023 wait complete. 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.577+0000 7f6b14bbc740 20 client.15948 trim_cache size 0 max 16384 2024-07-24T21:48:07.583 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 msgr2=0x7f6af40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 0x7f6af40b6200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b04020000 tx=0x7f6b04021000 comp rx=0 tx=0).stop 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 msgr2=0x7f6b0c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d950 tx=0x7f6b0000de10 comp rx=0 tx=0).stop 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 shutdown_connections 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6af40b3d40 0x7f6af40b6200 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b0c10eda0 0x7f6b0c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b0c0757d0 0x7f6b0c19b510 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 --2- 172.21.15.60:0/3157399147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b0c074eb0 0x7f6b0c19afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 >> 172.21.15.60:0/3157399147 conn(0x7f6b0c0fe200 msgr2=0x7f6b0c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 shutdown_connections 2024-07-24T21:48:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:07.578+0000 7f6b1280b640 1 -- 172.21.15.60:0/3157399147 wait complete. 2024-07-24T21:48:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:07 smithi123 ceph-mon[37143]: pgmap v420: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 96 KiB/s wr, 48 op/s 2024-07-24T21:48:08.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:48:08.708 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:09.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f980cff9640 1 -- 172.21.15.60:0/697915751 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562287711000 con 0x5622877aa5c0 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f980cff9640 10 client.25462 handle_command_reply: tid=0 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 client.25462 shutdown 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 msgr2=0x5622877af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098b0 tx=0x7f981c009600 comp rx=0 tx=0).stop 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 shutdown_connections 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5622877aa5c0 0x562287828460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56228779e630 0x5622877af700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56228779dd10 0x5622877af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 --2- 172.21.15.60:0/697915751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228779bb40 0x5622877b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f980cff9640 0 client.25462 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 >> 172.21.15.60:0/697915751 conn(0x56228779cfa0 msgr2=0x5622877a74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 shutdown_connections 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 1 -- 172.21.15.60:0/697915751 wait complete. 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.275+0000 7f982d85d740 20 client.25462 trim_cache size 0 max 16384 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 26, 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 6, 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:48:09.278 INFO:teuthology.orchestra.run.smithi060.stdout: "713d8825": { 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 6, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.3, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:48:09.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 msgr2=0x7f98100b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:09.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 0x7f98100b6200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98200a7120 tx=0x7f98200e9040 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 msgr2=0x7f9824101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f982000e9f0 tx=0x7f982000edf0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 shutdown_connections 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f98100b3d40 0x7f98100b6200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982410cae0 0x7f9824101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9824103fa0 0x7f9824101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 --2- 172.21.15.60:0/3696757236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9824103680 0x7f9824101330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 >> 172.21.15.60:0/3696757236 conn(0x7f98240fd0c0 msgr2=0x7f98240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 shutdown_connections 2024-07-24T21:48:09.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:09.276+0000 7f982b4ac640 1 -- 172.21.15.60:0/3696757236 wait complete. 2024-07-24T21:48:09.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:09 smithi060 ceph-mon[33000]: pgmap v421: 97 pgs: 97 active+clean; 5.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 415 KiB/s wr, 94 op/s 2024-07-24T21:48:09.544 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:09 smithi086 ceph-mon[37128]: pgmap v421: 97 pgs: 97 active+clean; 5.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 415 KiB/s wr, 94 op/s 2024-07-24T21:48:09.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:09 smithi123 ceph-mon[37143]: pgmap v421: 97 pgs: 97 active+clean; 5.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 415 KiB/s wr, 94 op/s 2024-07-24T21:48:10.115 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '713d8825', quiesce took 17.6 seconds. Will release after: 2.75 2024-07-24T21:48:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 -- 172.21.15.60:0/1762605503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2948103fc0 msgr2=0x7f294810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 --2- 172.21.15.60:0/1762605503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2948103fc0 0x7f294810c4b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ca0 tx=0x7f293802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 -- 172.21.15.60:0/1762605503 shutdown_connections 2024-07-24T21:48:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 --2- 172.21.15.60:0/1762605503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294810cb00 0x7f294810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 --2- 172.21.15.60:0/1762605503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2948103fc0 0x7f294810c4b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 --2- 172.21.15.60:0/1762605503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 0x7f2948103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 -- 172.21.15.60:0/1762605503 >> 172.21.15.60:0/1762605503 conn(0x7f29480fd100 msgr2=0x7f29480ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 -- 172.21.15.60:0/1762605503 shutdown_connections 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.026+0000 7f294eee5640 1 -- 172.21.15.60:0/1762605503 wait complete. 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 Processor -- start 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 -- start start 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 0x7f2948198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 0x7f29481991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29481127a0 con 0x7f294810cb00 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2948112620 con 0x7f2948103fc0 2024-07-24T21:48:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294eee5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2948112920 con 0x7f29481036a0 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294d482640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294d482640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51530/0 (socket says 172.21.15.60:51530) 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.027+0000 7f294d482640 1 -- 172.21.15.60:0/612123956 learned_addr learned my addr 172.21.15.60:0/612123956 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f2947fff640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 0x7f29481991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294cc81640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 0x7f2948198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 -- 172.21.15.60:0/612123956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 msgr2=0x7f2948198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 0x7f2948198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 -- 172.21.15.60:0/612123956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 msgr2=0x7f29481991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 0x7f29481991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 -- 172.21.15.60:0/612123956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294819ed40 con 0x7f294810cb00 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f2947fff640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 0x7f29481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294d482640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f294000d750 tx=0x7f294000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f2945ffb640 1 -- 172.21.15.60:0/612123956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940017070 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f2945ffb640 1 -- 172.21.15.60:0/612123956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29400054d0 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294819f030 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f2945ffb640 1 -- 172.21.15.60:0/612123956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004c60 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.028+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481a56f0 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2945ffb640 1 -- 172.21.15.60:0/612123956 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2940009bd0 con 0x7f294810cb00 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 Processor -- start 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 -- start start 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b226d0 0x55f085b22ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 0x55f085b31370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f085a999b0 con 0x55f085b22ff0 2024-07-24T21:48:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f085a88eb0 con 0x55f085b2ef80 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f2951296740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f085a8cef0 con 0x55f085b226d0 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.031+0000 7f294cc81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f2945ffb640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 0x7f29340b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51532/0 (socket says 172.21.15.60:51532) 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 -- 172.21.15.60:0/3249586343 learned_addr learned my addr 172.21.15.60:0/3249586343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f2947fff640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b226d0 0x55f085b22ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294d482640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 0x7f29340b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f2947fff640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 0x55f085b31370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f2945ffb640 1 -- 172.21.15.60:0/612123956 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f29400e3ad0 con 0x7f294810cb00 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 -- 172.21.15.60:0/3249586343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b226d0 msgr2=0x55f085b22ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b226d0 0x55f085b22ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 -- 172.21.15.60:0/3249586343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 msgr2=0x55f085b31370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 0x55f085b31370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 -- 172.21.15.60:0/3249586343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f085a12320 con 0x55f085b22ff0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f2947fff640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 0x55f085b31370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294cc81640 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f293000bcf0 tx=0x7f293000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b34233d4b488a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.032+0000 7f294d482640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 0x7f29340b6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f29400e1820 tx=0x7f294000df60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f293effd640 1 -- 172.21.15.60:0/3249586343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930015070 con 0x55f085b22ff0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f293effd640 1 -- 172.21.15.60:0/3249586343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29300118f0 con 0x55f085b22ff0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f293effd640 1 -- 172.21.15.60:0/3249586343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300107f0 con 0x55f085b22ff0 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 -- 172.21.15.60:0/3249586343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 msgr2=0x55f085b2ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f293000bcf0 tx=0x7f293000f730 comp rx=0 tx=0).stop 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 -- 172.21.15.60:0/3249586343 shutdown_connections 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b2ef80 0x55f085b31370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b2ea40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 --2- 172.21.15.60:0/3249586343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b226d0 0x55f085b22ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 -- 172.21.15.60:0/3249586343 >> 172.21.15.60:0/3249586343 conn(0x55f085b20500 msgr2=0x55f085b216a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 -- 172.21.15.60:0/3249586343 shutdown_connections 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.033+0000 7f2951296740 1 -- 172.21.15.60:0/3249586343 wait complete. 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 -- start start 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 Processor -- start 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b2c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f0858ec700 con 0x55f085b226d0 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f085a2b5f0 con 0x55f085b22ff0 2024-07-24T21:48:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f085b31c00 con 0x55f085b2ef80 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2951296740 10 client.0 _ll_register_callbacks cb 0x55f085a4b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51536/0 (socket says 172.21.15.60:51536) 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.034+0000 7f2947fff640 1 -- 172.21.15.60:0/3036235875 learned_addr learned my addr 172.21.15.60:0/3036235875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f294cc81640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 -- 172.21.15.60:0/3036235875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 msgr2=0x55f085b2c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b2c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 -- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 msgr2=0x55f085b2bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b2bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 -- 172.21.15.60:0/3036235875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f085a1e680 con 0x55f085b226d0 2024-07-24T21:48:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2947fff640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f294806bcf0 tx=0x7f293802f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c21ed9a9e941ac server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2927fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938003b70 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29300118f0 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f2938003d10 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2938004b40 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2951296740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0857e9650 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.035+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29300118f0 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f293802f610 con 0x55f085b226d0 2024-07-24T21:48:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:11.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:48:11.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 20 client.15960 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:11.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085b3ea00 0x55f085b40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 4 client.15960 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:11.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f085a9b900 con 0x55f085b3ea00 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f294cc81640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085b3ea00 0x55f085b40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f2927fff640 10 client.15960 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.036+0000 7f294cc81640 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085b3ea00 0x55f085b40e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59fcd1176214da32 server_cookie=43a9240c1c57db78 in_seq=0 out_seq=0 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2927fff640 1 -- 172.21.15.60:0/3036235875 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f085a9b900 con 0x55f085b3ea00 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2927fff640 10 client.15960 handle_command_reply: tid=0 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 client.15960 shutdown 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 msgr2=0x55f085b2ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f294806bcf0 tx=0x7f293802f3f0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 shutdown_connections 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085b3ea00 0x55f085b40e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b2c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b2bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2927fff640 0 client.15960 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 --2- 172.21.15.60:0/3036235875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b2ea40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 >> 172.21.15.60:0/3036235875 conn(0x55f085b20500 msgr2=0x55f085b260a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 shutdown_connections 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 1 -- 172.21.15.60:0/3036235875 wait complete. 2024-07-24T21:48:11.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.037+0000 7f2951296740 20 client.15960 trim_cache size 0 max 16384 2024-07-24T21:48:11.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 Processor -- start 2024-07-24T21:48:11.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 -- start start 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b46c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29481b0a60 con 0x55f085b226d0 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ae330 con 0x55f085b22ff0 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.051+0000 7f2951296740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ac180 con 0x55f085b2ef80 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51544/0 (socket says 172.21.15.60:51544) 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 -- 172.21.15.60:0/71812733 learned_addr learned my addr 172.21.15.60:0/71812733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f294cc81640 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 -- 172.21.15.60:0/71812733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 msgr2=0x55f085b46c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b46c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 -- 172.21.15.60:0/71812733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 msgr2=0x55f085b466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 -- 172.21.15.60:0/71812733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f085b1be20 con 0x55f085b226d0 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f2947fff640 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2938009bf0 tx=0x7f2938031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2969a6f30ac9f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f293e7fc640 1 -- 172.21.15.60:0/71812733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938057030 con 0x55f085b226d0 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f293e7fc640 1 -- 172.21.15.60:0/71812733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938036040 con 0x55f085b226d0 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.052+0000 7f293e7fc640 1 -- 172.21.15.60:0/71812733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938004040 con 0x55f085b226d0 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- 172.21.15.60:0/71812733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 msgr2=0x55f085b26820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2938009bf0 tx=0x7f2938031950 comp rx=0 tx=0).stop 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- 172.21.15.60:0/71812733 shutdown_connections 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b46c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b22ff0 0x55f085b466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 --2- 172.21.15.60:0/71812733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b226d0 0x55f085b26820 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- 172.21.15.60:0/71812733 >> 172.21.15.60:0/71812733 conn(0x55f085b20500 msgr2=0x55f085b27170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- 172.21.15.60:0/71812733 shutdown_connections 2024-07-24T21:48:11.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- 172.21.15.60:0/71812733 wait complete. 2024-07-24T21:48:11.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 -- start start 2024-07-24T21:48:11.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.053+0000 7f2951296740 1 Processor -- start 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b226d0 0x55f085b32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b33a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ac7f0 con 0x55f085b22ff0 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29481adc60 con 0x55f085b226d0 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29481acec0 con 0x55f085b2ef80 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2951296740 10 client.0 _ll_register_callbacks cb 0x7f2938009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51546/0 (socket says 172.21.15.60:51546) 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 -- 172.21.15.60:0/1074956388 learned_addr learned my addr 172.21.15.60:0/1074956388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2947fff640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b226d0 0x55f085b32fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f2947fff640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b33a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 -- 172.21.15.60:0/1074956388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 msgr2=0x55f085b33a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b33a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 -- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b226d0 msgr2=0x55f085b32fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b226d0 0x55f085b32fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.054+0000 7f294cc81640 1 -- 172.21.15.60:0/1074956388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f085ba8960 con 0x55f085b22ff0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f294cc81640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f293000bcf0 tx=0x7f2930005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe0324bc88edfff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930015040 con 0x55f085b22ff0 2024-07-24T21:48:11.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f2930005450 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2938036040 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2930034050 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f085ba9c20 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938036040 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2930010500 con 0x55f085b22ff0 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 20 client.15966 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085baa110 0x55f085bac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 4 client.15966 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.055+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f085a973a0 con 0x55f085baa110 2024-07-24T21:48:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2947fff640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085baa110 0x55f085bac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2947fff640 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085baa110 0x55f085bac530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc25c1cbb5193dbf server_cookie=77baea14c1d22c21 in_seq=0 out_seq=0 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2927fff640 10 client.15966 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2927fff640 1 -- 172.21.15.60:0/1074956388 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f085a973a0 con 0x55f085baa110 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2927fff640 10 client.15966 handle_command_reply: tid=0 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2951296740 1 client.15966 shutdown 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 msgr2=0x55f085b334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.056+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f293000bcf0 tx=0x7f2930005f90 comp rx=0 tx=0).stop 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 shutdown_connections 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f085baa110 0x55f085bac530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f085b2ef80 0x55f085b33a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2927fff640 0 client.15966 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:11.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f085b22ff0 0x55f085b334f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 --2- 172.21.15.60:0/1074956388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f085b226d0 0x55f085b32fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 >> 172.21.15.60:0/1074956388 conn(0x55f085b20500 msgr2=0x55f085b2d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 shutdown_connections 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 1 -- 172.21.15.60:0/1074956388 wait complete. 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f2951296740 20 client.15966 trim_cache size 0 max 16384 2024-07-24T21:48:11.065 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 msgr2=0x7f29340b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 0x7f29340b6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f29400e1820 tx=0x7f294000df60 comp rx=0 tx=0).stop 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 msgr2=0x7f294819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f294000d750 tx=0x7f294000dc10 comp rx=0 tx=0).stop 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 shutdown_connections 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29340b3d40 0x7f29340b6200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294810cb00 0x7f294819e570 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948103fc0 0x7f29481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 --2- 172.21.15.60:0/612123956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29481036a0 0x7f2948198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:11.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.057+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 >> 172.21.15.60:0/612123956 conn(0x7f29480fd100 msgr2=0x7f29480fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:11.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.058+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 shutdown_connections 2024-07-24T21:48:11.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:11.058+0000 7f294eee5640 1 -- 172.21.15.60:0/612123956 wait complete. 2024-07-24T21:48:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:11 smithi086 ceph-mon[37128]: pgmap v422: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 437 KiB/s wr, 110 op/s 2024-07-24T21:48:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:11 smithi060 ceph-mon[33000]: pgmap v422: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 437 KiB/s wr, 110 op/s 2024-07-24T21:48:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:11 smithi123 ceph-mon[37143]: pgmap v422: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 437 KiB/s wr, 110 op/s 2024-07-24T21:48:11.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:48:12.095 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:12.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:48:13.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:13 smithi060 ceph-mon[33000]: pgmap v423: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 442 KiB/s wr, 112 op/s 2024-07-24T21:48:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:13 smithi123 ceph-mon[37143]: pgmap v423: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 442 KiB/s wr, 112 op/s 2024-07-24T21:48:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:13 smithi086 ceph-mon[37128]: pgmap v423: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 442 KiB/s wr, 112 op/s 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 -- 172.21.15.60:0/1834072999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 msgr2=0x7fd38810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 --2- 172.21.15.60:0/1834072999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38810e1e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ca0 tx=0x7fd38002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 -- 172.21.15.60:0/1834072999 shutdown_connections 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 --2- 172.21.15.60:0/1834072999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38810e1e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 --2- 172.21.15.60:0/1834072999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388107060 0x7fd3881074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 --2- 172.21.15.60:0/1834072999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388101650 0x7fd388101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 -- 172.21.15.60:0/1834072999 >> 172.21.15.60:0/1834072999 conn(0x7fd3880fd0c0 msgr2=0x7fd3880ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 -- 172.21.15.60:0/1834072999 shutdown_connections 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.841+0000 7fd38fa17640 1 -- 172.21.15.60:0/1834072999 wait complete. 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 Processor -- start 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 -- start start 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 0x7fd388198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 0x7fd3881991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd388111660 con 0x7fd3881079e0 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881114e0 con 0x7fd388101650 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38fa17640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881117e0 con 0x7fd388107060 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.842+0000 7fd38dfb4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51560/0 (socket says 172.21.15.60:51560) 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 -- 172.21.15.60:0/401360316 learned_addr learned my addr 172.21.15.60:0/401360316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 0x7fd3881991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 -- 172.21.15.60:0/401360316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 msgr2=0x7fd3881991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 0x7fd3881991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 -- 172.21.15.60:0/401360316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 msgr2=0x7fd388198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 0x7fd388198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 0x7fd388198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 -- 172.21.15.60:0/401360316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819ed20 con 0x7fd3881079e0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 0x7fd3881991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 0x7fd388198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd380005cc0 tx=0x7fd38002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd37e7fc640 1 -- 172.21.15.60:0/401360316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380003cf0 con 0x7fd3881079e0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819efb0 con 0x7fd3881079e0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.843+0000 7fd37e7fc640 1 -- 172.21.15.60:0/401360316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3800044f0 con 0x7fd3881079e0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.844+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a56d0 con 0x7fd3881079e0 2024-07-24T21:48:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.844+0000 7fd37e7fc640 1 -- 172.21.15.60:0/401360316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38002fcb0 con 0x7fd3881079e0 2024-07-24T21:48:13.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 Processor -- start 2024-07-24T21:48:13.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd37e7fc640 1 -- 172.21.15.60:0/401360316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd380004040 con 0x7fd3881079e0 2024-07-24T21:48:13.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 -- start start 2024-07-24T21:48:13.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa9551eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa9552aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fa954959b0 con 0x55fa9552af80 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa95484eb0 con 0x55fa9551eff0 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd391dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa95488ef0 con 0x55fa9551e6d0 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.847+0000 7fd38cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51576/0 (socket says 172.21.15.60:51576) 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 -- 172.21.15.60:0/3350406846 learned_addr learned my addr 172.21.15.60:0/3350406846 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa9552aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa9551eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 -- 172.21.15.60:0/3350406846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 msgr2=0x55fa9551eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa9551eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 -- 172.21.15.60:0/3350406846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 msgr2=0x55fa9552aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa9552aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 -- 172.21.15.60:0/3350406846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9540e320 con 0x55fa9552af80 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa9552aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd37e7fc640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 0x7fd3740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd378016560 tx=0x7fd37800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b0e5af12aca63e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 0x7fd3740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd37e7fc640 1 -- 172.21.15.60:0/401360316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd380105830 con 0x7fd3881079e0 2024-07-24T21:48:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd3737fe640 1 -- 172.21.15.60:0/3350406846 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378018430 con 0x55fa9552af80 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.848+0000 7fd3737fe640 1 -- 172.21.15.60:0/3350406846 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd378018a80 con 0x55fa9552af80 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd3737fe640 1 -- 172.21.15.60:0/3350406846 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378014920 con 0x55fa9552af80 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 -- 172.21.15.60:0/3350406846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 msgr2=0x55fa9552d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd378016560 tx=0x7fd37800a520 comp rx=0 tx=0).stop 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 0x7fd3740b6200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55fa9551d3e0 tx=0x7fd38400b470 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 -- 172.21.15.60:0/3350406846 shutdown_connections 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa9552d370 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa9552aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 --2- 172.21.15.60:0/3350406846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa9551eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.849+0000 7fd391dc8740 1 -- 172.21.15.60:0/3350406846 >> 172.21.15.60:0/3350406846 conn(0x55fa9551c500 msgr2=0x55fa9551d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- 172.21.15.60:0/3350406846 shutdown_connections 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- 172.21.15.60:0/3350406846 wait complete. 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- start start 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 Processor -- start 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa95527ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa95528420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fa952e8700 con 0x55fa9552af80 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa954275f0 con 0x55fa9551eff0 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa9552dba0 con 0x55fa9551e6d0 2024-07-24T21:48:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.850+0000 7fd391dc8740 10 client.0 _ll_register_callbacks cb 0x55fa95447960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38dfb4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38dfb4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51588/0 (socket says 172.21.15.60:51588) 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38dfb4640 1 -- 172.21.15.60:0/572497499 learned_addr learned my addr 172.21.15.60:0/572497499 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa95528420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa95527ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 -- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 msgr2=0x55fa95527ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa95527ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 -- 172.21.15.60:0/572497499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 msgr2=0x55fa95528960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 -- 172.21.15.60:0/572497499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9541a680 con 0x55fa9551eff0 2024-07-24T21:48:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa95528420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd37801a040 tx=0x7fd3780186e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58cbc01fee374aee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.851+0000 7fd370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378027060 con 0x55fa9551eff0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd378014ab0 con 0x55fa9551eff0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd378018a80 con 0x55fa9551eff0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3780188d0 con 0x55fa9551eff0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd391dc8740 10 client.25499 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa951e5650 con 0x55fa9551eff0 2024-07-24T21:48:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.852+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd378018a80 con 0x55fa9551eff0 2024-07-24T21:48:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd378021070 con 0x55fa9551eff0 2024-07-24T21:48:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 10 client.25499 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 10 client.25499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 20 client.25499 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa9553a960 0x55fa9553cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 4 client.25499 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.853+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa95497900 con 0x55fa9553a960 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.854+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa9553a960 0x55fa9553cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.854+0000 7fd370ff9640 10 client.25499 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.854+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa9553a960 0x55fa9553cd50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98936f0b0c73223a server_cookie=c83eceef060f7bfa in_seq=0 out_seq=0 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd370ff9640 1 -- 172.21.15.60:0/572497499 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa95497900 con 0x55fa9553a960 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd370ff9640 10 client.25499 handle_command_reply: tid=0 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 client.25499 shutdown 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 msgr2=0x55fa95528420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa95528420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd37801a040 tx=0x7fd3780186e0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 shutdown_connections 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa9553a960 0x55fa9553cd50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9552af80 0x55fa95528960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd370ff9640 0 client.25499 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551eff0 0x55fa95528420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 --2- 172.21.15.60:0/572497499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551e6d0 0x55fa95527ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 >> 172.21.15.60:0/572497499 conn(0x55fa9551c500 msgr2=0x55fa955220a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 shutdown_connections 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 1 -- 172.21.15.60:0/572497499 wait complete. 2024-07-24T21:48:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.855+0000 7fd391dc8740 20 client.25499 trim_cache size 0 max 16384 2024-07-24T21:48:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 Processor -- start 2024-07-24T21:48:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 -- start start 2024-07-24T21:48:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 0x55fa95525630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9552af80 0x55fa9551ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b0830 con 0x55fa9551e6d0 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ae100 con 0x55fa9552af80 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.870+0000 7fd391dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ac000 con 0x55fa9551eff0 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51604/0 (socket says 172.21.15.60:51604) 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 -- 172.21.15.60:0/2931050023 learned_addr learned my addr 172.21.15.60:0/2931050023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 0x55fa95525630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9552af80 0x55fa9551ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 -- 172.21.15.60:0/2931050023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 msgr2=0x55fa95525630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 0x55fa95525630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 -- 172.21.15.60:0/2931050023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9552af80 msgr2=0x55fa9551ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9552af80 0x55fa9551ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 -- 172.21.15.60:0/2931050023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa95522c20 con 0x55fa9551e6d0 2024-07-24T21:48:13.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 0x55fa95525630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:13.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd38400bfd0 tx=0x7fd384013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1180885154e96faa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd372ffd640 1 -- 172.21.15.60:0/2931050023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38401cab0 con 0x55fa9551e6d0 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd372ffd640 1 -- 172.21.15.60:0/2931050023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3840054d0 con 0x55fa9551e6d0 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.871+0000 7fd372ffd640 1 -- 172.21.15.60:0/2931050023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384004c90 con 0x55fa9551e6d0 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 -- 172.21.15.60:0/2931050023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 msgr2=0x55fa955250f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd38400bfd0 tx=0x7fd384013000 comp rx=0 tx=0).stop 2024-07-24T21:48:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 -- 172.21.15.60:0/2931050023 shutdown_connections 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9552af80 0x55fa9551ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9551eff0 0x55fa95525630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 --2- 172.21.15.60:0/2931050023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551e6d0 0x55fa955250f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 -- 172.21.15.60:0/2931050023 >> 172.21.15.60:0/2931050023 conn(0x55fa9551c500 msgr2=0x55fa95529d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 -- 172.21.15.60:0/2931050023 shutdown_connections 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.872+0000 7fd391dc8740 1 -- 172.21.15.60:0/2931050023 wait complete. 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 -- start start 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 Processor -- start 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551e6d0 0x55fa9553d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9552af80 0x55fa955abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ac5c0 con 0x55fa9551eff0 2024-07-24T21:48:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ada30 con 0x55fa9551e6d0 2024-07-24T21:48:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881acc90 con 0x55fa9552af80 2024-07-24T21:48:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd391dc8740 10 client.0 _ll_register_callbacks cb 0x7fd38400b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd38cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd38cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51616/0 (socket says 172.21.15.60:51616) 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd38cfb2640 1 -- 172.21.15.60:0/1591745802 learned_addr learned my addr 172.21.15.60:0/1591745802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.873+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551e6d0 0x55fa9553d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38dfb4640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9552af80 0x55fa955abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 -- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9552af80 msgr2=0x55fa955abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9552af80 0x55fa955abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 -- 172.21.15.60:0/1591745802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551e6d0 msgr2=0x55fa9553d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551e6d0 0x55fa9553d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 -- 172.21.15.60:0/1591745802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa955ac190 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd38cfb2640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd378016eb0 tx=0x7fd37800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a07f099166c40c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378038070 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd3780173e0 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.874+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3840054d0 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd378044050 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa955ad470 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3840054d0 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd378021050 con 0x55fa9551eff0 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:13.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 20 client.15984 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa955adbc0 0x55fa955affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 4 client.15984 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa954933a0 con 0x55fa955adbc0 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.875+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa955adbc0 0x55fa955affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd370ff9640 10 client.15984 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd38d7b3640 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa955adbc0 0x55fa955affe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4161cc13f50d06 server_cookie=7803d03c24aaea57 in_seq=0 out_seq=0 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd370ff9640 1 -- 172.21.15.60:0/1591745802 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa954933a0 con 0x55fa955adbc0 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd370ff9640 10 client.15984 handle_command_reply: tid=0 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 client.15984 shutdown 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 msgr2=0x55fa9553d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd378016eb0 tx=0x7fd37800a650 comp rx=0 tx=0).stop 2024-07-24T21:48:13.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 shutdown_connections 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fa955adbc0 0x55fa955affe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9552af80 0x55fa955abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa9551eff0 0x55fa9553d980 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 --2- 172.21.15.60:0/1591745802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9551e6d0 0x55fa9553d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 >> 172.21.15.60:0/1591745802 conn(0x55fa9551c500 msgr2=0x55fa95535e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.876+0000 7fd370ff9640 0 client.15984 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.877+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 shutdown_connections 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.877+0000 7fd391dc8740 1 -- 172.21.15.60:0/1591745802 wait complete. 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.877+0000 7fd391dc8740 20 client.15984 trim_cache size 0 max 16384 2024-07-24T21:48:13.885 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 msgr2=0x7fd3740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 0x7fd3740b6200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55fa9551d3e0 tx=0x7fd38400b470 comp rx=0 tx=0).stop 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 msgr2=0x7fd38819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd380005cc0 tx=0x7fd38002f470 comp rx=0 tx=0).stop 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 shutdown_connections 2024-07-24T21:48:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd3740b3d40 0x7fd3740b6200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3881079e0 0x7fd38819e550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd388107060 0x7fd3881991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 --2- 172.21.15.60:0/401360316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd388101650 0x7fd388198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 >> 172.21.15.60:0/401360316 conn(0x7fd3880fd0c0 msgr2=0x7fd38806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 shutdown_connections 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:13.878+0000 7fd38fa17640 1 -- 172.21.15.60:0/401360316 wait complete. 2024-07-24T21:48:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:15.378 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:15 smithi060 ceph-mon[33000]: pgmap v424: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 400 KiB/s wr, 110 op/s 2024-07-24T21:48:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:15 smithi123 ceph-mon[37143]: pgmap v424: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 400 KiB/s wr, 110 op/s 2024-07-24T21:48:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:15 smithi086 ceph-mon[37128]: pgmap v424: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 400 KiB/s wr, 110 op/s 2024-07-24T21:48:15.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:48:15.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:48:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 -- 172.21.15.60:0/2549641029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101230 msgr2=0x7fe650101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2549641029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101230 0x7fe650101610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe640009900 tx=0x7fe64002a410 comp rx=0 tx=0).stop 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 -- 172.21.15.60:0/2549641029 shutdown_connections 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2549641029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65010ede0 0x7fe6501111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2549641029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe650101b50 0x7fe65010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2549641029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101230 0x7fe650101610 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 -- 172.21.15.60:0/2549641029 >> 172.21.15.60:0/2549641029 conn(0x7fe6500fd100 msgr2=0x7fe6500ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 -- 172.21.15.60:0/2549641029 shutdown_connections 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 -- 172.21.15.60:0/2549641029 wait complete. 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.810+0000 7fe6549b8640 1 Processor -- start 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 -- start start 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 0x7fe65019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 0x7fe6501a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501149b0 con 0x7fe650101b50 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe650114830 con 0x7fe650101230 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe6549b8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe650114b30 con 0x7fe65010ede0 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52304/0 (socket says 172.21.15.60:52304) 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 -- 172.21.15.60:0/2002324696 learned_addr learned my addr 172.21.15.60:0/2002324696 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64e59c640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 0x7fe65019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64ed9d640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 0x7fe6501a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 -- 172.21.15.60:0/2002324696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 msgr2=0x7fe6501a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 0x7fe6501a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:48:17.372 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 -- 172.21.15.60:0/2002324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 msgr2=0x7fe65019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 0x7fe65019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 -- 172.21.15.60:0/2002324696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6501a0fb0 con 0x7fe650101b50 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64ed9d640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 0x7fe6501a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64e59c640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 0x7fe65019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.811+0000 7fe64dd9b640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c890 tx=0x7fe64400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.812+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644004220 con 0x7fe650101b50 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.812+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6440043c0 con 0x7fe650101b50 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.812+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6501a12a0 con 0x7fe650101b50 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.812+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644004a00 con 0x7fe650101b50 2024-07-24T21:48:17.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.812+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501a7960 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.813+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650101610 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.814+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe6440079b0 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.815+0000 7fe63f7fe640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 0x7fe6380b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.815+0000 7fe64e59c640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 0x7fe6380b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.815+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6440e3e30 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.815+0000 7fe64e59c640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 0x7fe6380b6200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe640005ab0 tx=0x7fe640000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:16.819+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe644014030 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.059+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe65002e000 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.060+0000 7fe63f7fe640 1 -- 172.21.15.60:0/2002324696 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fe6440ab140 con 0x7fe650101b50 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 msgr2=0x7fe6380b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 0x7fe6380b6200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe640005ab0 tx=0x7fe640000bc0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 msgr2=0x7fe65019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c890 tx=0x7fe64400cd50 comp rx=0 tx=0).stop 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 shutdown_connections 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6380b3d40 0x7fe6380b6200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe65010ede0 0x7fe6501a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe650101b50 0x7fe65019b440 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 --2- 172.21.15.60:0/2002324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe650101230 0x7fe65019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 >> 172.21.15.60:0/2002324696 conn(0x7fe6500fd100 msgr2=0x7fe650110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 shutdown_connections 2024-07-24T21:48:17.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:17.062+0000 7fe6549b8640 1 -- 172.21.15.60:0/2002324696 wait complete. 2024-07-24T21:48:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- 172.21.15.60:0/4139990357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e81015d0 msgr2=0x7ff7e81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/4139990357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e81015d0 0x7ff7e81019b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- 172.21.15.60:0/4139990357 shutdown_connections 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/4139990357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e8107950 0x7ff7e810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/4139990357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/4139990357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e81015d0 0x7ff7e81019b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- 172.21.15.60:0/4139990357 >> 172.21.15.60:0/4139990357 conn(0x7ff7e80fd030 msgr2=0x7ff7e80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- 172.21.15.60:0/4139990357 shutdown_connections 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- 172.21.15.60:0/4139990357 wait complete. 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 Processor -- start 2024-07-24T21:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.451+0000 7ff7efa3f640 1 -- start start 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e81015d0 0x7ff7e8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 0x7ff7e819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81115f0 con 0x7ff7e8106fd0 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8111470 con 0x7ff7e8107950 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7efa3f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8111770 con 0x7ff7e81015d0 2024-07-24T21:48:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52328/0 (socket says 172.21.15.60:52328) 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 -- 172.21.15.60:0/539524776 learned_addr learned my addr 172.21.15.60:0/539524776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e81015d0 0x7ff7e8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 0x7ff7e819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 -- 172.21.15.60:0/539524776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e81015d0 msgr2=0x7ff7e8198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e81015d0 0x7ff7e8198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 -- 172.21.15.60:0/539524776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 msgr2=0x7ff7e819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 0x7ff7e819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 -- 172.21.15.60:0/539524776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e819ec60 con 0x7ff7e8106fd0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 0x7ff7e819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d950 tx=0x7ff7d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.452+0000 7ff7de7fc640 1 -- 172.21.15.60:0/539524776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8017070 con 0x7ff7e8106fd0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.453+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e819ef50 con 0x7ff7e8106fd0 2024-07-24T21:48:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.453+0000 7ff7de7fc640 1 -- 172.21.15.60:0/539524776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d8009c30 con 0x7ff7e8106fd0 2024-07-24T21:48:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.453+0000 7ff7de7fc640 1 -- 172.21.15.60:0/539524776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f620 con 0x7ff7e8106fd0 2024-07-24T21:48:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.453+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81a57f0 con 0x7ff7e8106fd0 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 Processor -- start 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 -- start start 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729df5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729e02170 0x55b729e04560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b729d6b280 con 0x55b729df61e0 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b729d5a780 con 0x55b729e02170 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7f2792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b729d5e7c0 con 0x55b729df58c0 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.455+0000 7ff7de7fc640 1 -- 172.21.15.60:0/539524776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff7d8010460 con 0x7ff7e8106fd0 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52340/0 (socket says 172.21.15.60:52340) 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 -- 172.21.15.60:0/3411258184 learned_addr learned my addr 172.21.15.60:0/3411258184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729df5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729e02170 0x55b729e04560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 -- 172.21.15.60:0/3411258184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 msgr2=0x55b729df5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729df5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 -- 172.21.15.60:0/3411258184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729e02170 msgr2=0x55b729e04560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729e02170 0x55b729e04560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 -- 172.21.15.60:0/3411258184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b729abe790 con 0x55b729df61e0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca11fcab9d870b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7d37fe640 1 -- 172.21.15.60:0/3411258184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4035040 con 0x55b729df61e0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.456+0000 7ff7de7fc640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 0x7ff7d40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7d37fe640 1 -- 172.21.15.60:0/3411258184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e40054f0 con 0x55b729df61e0 2024-07-24T21:48:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7d37fe640 1 -- 172.21.15.60:0/3411258184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4003850 con 0x55b729df61e0 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 0x7ff7d40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7de7fc640 1 -- 172.21.15.60:0/539524776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff7d80e3c40 con 0x7ff7e8106fd0 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 0x7ff7d40b6200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005fd0 tx=0x7ff7e0009700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 -- 172.21.15.60:0/3411258184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 msgr2=0x55b729e01c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4004180 comp rx=0 tx=0).stop 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 -- 172.21.15.60:0/3411258184 shutdown_connections 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729e02170 0x55b729e04560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729df61e0 0x55b729e01c30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 --2- 172.21.15.60:0/3411258184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729df5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 -- 172.21.15.60:0/3411258184 >> 172.21.15.60:0/3411258184 conn(0x55b729df36f0 msgr2=0x55b729df4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 -- 172.21.15.60:0/3411258184 shutdown_connections 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.457+0000 7ff7f2792740 1 -- 172.21.15.60:0/3411258184 wait complete. 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 -- start start 2024-07-24T21:48:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 Processor -- start 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729dfeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b729c94ab0 con 0x55b729e02170 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b729ce29e0 con 0x55b729df61e0 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b729e04df0 con 0x55b729df58c0 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7f2792740 10 client.0 _ll_register_callbacks cb 0x55b729a6aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52348/0 (socket says 172.21.15.60:52348) 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4080445281 learned_addr learned my addr 172.21.15.60:0/4080445281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.458+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729dfeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 msgr2=0x55b729e018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4080445281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 msgr2=0x55b729dfeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729dfeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4080445281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b729b37620 con 0x55b729e02170 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0013fd0 tx=0x7ff7e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb2c9871ccf5b48 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001e6d0 con 0x55b729e02170 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff7e001e870 con 0x55b729e02170 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b729d09ce0 con 0x55b729e02170 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7e001eab0 con 0x55b729e02170 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7f2792740 10 client.16002 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b729e10610 con 0x55b729e02170 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.459+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b729d09ce0 con 0x55b729e02170 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e001d410 con 0x55b729e02170 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 20 client.16002 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e11b90 0x55b729e13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 4 client.16002 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b729d6d1d0 con 0x55b729e11b90 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e11b90 0x55b729e13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7d0ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.460+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e11b90 0x55b729e13f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fde96671d85b39 server_cookie=e2dded14247b7ea0 in_seq=0 out_seq=0 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/4080445281 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b729d6d1d0 con 0x55b729e11b90 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7d0ff9640 10 client.16002 handle_command_reply: tid=0 2024-07-24T21:48:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 client.16002 shutdown 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 msgr2=0x55b729dff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0013fd0 tx=0x7ff7e0004a40 comp rx=0 tx=0).stop 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 shutdown_connections 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e11b90 0x55b729e13f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729dff2e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729dfeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7d0ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4080445281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 >> 172.21.15.60:0/4080445281 conn(0x55b729df36f0 msgr2=0x55b729df9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 shutdown_connections 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 1 -- 172.21.15.60:0/4080445281 wait complete. 2024-07-24T21:48:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.461+0000 7ff7f2792740 20 client.16002 trim_cache size 0 max 16384 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 Processor -- start 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 -- start start 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df58c0 0x55b729df26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 0x55b729e19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0a30 con 0x55b729e02170 2024-07-24T21:48:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ae300 con 0x55b729df58c0 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.477+0000 7ff7f2792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ac150 con 0x55b729df61e0 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52356/0 (socket says 172.21.15.60:52356) 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4112706199 learned_addr learned my addr 172.21.15.60:0/4112706199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 0x55b729e19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df58c0 0x55b729df26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4112706199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 msgr2=0x55b729e19850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 0x55b729e19850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4112706199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df58c0 msgr2=0x55b729df26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df58c0 0x55b729df26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 -- 172.21.15.60:0/4112706199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b729df65c0 con 0x55b729e02170 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 0x55b729e19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009890 tx=0x7ff7e00099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0392740af9e1c18 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7d2ffd640 1 -- 172.21.15.60:0/4112706199 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001dcf0 con 0x55b729e02170 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7d2ffd640 1 -- 172.21.15.60:0/4112706199 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e0010540 con 0x55b729e02170 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.478+0000 7ff7d2ffd640 1 -- 172.21.15.60:0/4112706199 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00052a0 con 0x55b729e02170 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- 172.21.15.60:0/4112706199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 msgr2=0x55b729e19d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009890 tx=0x7ff7e00099d0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- 172.21.15.60:0/4112706199 shutdown_connections 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e19d90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df61e0 0x55b729e19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 --2- 172.21.15.60:0/4112706199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df58c0 0x55b729df26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- 172.21.15.60:0/4112706199 >> 172.21.15.60:0/4112706199 conn(0x55b729df36f0 msgr2=0x55b729df1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- 172.21.15.60:0/4112706199 shutdown_connections 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- 172.21.15.60:0/4112706199 wait complete. 2024-07-24T21:48:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 -- start start 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.479+0000 7ff7f2792740 1 Processor -- start 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e07a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729e08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ac7c0 con 0x55b729e02170 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81adc30 con 0x55b729df61e0 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ace90 con 0x55b729df58c0 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 10 client.0 _ll_register_callbacks cb 0x7ff7e0009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52364/0 (socket says 172.21.15.60:52364) 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 -- 172.21.15.60:0/2576470171 learned_addr learned my addr 172.21.15.60:0/2576470171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e07a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729e08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 -- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 msgr2=0x55b729e07a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e07a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 -- 172.21.15.60:0/2576470171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 msgr2=0x55b729e08000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729e08000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 -- 172.21.15.60:0/2576470171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b729dfeb20 con 0x55b729e02170 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7ecfda640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729e08000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7edfdc640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0012cb0 tx=0x7ff7e001e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e847f2f676241f27 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001dc70 con 0x55b729e02170 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff7e0004a00 con 0x55b729e02170 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.480+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b729d68c70 con 0x55b729e02170 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7e0004c40 con 0x55b729e02170 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 10 client.16008 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b729e7c280 con 0x55b729e02170 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b729d68c70 con 0x55b729e02170 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e00385a0 con 0x55b729e02170 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 10 client.16008 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 20 client.16008 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e7c940 0x55b729e7ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 4 client.16008 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b729d663b0 con 0x55b729e7c940 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.481+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e7c940 0x55b729e7ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.482+0000 7ff7d0ff9640 10 client.16008 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.482+0000 7ff7ed7db640 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e7c940 0x55b729e7ed60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5488a4105ba00ee server_cookie=22119a14daefadab in_seq=0 out_seq=0 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7d0ff9640 1 -- 172.21.15.60:0/2576470171 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b729d663b0 con 0x55b729e7c940 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7d0ff9640 10 client.16008 handle_command_reply: tid=0 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 client.16008 shutdown 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 msgr2=0x55b729e73850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0012cb0 tx=0x7ff7e001e820 comp rx=0 tx=0).stop 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 shutdown_connections 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b729e7c940 0x55b729e7ed60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b729e02170 0x55b729e73850 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b729df61e0 0x55b729e08000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7d0ff9640 0 client.16008 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 --2- 172.21.15.60:0/2576470171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b729df58c0 0x55b729e07a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 >> 172.21.15.60:0/2576470171 conn(0x55b729df36f0 msgr2=0x55b729df27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 shutdown_connections 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 1 -- 172.21.15.60:0/2576470171 wait complete. 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.508+0000 7ff7f2792740 20 client.16008 trim_cache size 0 max 16384 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:48:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 msgr2=0x7ff7d40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 0x7ff7d40b6200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005fd0 tx=0x7ff7e0009700 comp rx=0 tx=0).stop 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 msgr2=0x7ff7e8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d950 tx=0x7ff7d800de10 comp rx=0 tx=0).stop 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 shutdown_connections 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff7d40b3d40 0x7ff7d40b6200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e8107950 0x7ff7e819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e8106fd0 0x7ff7e8199100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 --2- 172.21.15.60:0/539524776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e81015d0 0x7ff7e8198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 >> 172.21.15.60:0/539524776 conn(0x7ff7e80fd030 msgr2=0x7ff7e8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 shutdown_connections 2024-07-24T21:48:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:17.509+0000 7ff7efa3f640 1 -- 172.21.15.60:0/539524776 wait complete. 2024-07-24T21:48:17.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:17 smithi060 ceph-mon[33000]: pgmap v425: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 376 KiB/s wr, 94 op/s 2024-07-24T21:48:17.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:17 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2002324696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:17 smithi123 ceph-mon[37143]: pgmap v425: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 376 KiB/s wr, 94 op/s 2024-07-24T21:48:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:17 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2002324696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:17 smithi086 ceph-mon[37128]: pgmap v425: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 376 KiB/s wr, 94 op/s 2024-07-24T21:48:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:17 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2002324696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:18.124 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 713d8825 --release --await' 2024-07-24T21:48:18.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 713d8825 --release --await 2024-07-24T21:48:18.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:48:18.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:48:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:18.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.412+0000 7f9bbc62a640 1 -- 172.21.15.60:0/2507956960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 msgr2=0x7f9bb410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.412+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/2507956960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb410c4b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009900 tx=0x7f9ba402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 -- 172.21.15.60:0/2507956960 shutdown_connections 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/2507956960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 0x7f9bb410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/2507956960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb410c4b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/2507956960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 0x7f9bb4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 -- 172.21.15.60:0/2507956960 >> 172.21.15.60:0/2507956960 conn(0x7f9bb40fd100 msgr2=0x7f9bb40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 -- 172.21.15.60:0/2507956960 shutdown_connections 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 -- 172.21.15.60:0/2507956960 wait complete. 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 Processor -- start 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 -- start start 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 0x7f9bb4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.413+0000 7f9bbc62a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 0x7f9bb4101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbc62a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4112750 con 0x7f9bb4103fc0 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbc62a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41125d0 con 0x7f9bb410cb00 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbc62a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41128d0 con 0x7f9bb41036a0 2024-07-24T21:48:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52382/0 (socket says 172.21.15.60:52382) 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1170937139 learned_addr learned my addr 172.21.15.60:0/1170937139 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 0x7f9bb4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 0x7f9bb4101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1170937139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 msgr2=0x7f9bb4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 0x7f9bb4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1170937139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 msgr2=0x7f9bb4101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 0x7f9bb4101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1170937139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb41a70c0 con 0x7f9bb4103fc0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4004270 tx=0x7f9ba40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bab7fe640 1 -- 172.21.15.60:0/1170937139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba403f030 con 0x7f9bb4103fc0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bab7fe640 1 -- 172.21.15.60:0/1170937139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba40044c0 con 0x7f9bb4103fc0 2024-07-24T21:48:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb41a7350 con 0x7f9bb4103fc0 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bab7fe640 1 -- 172.21.15.60:0/1170937139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4035070 con 0x7f9bb4103fc0 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.414+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb41a7880 con 0x7f9bb4103fc0 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.417+0000 7f9bab7fe640 1 -- 172.21.15.60:0/1170937139 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9ba4042030 con 0x7f9bb4103fc0 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.417+0000 7f9bbe9db740 1 Processor -- start 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.417+0000 7f9bbe9db740 1 -- start start 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5594be93f9f0 con 0x5594be9dcff0 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594be92eef0 con 0x5594be9c7f30 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5594be932f30 con 0x5594be9c8850 2024-07-24T21:48:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52396/0 (socket says 172.21.15.60:52396) 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1336558367 learned_addr learned my addr 172.21.15.60:0/1336558367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bab7fe640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 0x7f9ba00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bab7fe640 1 -- 172.21.15.60:0/1170937139 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9ba4104590 con 0x7f9bb4103fc0 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1336558367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 msgr2=0x5594be9dcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9dcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1336558367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 msgr2=0x5594be9c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1336558367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594be68f930 con 0x5594be9dcff0 2024-07-24T21:48:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.418+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 0x7f9ba00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.419+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40c83c0 tx=0x7f9ba4031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec995ecae001c57 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.419+0000 7f9ba8ff9640 1 -- 172.21.15.60:0/1336558367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4049070 con 0x5594be9dcff0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.419+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 0x7f9ba00b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4103410 tx=0x7f9bac004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.419+0000 7f9ba8ff9640 1 -- 172.21.15.60:0/1336558367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba40cac60 con 0x5594be9dcff0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.419+0000 7f9ba8ff9640 1 -- 172.21.15.60:0/1336558367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4101f00 con 0x5594be9dcff0 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1336558367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 msgr2=0x5594be9df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40c83c0 tx=0x7f9ba4031a10 comp rx=0 tx=0).stop 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1336558367 shutdown_connections 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9df3e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9dcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1336558367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1336558367 >> 172.21.15.60:0/1336558367 conn(0x5594be9c5d60 msgr2=0x5594be9c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1336558367 shutdown_connections 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1336558367 wait complete. 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 -- start start 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.420+0000 7f9bbe9db740 1 Processor -- start 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5594be792890 con 0x5594be9dcff0 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594be8d1390 con 0x5594be9c7f30 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5594be93f9f0 con 0x5594be9c8850 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 10 client.0 _ll_register_callbacks cb 0x5594be8db500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbabc7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.466 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:19.466 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:48:19.466 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "adf5c8ff-6191-4614-84bf-ad122964d88d", 2024-07-24T21:48:19.466 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:48:19.466 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbabc7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52404/0 (socket says 172.21.15.60:52404) 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbabc7640 1 -- 172.21.15.60:0/1565632764 learned_addr learned my addr 172.21.15.60:0/1565632764 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1565632764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 msgr2=0x5594be9d9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9d9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1565632764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 msgr2=0x5594be9daa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/1565632764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594be8cbd60 con 0x5594be9c8850 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9da4d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40cf860 tx=0x7f9ba4107c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f1465e10c095039 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4049070 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ba40cac60 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.421+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9ba4101f00 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ba4102100 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 10 client.25528 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594be9eb830 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba40cac60 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9ba403bbe0 con 0x5594be9c8850 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 10 client.25528 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 20 client.25528 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594be9ecb80 0x5594be9eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 4 client.25528 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594be941940 con 0x5594be9ecb80 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.422+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594be9ecb80 0x5594be9eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9b9dffb640 10 client.25528 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594be9ecb80 0x5594be9eef70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf23a4a07db22e5 server_cookie=2859b2da1073edb4 in_seq=0 out_seq=0 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9b9dffb640 1 -- 172.21.15.60:0/1565632764 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594be941940 con 0x5594be9ecb80 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9b9dffb640 10 client.25528 handle_command_reply: tid=0 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 client.25528 shutdown 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 msgr2=0x5594be9da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9da4d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40cf860 tx=0x7f9ba4107c00 comp rx=0 tx=0).stop 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 shutdown_connections 2024-07-24T21:48:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594be9ecb80 0x5594be9eef70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9dcff0 0x5594be9daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9da4d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9b9dffb640 0 client.25528 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/1565632764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9c7f30 0x5594be9d9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 >> 172.21.15.60:0/1565632764 conn(0x5594be9c5d60 msgr2=0x5594be9cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.423+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 shutdown_connections 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.424+0000 7f9bbe9db740 1 -- 172.21.15.60:0/1565632764 wait complete. 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.424+0000 7f9bbe9db740 20 client.25528 trim_cache size 0 max 16384 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 Processor -- start 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 -- start start 2024-07-24T21:48:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41ae790 con 0x5594be9c7f30 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41ac060 con 0x5594be9dcff0 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41a9ed0 con 0x5594be9c8850 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bba3c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bba3c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52414/0 (socket says 172.21.15.60:52414) 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bba3c6640 1 -- 172.21.15.60:0/3323635715 learned_addr learned my addr 172.21.15.60:0/3323635715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bba3c6640 1 -- 172.21.15.60:0/3323635715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 msgr2=0x5594be9c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.442+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bba3c6640 1 -- 172.21.15.60:0/3323635715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 msgr2=0x5594be9f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bba3c6640 1 -- 172.21.15.60:0/3323635715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594be9c8680 con 0x5594be9c7f30 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbabc7640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5594be9ef650 tx=0x7f9bb000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d9e22d85bc1479 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9b9ffff640 1 -- 172.21.15.60:0/3323635715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0005220 con 0x5594be9c7f30 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9b9ffff640 1 -- 172.21.15.60:0/3323635715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb0004c50 con 0x5594be9c7f30 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9b9ffff640 1 -- 172.21.15.60:0/3323635715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb001d8b0 con 0x5594be9c7f30 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3323635715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 msgr2=0x5594be9d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5594be9ef650 tx=0x7f9bb000a250 comp rx=0 tx=0).stop 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3323635715 shutdown_connections 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c8850 0x5594be9c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.443+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3323635715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c7f30 0x5594be9d7a80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3323635715 >> 172.21.15.60:0/3323635715 conn(0x5594be9c5d60 msgr2=0x5594be9c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3323635715 shutdown_connections 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3323635715 wait complete. 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- start start 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 Processor -- start 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c7f30 0x5594be9e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41aa520 con 0x5594be9c8850 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41ab990 con 0x5594be9dcff0 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb41aabf0 con 0x5594be9c7f30 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bbe9db740 10 client.0 _ll_register_callbacks cb 0x7f9ba4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.444+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52418/0 (socket says 172.21.15.60:52418) 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bbabc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/3823130761 learned_addr learned my addr 172.21.15.60:0/3823130761 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c7f30 0x5594be9e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c7f30 msgr2=0x5594be9e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c7f30 0x5594be9e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/3823130761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 msgr2=0x5594be9dfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9dfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 -- 172.21.15.60:0/3823130761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594be9e00a0 con 0x5594be9c8850 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bb9bc5640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4101bc0 tx=0x7f9ba40c73e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45abdd0b15fe9ab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40c7010 con 0x5594be9c8850 2024-07-24T21:48:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9ba4101f00 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb0004c50 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ba40cf6a0 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bbe9db740 10 client.16026 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594be9e00a0 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.445+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb0004c50 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9ba40d3cc0 con 0x5594be9c8850 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 10 client.16026 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 20 client.16026 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594bea58150 0x5594bea5a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 4 client.16026 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5594be93d3e0 con 0x5594bea58150 2024-07-24T21:48:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594bea58150 0x5594bea5a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9bba3c6640 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594bea58150 0x5594bea5a570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cdf6ff06efa5d8 server_cookie=113dec81abecf545 in_seq=0 out_seq=0 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.446+0000 7f9b9dffb640 10 client.16026 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.447+0000 7f9b9dffb640 1 -- 172.21.15.60:0/3823130761 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5594be93d3e0 con 0x5594bea58150 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.447+0000 7f9b9dffb640 10 client.16026 handle_command_reply: tid=0 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.447+0000 7f9bbe9db740 1 client.16026 shutdown 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.447+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 msgr2=0x5594be9e50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.447+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4101bc0 tx=0x7f9ba40c73e0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 shutdown_connections 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594bea58150 0x5594bea5a570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594be9dcff0 0x5594be9dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9b9dffb640 0 client.16026 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594be9c8850 0x5594be9e50f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 --2- 172.21.15.60:0/3823130761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594be9c7f30 0x5594be9e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 >> 172.21.15.60:0/3823130761 conn(0x5594be9c5d60 msgr2=0x5594be9c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 shutdown_connections 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 1 -- 172.21.15.60:0/3823130761 wait complete. 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbe9db740 20 client.16026 trim_cache size 0 max 16384 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 msgr2=0x7f9ba00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 0x7f9ba00b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4103410 tx=0x7f9bac004ab0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 msgr2=0x7f9bb4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4004270 tx=0x7f9ba40042a0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 shutdown_connections 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ba00b3dc0 0x7f9ba00b6280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb410cb00 0x7f9bb4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bb4103fc0 0x7f9bb4101890 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 --2- 172.21.15.60:0/1170937139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bb41036a0 0x7f9bb4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:19.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 >> 172.21.15.60:0/1170937139 conn(0x7f9bb40fd100 msgr2=0x7f9bb410ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 shutdown_connections 2024-07-24T21:48:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:19.448+0000 7f9bbc62a640 1 -- 172.21.15.60:0/1170937139 wait complete. 2024-07-24T21:48:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:19 smithi060 ceph-mon[33000]: pgmap v426: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 350 KiB/s wr, 73 op/s 2024-07-24T21:48:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:48:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:48:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:48:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:19 smithi123 ceph-mon[37143]: pgmap v426: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 350 KiB/s wr, 73 op/s 2024-07-24T21:48:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:48:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:48:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:48:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:19 smithi086 ceph-mon[37128]: pgmap v426: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 350 KiB/s wr, 73 op/s 2024-07-24T21:48:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:48:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:48:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:48:20.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:48:20.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:20 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:48:20.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:20 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:20.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:20 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:20.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:20 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:20.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:20 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:48:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:20 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:20 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:20 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:20 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:48:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:20 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:20 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:20 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 -- 172.21.15.60:0/3577013459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3801079e0 msgr2=0x7fb38010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3577013459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3801079e0 0x7fb38010e1e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb378009be0 tx=0x7fb37802a6c0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 -- 172.21.15.60:0/3577013459 shutdown_connections 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3577013459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3801079e0 0x7fb38010e1e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3577013459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380107060 0x7fb3801074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3577013459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101650 0x7fb380101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 -- 172.21.15.60:0/3577013459 >> 172.21.15.60:0/3577013459 conn(0x7fb3800fd100 msgr2=0x7fb3800ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.388+0000 7fb387a4d640 1 -- 172.21.15.60:0/3577013459 shutdown_connections 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 -- 172.21.15.60:0/3577013459 wait complete. 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 Processor -- start 2024-07-24T21:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 -- start start 2024-07-24T21:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380101650 0x7fb380198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3801079e0 0x7fb38019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb380111610 con 0x7fb380107060 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb380111490 con 0x7fb380101650 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.389+0000 7fb387a4d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb380111790 con 0x7fb3801079e0 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52442/0 (socket says 172.21.15.60:52442) 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 -- 172.21.15.60:0/3971954508 learned_addr learned my addr 172.21.15.60:0/3971954508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb3857e9640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380101650 0x7fb380198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb385fea640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3801079e0 0x7fb38019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 -- 172.21.15.60:0/3971954508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3801079e0 msgr2=0x7fb38019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3801079e0 0x7fb38019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 -- 172.21.15.60:0/3971954508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380101650 msgr2=0x7fb380198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380101650 0x7fb380198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 -- 172.21.15.60:0/3971954508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38019ec30 con 0x7fb380107060 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.390+0000 7fb384fe8640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9c0 tx=0x7fb37000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.391+0000 7fb3767fc640 1 -- 172.21.15.60:0/3971954508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000ccc0 con 0x7fb380107060 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.391+0000 7fb3767fc640 1 -- 172.21.15.60:0/3971954508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3700054d0 con 0x7fb380107060 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.391+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38019ef20 con 0x7fb380107060 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.391+0000 7fb3767fc640 1 -- 172.21.15.60:0/3971954508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010430 con 0x7fb380107060 2024-07-24T21:48:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.391+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3801a55e0 con 0x7fb380107060 2024-07-24T21:48:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.393+0000 7fb3767fc640 1 -- 172.21.15.60:0/3971954508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fb370004f80 con 0x7fb380107060 2024-07-24T21:48:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 Processor -- start 2024-07-24T21:48:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 -- start start 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e6176ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6177dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562e616e1dd0 con 0x562e6176b350 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e616d12d0 con 0x562e6176aa30 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb389dfe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e616d5310 con 0x562e6177b6f0 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52446/0 (socket says 172.21.15.60:52446) 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb3857e9640 1 -- 172.21.15.60:0/816957200 learned_addr learned my addr 172.21.15.60:0/816957200 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb384fe8640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6177dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.394+0000 7fb385fea640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e6176ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 -- 172.21.15.60:0/816957200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 msgr2=0x562e6177dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6177dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 -- 172.21.15.60:0/816957200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 msgr2=0x562e6176ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e6176ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 -- 172.21.15.60:0/816957200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e61431b00 con 0x562e6176b350 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3767fc640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 0x7fb36c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3857e9640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009d20 tx=0x7fb37c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d81150a95172349 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb3767fc640 1 -- 172.21.15.60:0/3971954508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb3700e43f0 con 0x7fb380107060 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb385fea640 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e6176ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb36b7fe640 1 -- 172.21.15.60:0/816957200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004230 con 0x562e6176b350 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb36b7fe640 1 -- 172.21.15.60:0/816957200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb37c0043d0 con 0x562e6176b350 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb36b7fe640 1 -- 172.21.15.60:0/816957200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c019820 con 0x562e6176b350 2024-07-24T21:48:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.395+0000 7fb385fea640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 0x7fb36c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb385fea640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 0x7fb36c0b6200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb378009370 tx=0x7fb378004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- 172.21.15.60:0/816957200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 msgr2=0x562e6177b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009d20 tx=0x7fb37c011810 comp rx=0 tx=0).stop 2024-07-24T21:48:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- 172.21.15.60:0/816957200 shutdown_connections 2024-07-24T21:48:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6177dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e6177b1b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 --2- 172.21.15.60:0/816957200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e6176ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- 172.21.15.60:0/816957200 >> 172.21.15.60:0/816957200 conn(0x562e61768860 msgr2=0x562e61769a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- 172.21.15.60:0/816957200 shutdown_connections 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- 172.21.15.60:0/816957200 wait complete. 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.396+0000 7fb389dfe740 1 -- start start 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 Processor -- start 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 0x562e6177b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176b350 0x562e617785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562e61534bb0 con 0x562e6177b6f0 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e61673900 con 0x562e6176b350 2024-07-24T21:48:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e6177e3a0 con 0x562e6176aa30 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb389dfe740 10 client.0 _ll_register_callbacks cb 0x562e6167dc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb385fea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb385fea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52448/0 (socket says 172.21.15.60:52448) 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb385fea640 1 -- 172.21.15.60:0/3371435253 learned_addr learned my addr 172.21.15.60:0/3371435253 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.397+0000 7fb3857e9640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 0x562e6177b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb384fe8640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176b350 0x562e617785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 -- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 msgr2=0x562e6177b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 0x562e6177b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 -- 172.21.15.60:0/3371435253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176b350 msgr2=0x562e617785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176b350 0x562e617785f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 -- 172.21.15.60:0/3371435253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e614a8960 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb3857e9640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 0x562e6177b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb385fea640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb378009be0 tx=0x7fb37802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b460908b9194ec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37804f020 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37c0043d0 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb378030b70 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb37805a050 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb389dfe740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.398+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e61789ed0 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb37c0043d0 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb37804b480 con 0x562e6177b6f0 2024-07-24T21:48:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 10 client.16038 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 20 client.16038 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e6178b350 0x562e6178d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 4 client.16038 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e616e3d20 con 0x562e6178b350 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.399+0000 7fb3857e9640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e6178b350 0x562e6178d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.400+0000 7fb3857e9640 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e6178b350 0x562e6178d7d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f0ea4d6d8c2597 server_cookie=965a015f84baa9df in_seq=0 out_seq=0 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.400+0000 7fb368ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.400+0000 7fb368ff9640 1 -- 172.21.15.60:0/3371435253 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e616e3d20 con 0x562e6178b350 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.400+0000 7fb368ff9640 10 client.16038 handle_command_reply: tid=0 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.400+0000 7fb389dfe740 1 client.16038 shutdown 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 msgr2=0x562e61778d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb378009be0 tx=0x7fb37802a6c0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 shutdown_connections 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e6178b350 0x562e6178d7d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6177b6f0 0x562e61778d80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176b350 0x562e617785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 --2- 172.21.15.60:0/3371435253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176aa30 0x562e6177b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb368ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 >> 172.21.15.60:0/3371435253 conn(0x562e61768860 msgr2=0x562e6176e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 shutdown_connections 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 1 -- 172.21.15.60:0/3371435253 wait complete. 2024-07-24T21:48:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.401+0000 7fb389dfe740 20 client.16038 trim_cache size 0 max 16384 2024-07-24T21:48:21.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 Processor -- start 2024-07-24T21:48:21.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 -- start start 2024-07-24T21:48:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176b350 0x562e61763820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.422 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:21 smithi060 ceph-mon[33000]: pgmap v427: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-07-24T21:48:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 0x562e61768110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ae780 con 0x562e6176aa30 2024-07-24T21:48:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ac050 con 0x562e6177b6f0 2024-07-24T21:48:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb389dfe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801a9ea0 con 0x562e6176b350 2024-07-24T21:48:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52458/0 (socket says 172.21.15.60:52458) 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb3857e9640 1 -- 172.21.15.60:0/1198115033 learned_addr learned my addr 172.21.15.60:0/1198115033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb384fe8640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176b350 0x562e61763820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.420+0000 7fb385fea640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 0x562e61768110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 -- 172.21.15.60:0/1198115033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176b350 msgr2=0x562e61763820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176b350 0x562e61763820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 -- 172.21.15.60:0/1198115033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 msgr2=0x562e61768110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 0x562e61768110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 -- 172.21.15.60:0/1198115033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e617692b0 con 0x562e6176aa30 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb385fea640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 0x562e61768110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb3857e9640 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x562e6176a510 tx=0x7fb37c0194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4b54983014517c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb36affd640 1 -- 172.21.15.60:0/1198115033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c005220 con 0x562e6176aa30 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb36affd640 1 -- 172.21.15.60:0/1198115033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb37c032070 con 0x562e6176aa30 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.421+0000 7fb36affd640 1 -- 172.21.15.60:0/1198115033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c019530 con 0x562e6176aa30 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 -- 172.21.15.60:0/1198115033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 msgr2=0x562e61776180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x562e6176a510 tx=0x7fb37c0194c0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 -- 172.21.15.60:0/1198115033 shutdown_connections 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6177b6f0 0x562e61768110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6176b350 0x562e61763820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 --2- 172.21.15.60:0/1198115033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176aa30 0x562e61776180 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 -- 172.21.15.60:0/1198115033 >> 172.21.15.60:0/1198115033 conn(0x562e61768860 msgr2=0x562e6177d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 -- 172.21.15.60:0/1198115033 shutdown_connections 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.422+0000 7fb389dfe740 1 -- 172.21.15.60:0/1198115033 wait complete. 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.423+0000 7fb389dfe740 1 -- start start 2024-07-24T21:48:21.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.423+0000 7fb389dfe740 1 Processor -- start 2024-07-24T21:48:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e61783d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6176ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801aa510 con 0x562e6176b350 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ab980 con 0x562e6176aa30 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801aabe0 con 0x562e6177b6f0 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb389dfe740 10 client.0 _ll_register_callbacks cb 0x7fb37c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb384fe8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e61783d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb384fe8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e61783d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52460/0 (socket says 172.21.15.60:52460) 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51682/0 (socket says 172.21.15.60:51682) 2024-07-24T21:48:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb384fe8640 1 -- 172.21.15.60:0/2154024747 learned_addr learned my addr 172.21.15.60:0/2154024747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb385fea640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6176ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 -- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 msgr2=0x562e6176ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6176ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 -- 172.21.15.60:0/2154024747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 msgr2=0x562e61783d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e61783d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb3857e9640 1 -- 172.21.15.60:0/2154024747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6176d180 con 0x562e6176aa30 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.424+0000 7fb385fea640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6176ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb3857e9640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb37c01b040 tx=0x7fb37c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a296faa044995c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0054f0 con 0x562e6176aa30 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb37c0035f0 con 0x562e6176aa30 2024-07-24T21:48:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37c032070 con 0x562e6176aa30 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb37c044050 con 0x562e6176aa30 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb389dfe740 10 client.25547 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.425+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e617932d0 con 0x562e6176aa30 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb37c032070 con 0x562e6176aa30 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb37c0199b0 con 0x562e6176aa30 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 10 client.25547 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 10 client.25547 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 20 client.25547 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e617f6220 0x562e617f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 4 client.25547 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "713d8825", "release": true, "await": true}] 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "713d8825", "release": true, "await": true}) -- 0x562e616df7c0 con 0x562e617f6220 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.426+0000 7fb384fe8640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e617f6220 0x562e617f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.427+0000 7fb384fe8640 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e617f6220 0x562e617f8610 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7062a89378c9f0a9 server_cookie=ba99f1a6372ada3a in_seq=0 out_seq=0 2024-07-24T21:48:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.427+0000 7fb368ff9640 10 client.25547 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb368ff9640 1 -- 172.21.15.60:0/2154024747 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562e616df7c0 con 0x562e617f6220 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb368ff9640 10 client.25547 handle_command_reply: tid=0 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 client.25547 shutdown 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 msgr2=0x562e61783810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb37c01b040 tx=0x7fb37c005dc0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 shutdown_connections 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562e617f6220 0x562e617f8610 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb368ff9640 0 client.25547 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e6177b6f0 0x562e6176ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e6176b350 0x562e61783d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 --2- 172.21.15.60:0/2154024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6176aa30 0x562e61783810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 >> 172.21.15.60:0/2154024747 conn(0x562e61768860 msgr2=0x562e61779c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 shutdown_connections 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 1 -- 172.21.15.60:0/2154024747 wait complete. 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.432+0000 7fb389dfe740 20 client.25547 trim_cache size 0 max 16384 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 26, 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:48:21.435 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 8, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "713d8825": { 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 8, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.3, 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:48:21.436 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:21.437 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 msgr2=0x7fb36c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 0x7fb36c0b6200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb378009370 tx=0x7fb378004a40 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 msgr2=0x7fb3801990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9c0 tx=0x7fb37000ee80 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 shutdown_connections 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb36c0b3d40 0x7fb36c0b6200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3801079e0 0x7fb38019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb380107060 0x7fb3801990c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 --2- 172.21.15.60:0/3971954508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380101650 0x7fb380198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 >> 172.21.15.60:0/3971954508 conn(0x7fb3800fd100 msgr2=0x7fb38006ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:21.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 shutdown_connections 2024-07-24T21:48:21.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:21.433+0000 7fb387a4d640 1 -- 172.21.15.60:0/3971954508 wait complete. 2024-07-24T21:48:21.508 INFO:tasks.workunit.client.0.smithi060.stdout:config.status: creating Makefile 2024-07-24T21:48:21.528 INFO:tasks.workunit.client.0.smithi060.stdout:config.status: creating src/Makefile 2024-07-24T21:48:21.550 INFO:tasks.workunit.client.0.smithi060.stdout:config.status: creating man/Makefile 2024-07-24T21:48:21.562 INFO:tasks.workunit.client.0.smithi060.stdout:config.status: creating config.h 2024-07-24T21:48:21.617 INFO:tasks.workunit.client.0.smithi060.stdout:config.status: executing depfiles commands 2024-07-24T21:48:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:21 smithi123 ceph-mon[37143]: pgmap v427: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-07-24T21:48:21.756 INFO:tasks.workunit.client.0.smithi060.stderr:+ make 2024-07-24T21:48:21.765 INFO:tasks.workunit.client.0.smithi060.stdout:make all-recursive 2024-07-24T21:48:21.767 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-07-24T21:48:21.769 INFO:tasks.workunit.client.0.smithi060.stdout:Making all in src 2024-07-24T21:48:21.773 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-07-24T21:48:21.773 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-07-24T21:48:21.773 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-07-24T21:48:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:21 smithi086 ceph-mon[37128]: pgmap v427: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-07-24T21:48:21.932 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-07-24T21:48:21.933 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-07-24T21:48:22.012 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-07-24T21:48:22.012 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-07-24T21:48:22.075 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-07-24T21:48:22.075 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-07-24T21:48:22.128 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-07-24T21:48:22.129 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-07-24T21:48:22.190 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-07-24T21:48:22.190 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-07-24T21:48:22.238 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-07-24T21:48:22.238 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-07-24T21:48:22.353 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-07-24T21:48:22.353 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-07-24T21:48:22.389 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-07-24T21:48:22.389 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-07-24T21:48:22.425 INFO:tasks.workunit.client.0.smithi060.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-07-24T21:48:22.425 INFO:tasks.workunit.client.0.smithi060.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-07-24T21:48:22.463 INFO:tasks.workunit.client.0.smithi060.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-07-24T21:48:22.482 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-07-24T21:48:22.482 INFO:tasks.workunit.client.0.smithi060.stdout:Making all in man 2024-07-24T21:48:22.484 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-07-24T21:48:22.484 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Nothing to be done for 'all'. 2024-07-24T21:48:22.484 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-07-24T21:48:22.486 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-07-24T21:48:22.487 INFO:tasks.workunit.client.0.smithi060.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-07-24T21:48:22.487 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-07-24T21:48:22.487 INFO:tasks.workunit.client.0.smithi060.stderr:+ cd src 2024-07-24T21:48:22.488 INFO:tasks.workunit.client.0.smithi060.stderr:+ mkdir blogtest_in 2024-07-24T21:48:22.489 INFO:tasks.workunit.client.0.smithi060.stdout:running blogbench 2024-07-24T21:48:22.490 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'running blogbench' 2024-07-24T21:48:22.490 INFO:tasks.workunit.client.0.smithi060.stderr:+ ./blogbench -d blogtest_in 2024-07-24T21:48:22.490 INFO:tasks.workunit.client.0.smithi060.stdout: 2024-07-24T21:48:22.490 INFO:tasks.workunit.client.0.smithi060.stdout:Frequency = 10 secs 2024-07-24T21:48:22.490 INFO:tasks.workunit.client.0.smithi060.stdout:Scratch dir = [blogtest_in] 2024-07-24T21:48:23.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:23 smithi060 ceph-mon[33000]: pgmap v428: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-24T21:48:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:23 smithi123 ceph-mon[37143]: pgmap v428: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-24T21:48:23.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:23 smithi086 ceph-mon[37128]: pgmap v428: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-24T21:48:24.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:24 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:48:24.703 INFO:tasks.workunit.client.0.smithi060.stdout:Spawning 3 writers... 2024-07-24T21:48:24.704 INFO:tasks.workunit.client.0.smithi060.stdout:Spawning 1 rewriters... 2024-07-24T21:48:24.704 INFO:tasks.workunit.client.0.smithi060.stdout:Spawning 5 commenters... 2024-07-24T21:48:24.704 INFO:tasks.workunit.client.0.smithi060.stdout:Spawning 100 readers... 2024-07-24T21:48:24.707 INFO:tasks.workunit.client.0.smithi060.stdout:Benchmarking for 30 iterations. 2024-07-24T21:48:24.707 INFO:tasks.workunit.client.0.smithi060.stdout:The test will run during 5 minutes. 2024-07-24T21:48:24.707 INFO:tasks.workunit.client.0.smithi060.stdout: 2024-07-24T21:48:24.707 INFO:tasks.workunit.client.0.smithi060.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-07-24T21:48:24.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:24 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:48:24.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:24 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:48:25.557 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '713d8825', total seconds elapsed: 33.0 2024-07-24T21:48:25.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:25 smithi060 ceph-mon[33000]: pgmap v429: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-07-24T21:48:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:25 smithi123 ceph-mon[37143]: pgmap v429: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-07-24T21:48:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:25 smithi086 ceph-mon[37128]: pgmap v429: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-07-24T21:48:25.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 -- 172.21.15.60:0/2435360652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 msgr2=0x7fe52c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 --2- 172.21.15.60:0/2435360652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c10e8a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 -- 172.21.15.60:0/2435360652 shutdown_connections 2024-07-24T21:48:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 --2- 172.21.15.60:0/2435360652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c10ede0 0x7fe52c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 --2- 172.21.15.60:0/2435360652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c10e8a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 --2- 172.21.15.60:0/2435360652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 -- 172.21.15.60:0/2435360652 >> 172.21.15.60:0/2435360652 conn(0x7fe52c0fd100 msgr2=0x7fe52c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 -- 172.21.15.60:0/2435360652 shutdown_connections 2024-07-24T21:48:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.967+0000 7fe5316de640 1 -- 172.21.15.60:0/2435360652 wait complete. 2024-07-24T21:48:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 Processor -- start 2024-07-24T21:48:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 -- start start 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c10ede0 0x7fe52c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c114960 con 0x7fe52c101b50 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1147e0 con 0x7fe52c10ede0 2024-07-24T21:48:25.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe5316de640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c114ae0 con 0x7fe52c101230 2024-07-24T21:48:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe523fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51800/0 (socket says 172.21.15.60:51800) 2024-07-24T21:48:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 -- 172.21.15.60:0/3825712830 learned_addr learned my addr 172.21.15.60:0/3825712830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe523fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52214/0 (socket says 172.21.15.60:52214) 2024-07-24T21:48:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 -- 172.21.15.60:0/3825712830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c10ede0 msgr2=0x7fe52c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c10ede0 0x7fe52c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 -- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 msgr2=0x7fe52c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.968+0000 7fe52affd640 1 -- 172.21.15.60:0/3825712830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c1a0f40 con 0x7fe52c101230 2024-07-24T21:48:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe523fff640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe52affd640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d950 tx=0x7fe51c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe528ff9640 1 -- 172.21.15.60:0/3825712830 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c017030 con 0x7fe52c101230 2024-07-24T21:48:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1a1230 con 0x7fe52c101230 2024-07-24T21:48:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe528ff9640 1 -- 172.21.15.60:0/3825712830 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51c004020 con 0x7fe52c101230 2024-07-24T21:48:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe528ff9640 1 -- 172.21.15.60:0/3825712830 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0047a0 con 0x7fe52c101230 2024-07-24T21:48:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.969+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1a7ac0 con 0x7fe52c101230 2024-07-24T21:48:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe528ff9640 1 -- 172.21.15.60:0/3825712830 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe51c009a30 con 0x7fe52c101230 2024-07-24T21:48:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 Processor -- start 2024-07-24T21:48:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 -- start start 2024-07-24T21:48:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 0x562d28c669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562d28bd1960 con 0x562d28c5afa0 2024-07-24T21:48:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562d28bc0e60 con 0x562d28c5a680 2024-07-24T21:48:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe533a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d28bc4ea0 con 0x562d28c66f30 2024-07-24T21:48:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe52affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 0x562d28c669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.972+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59420/0 (socket says 172.21.15.60:59420) 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 -- 172.21.15.60:0/1512354864 learned_addr learned my addr 172.21.15.60:0/1512354864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe523fff640 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 -- 172.21.15.60:0/1512354864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 msgr2=0x562d28c69320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c69320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 -- 172.21.15.60:0/1512354864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 msgr2=0x562d28c669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 0x562d28c669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe528ff9640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 0x7fe5180b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 -- 172.21.15.60:0/1512354864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d28b4a2d0 con 0x562d28c5a680 2024-07-24T21:48:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52affd640 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 0x562d28c669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 0x7fe5180b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe528ff9640 1 -- 172.21.15.60:0/3825712830 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe51c0e3af0 con 0x7fe52c101230 2024-07-24T21:48:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe524009cd0 tx=0x7fe52400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be73648a19ed371 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.973+0000 7fe5217fa640 1 -- 172.21.15.60:0/1512354864 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240117b0 con 0x562d28c5a680 2024-07-24T21:48:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe5217fa640 1 -- 172.21.15.60:0/1512354864 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524016050 con 0x562d28c5a680 2024-07-24T21:48:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe5217fa640 1 -- 172.21.15.60:0/1512354864 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52401db00 con 0x562d28c5a680 2024-07-24T21:48:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 0x7fe5180b6200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe52401d000 tx=0x7fe524020000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:26.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 -- 172.21.15.60:0/1512354864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 msgr2=0x562d28c5aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe524009cd0 tx=0x7fe52400e6c0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 -- 172.21.15.60:0/1512354864 shutdown_connections 2024-07-24T21:48:26.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c69320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5afa0 0x562d28c669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 --2- 172.21.15.60:0/1512354864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5a680 0x562d28c5aa60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.974+0000 7fe533a8f740 1 -- 172.21.15.60:0/1512354864 >> 172.21.15.60:0/1512354864 conn(0x562d28c584b0 msgr2=0x562d28c59650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- 172.21.15.60:0/1512354864 shutdown_connections 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- 172.21.15.60:0/1512354864 wait complete. 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- start start 2024-07-24T21:48:26.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 Processor -- start 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c64390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562d28a24700 con 0x562d28c5a680 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562d28b635a0 con 0x562d28c5afa0 2024-07-24T21:48:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d28c69bb0 con 0x562d28c66f30 2024-07-24T21:48:26.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe52affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.975+0000 7fe533a8f740 10 client.0 _ll_register_callbacks cb 0x562d28b83910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:26.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52228/0 (socket says 172.21.15.60:52228) 2024-07-24T21:48:26.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 -- 172.21.15.60:0/2044116304 learned_addr learned my addr 172.21.15.60:0/2044116304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:26.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe523fff640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c64390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 -- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 msgr2=0x562d28c648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 -- 172.21.15.60:0/2044116304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 msgr2=0x562d28c64390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c64390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 -- 172.21.15.60:0/2044116304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d28b56630 con 0x562d28c5a680 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe523fff640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c64390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe52affd640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0a7970 tx=0x7fe51c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44df4e35ad934935 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe5127fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0a9e70 con 0x562d28c5a680 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe524016050 con 0x562d28c5a680 2024-07-24T21:48:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe51c0a6cf0 con 0x562d28c5a680 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe51c0aa210 con 0x562d28c5a680 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe533a8f740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.976+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d28921650 con 0x562d28c5a680 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524016050 con 0x562d28c5a680 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe51c0e14c0 con 0x562d28c5a680 2024-07-24T21:48:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 20 client.16056 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28c76a10 0x562d28c78e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 4 client.16056 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d28bd38b0 con 0x562d28c76a10 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe523fff640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28c76a10 0x562d28c78e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe523fff640 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28c76a10 0x562d28c78e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36dfdb5ae9f5b4e4 server_cookie=58bd20d76e7b266a in_seq=0 out_seq=0 2024-07-24T21:48:26.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.977+0000 7fe5127fc640 10 client.16056 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:26.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe5127fc640 1 -- 172.21.15.60:0/2044116304 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d28bd38b0 con 0x562d28c76a10 2024-07-24T21:48:26.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe5127fc640 10 client.16056 handle_command_reply: tid=0 2024-07-24T21:48:26.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 client.16056 shutdown 2024-07-24T21:48:26.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 msgr2=0x562d28c63e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0a7970 tx=0x7fe51c004590 comp rx=0 tx=0).stop 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 shutdown_connections 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28c76a10 0x562d28c78e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c64390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe5127fc640 0 client.16056 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 --2- 172.21.15.60:0/2044116304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c5a680 0x562d28c63e50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 >> 172.21.15.60:0/2044116304 conn(0x562d28c584b0 msgr2=0x562d28c5e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 shutdown_connections 2024-07-24T21:48:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 1 -- 172.21.15.60:0/2044116304 wait complete. 2024-07-24T21:48:26.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:25.983+0000 7fe533a8f740 20 client.16056 trim_cache size 0 max 16384 2024-07-24T21:48:26.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 Processor -- start 2024-07-24T21:48:26.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 -- start start 2024-07-24T21:48:26.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1b2c80 con 0x562d28c65ca0 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1b0550 con 0x562d28c5afa0 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.003+0000 7fe533a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1ae3a0 con 0x562d28c66f30 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52238/0 (socket says 172.21.15.60:52238) 2024-07-24T21:48:26.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 -- 172.21.15.60:0/796317574 learned_addr learned my addr 172.21.15.60:0/796317574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe523fff640 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52affd640 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 -- 172.21.15.60:0/796317574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 msgr2=0x562d28c7e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c7e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 -- 172.21.15.60:0/796317574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 msgr2=0x562d28c598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 -- 172.21.15.60:0/796317574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d28c5add0 con 0x562d28c65ca0 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe524023000 tx=0x7fe524024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b50516bfee3317b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe520ff9640 1 -- 172.21.15.60:0/796317574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240096b0 con 0x562d28c65ca0 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe520ff9640 1 -- 172.21.15.60:0/796317574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524009850 con 0x562d28c65ca0 2024-07-24T21:48:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.004+0000 7fe520ff9640 1 -- 172.21.15.60:0/796317574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524003e30 con 0x562d28c65ca0 2024-07-24T21:48:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 -- 172.21.15.60:0/796317574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 msgr2=0x562d28c7ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe524023000 tx=0x7fe524024000 comp rx=0 tx=0).stop 2024-07-24T21:48:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 -- 172.21.15.60:0/796317574 shutdown_connections 2024-07-24T21:48:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c65ca0 0x562d28c7ebc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c66f30 0x562d28c7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 --2- 172.21.15.60:0/796317574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 -- 172.21.15.60:0/796317574 >> 172.21.15.60:0/796317574 conn(0x562d28c584b0 msgr2=0x562d28c59190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 -- 172.21.15.60:0/796317574 shutdown_connections 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.005+0000 7fe533a8f740 1 -- 172.21.15.60:0/796317574 wait complete. 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 -- start start 2024-07-24T21:48:26.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 Processor -- start 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 0x562d28c6b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1aea10 con 0x562d28c66f30 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1afe80 con 0x562d28c5afa0 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1af0e0 con 0x562d28c65ca0 2024-07-24T21:48:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe533a8f740 10 client.0 _ll_register_callbacks cb 0x7fe51c0e7320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe523fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 0x562d28c6b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52246/0 (socket says 172.21.15.60:52246) 2024-07-24T21:48:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 -- 172.21.15.60:0/3206971989 learned_addr learned my addr 172.21.15.60:0/3206971989 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 -- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 msgr2=0x562d28c6b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 0x562d28c6b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 -- 172.21.15.60:0/3206971989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 msgr2=0x562d28c6f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c6f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.006+0000 7fe52b7fe640 1 -- 172.21.15.60:0/3206971989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d28c6c210 con 0x562d28c66f30 2024-07-24T21:48:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe523fff640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 0x562d28c6b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe52b7fe640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe52400ebc0 tx=0x7fe52401db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bf970e1be8fb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52403d070 con 0x562d28c66f30 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe5240119c0 con 0x562d28c66f30 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe524009850 con 0x562d28c66f30 2024-07-24T21:48:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe524049050 con 0x562d28c66f30 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 10 client.16062 fetch_fsmap learned FSMap version 26 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d28ce2420 con 0x562d28c66f30 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524009850 con 0x562d28c66f30 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe524033020 con 0x562d28c66f30 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 10 client.16062 fetch_fsmap finished waiting for FSMap version 26 2024-07-24T21:48:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 20 client.16062 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.007+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28ce28c0 0x562d28ce4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.008+0000 7fe533a8f740 4 client.16062 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:48:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.008+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d28bcf350 con 0x562d28ce28c0 2024-07-24T21:48:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.008+0000 7fe52affd640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28ce28c0 0x562d28ce4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.008+0000 7fe52affd640 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28ce28c0 0x562d28ce4ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb6095c5d6bc99d server_cookie=416e9a0382c9993a in_seq=0 out_seq=0 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.008+0000 7fe5127fc640 10 client.16062 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe5127fc640 1 -- 172.21.15.60:0/3206971989 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d28bcf350 con 0x562d28ce28c0 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe5127fc640 10 client.16062 handle_command_reply: tid=0 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 client.16062 shutdown 2024-07-24T21:48:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 msgr2=0x562d28c6bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe52400ebc0 tx=0x7fe52401db90 comp rx=0 tx=0).stop 2024-07-24T21:48:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 shutdown_connections 2024-07-24T21:48:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562d28ce28c0 0x562d28ce4ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d28c66f30 0x562d28c6bc50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d28c65ca0 0x562d28c6b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 --2- 172.21.15.60:0/3206971989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d28c5afa0 0x562d28c6f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe5127fc640 0 client.16062 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.013+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 >> 172.21.15.60:0/3206971989 conn(0x562d28c584b0 msgr2=0x562d28c63dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 shutdown_connections 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe533a8f740 1 -- 172.21.15.60:0/3206971989 wait complete. 2024-07-24T21:48:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe533a8f740 20 client.16062 trim_cache size 0 max 16384 2024-07-24T21:48:26.039 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:48:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:48:26.039 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 msgr2=0x7fe5180b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 0x7fe5180b6200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe52401d000 tx=0x7fe524020000 comp rx=0 tx=0).stop 2024-07-24T21:48:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 msgr2=0x7fe52c19ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d950 tx=0x7fe51c00de10 comp rx=0 tx=0).stop 2024-07-24T21:48:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 shutdown_connections 2024-07-24T21:48:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe5180b3d40 0x7fe5180b6200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c10ede0 0x7fe52c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52c101b50 0x7fe52c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 --2- 172.21.15.60:0/3825712830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52c101230 0x7fe52c19ae90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 >> 172.21.15.60:0/3825712830 conn(0x7fe52c0fd100 msgr2=0x7fe52c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 shutdown_connections 2024-07-24T21:48:26.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:26.014+0000 7fe5316de640 1 -- 172.21.15.60:0/3825712830 wait complete. 2024-07-24T21:48:26.837 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:48:26.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:48:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:27.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:27 smithi060 ceph-mon[33000]: pgmap v430: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-07-24T21:48:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:27 smithi123 ceph-mon[37143]: pgmap v430: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-07-24T21:48:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:27 smithi086 ceph-mon[37128]: pgmap v430: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-07-24T21:48:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 -- 172.21.15.60:0/3370522104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0107060 msgr2=0x7f10c01074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3370522104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0107060 0x7f10c01074a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009900 tx=0x7f10b002a410 comp rx=0 tx=0).stop 2024-07-24T21:48:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 -- 172.21.15.60:0/3370522104 shutdown_connections 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3370522104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c01079e0 0x7f10c010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3370522104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0107060 0x7f10c01074a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3370522104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c0101650 0x7f10c0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.883+0000 7f10c7a93640 1 -- 172.21.15.60:0/3370522104 >> 172.21.15.60:0/3370522104 conn(0x7f10c00fd0c0 msgr2=0x7f10c00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 -- 172.21.15.60:0/3370522104 shutdown_connections 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 -- 172.21.15.60:0/3370522104 wait complete. 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 Processor -- start 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 -- start start 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 0x7f10c00fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c01079e0 0x7f10c00fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.884+0000 7f10c7a93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111600 con 0x7f10c0101650 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c7a93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111480 con 0x7f10c0107060 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c7a93640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111780 con 0x7f10c01079e0 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52268/0 (socket says 172.21.15.60:52268) 2024-07-24T21:48:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 -- 172.21.15.60:0/3918662371 learned_addr learned my addr 172.21.15.60:0/3918662371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c6030640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c01079e0 0x7f10c00fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c502e640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 0x7f10c00fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 -- 172.21.15.60:0/3918662371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c01079e0 msgr2=0x7f10c00fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c01079e0 0x7f10c00fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 -- 172.21.15.60:0/3918662371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 msgr2=0x7f10c00fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 0x7f10c00fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 -- 172.21.15.60:0/3918662371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01a70a0 con 0x7f10c0101650 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c502e640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 0x7f10c00fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c582f640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e9f0 tx=0x7f10bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00cd80 con 0x7f10c0101650 2024-07-24T21:48:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a7390 con 0x7f10c0101650 2024-07-24T21:48:28.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc0054d0 con 0x7f10c0101650 2024-07-24T21:48:28.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc010690 con 0x7f10c0101650 2024-07-24T21:48:28.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.885+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a7850 con 0x7f10c0101650 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.886+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c0101a30 con 0x7f10c0101650 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.888+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f10bc004e30 con 0x7f10c0101650 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.889+0000 7f10b6ffd640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 0x7f10ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.889+0000 7f10c502e640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 0x7f10ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.889+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f10bc0e46f0 con 0x7f10c0101650 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.889+0000 7f10c502e640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 0x7f10ac0b6200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f10b0031440 tx=0x7f10b0005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:28.891+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f10bc0555e0 con 0x7f10c0101650 2024-07-24T21:48:29.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.156+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c01a7b70 con 0x7f10c0101650 2024-07-24T21:48:29.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.156+0000 7f10b6ffd640 1 -- 172.21.15.60:0/3918662371 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f10bc0aba00 con 0x7f10c0101650 2024-07-24T21:48:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:48:29.159 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-07-24T21:46:19:394153+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:46:18.571687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:48:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-07-24T21:48:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 msgr2=0x7f10ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 0x7f10ac0b6200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f10b0031440 tx=0x7f10b0005a20 comp rx=0 tx=0).stop 2024-07-24T21:48:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 msgr2=0x7f10c00fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e9f0 tx=0x7f10bc00edf0 comp rx=0 tx=0).stop 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 shutdown_connections 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f10ac0b3d40 0x7f10ac0b6200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c01079e0 0x7f10c00fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0107060 0x7f10c00fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 --2- 172.21.15.60:0/3918662371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c0101650 0x7f10c00fd5c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.158+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 >> 172.21.15.60:0/3918662371 conn(0x7f10c00fd0c0 msgr2=0x7f10c00fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.159+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 shutdown_connections 2024-07-24T21:48:29.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:29.159+0000 7f10c7a93640 1 -- 172.21.15.60:0/3918662371 wait complete. 2024-07-24T21:48:29.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:29 smithi060 ceph-mon[33000]: pgmap v431: 97 pgs: 97 active+clean; 6.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 139 KiB/s wr, 44 op/s 2024-07-24T21:48:29.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3918662371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:29 smithi123 ceph-mon[37143]: pgmap v431: 97 pgs: 97 active+clean; 6.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 139 KiB/s wr, 44 op/s 2024-07-24T21:48:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3918662371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:29 smithi086 ceph-mon[37128]: pgmap v431: 97 pgs: 97 active+clean; 6.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 139 KiB/s wr, 44 op/s 2024-07-24T21:48:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3918662371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:48:29.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:48:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:30.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:30 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 -- 172.21.15.60:0/232213451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 msgr2=0x7f857410e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 --2- 172.21.15.60:0/232213451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857410e110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8564009900 tx=0x7f856402a410 comp rx=0 tx=0).stop 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 -- 172.21.15.60:0/232213451 shutdown_connections 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 --2- 172.21.15.60:0/232213451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857410e110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 --2- 172.21.15.60:0/232213451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8574106f90 0x7f85741073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 --2- 172.21.15.60:0/232213451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85741015e0 0x7f85741019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 -- 172.21.15.60:0/232213451 >> 172.21.15.60:0/232213451 conn(0x7f85740782e0 msgr2=0x7f85740ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 -- 172.21.15.60:0/232213451 shutdown_connections 2024-07-24T21:48:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.604+0000 7f85794cb640 1 -- 172.21.15.60:0/232213451 wait complete. 2024-07-24T21:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.605+0000 7f85794cb640 1 Processor -- start 2024-07-24T21:48:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 -- start start 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85741015e0 0x7f8574197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111540 con 0x7f8574106f90 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85741113c0 con 0x7f8574107910 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85794cb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85741116c0 con 0x7f85741015e0 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52286/0 (socket says 172.21.15.60:52286) 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 -- 172.21.15.60:0/1418257617 learned_addr learned my addr 172.21.15.60:0/1418257617 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f8572ffd640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85741015e0 0x7f8574197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 -- 172.21.15.60:0/1418257617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85741015e0 msgr2=0x7f8574197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85741015e0 0x7f8574197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 -- 172.21.15.60:0/1418257617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 msgr2=0x7f857419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85727fc640 1 -- 172.21.15.60:0/1418257617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857419db40 con 0x7f8574106f90 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.606+0000 7f85737fe640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f85727fc640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e9f0 tx=0x7f855c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f856bfff640 1 -- 172.21.15.60:0/1418257617 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c00cd80 con 0x7f8574106f90 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857419de30 con 0x7f8574106f90 2024-07-24T21:48:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f856bfff640 1 -- 172.21.15.60:0/1418257617 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c0108e0 con 0x7f8574106f90 2024-07-24T21:48:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f856bfff640 1 -- 172.21.15.60:0/1418257617 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c018980 con 0x7f8574106f90 2024-07-24T21:48:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.607+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857406a3d0 con 0x7f8574106f90 2024-07-24T21:48:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.611+0000 7f856bfff640 1 -- 172.21.15.60:0/1418257617 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f855c010430 con 0x7f8574106f90 2024-07-24T21:48:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.611+0000 7f857c21e740 1 Processor -- start 2024-07-24T21:48:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.611+0000 7f857c21e740 1 -- start start 2024-07-24T21:48:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd189170 0x5642fd18b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5642fd0f2280 con 0x5642fd17c8c0 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642fd0e1780 con 0x5642fd189170 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f857c21e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642fd0e57c0 con 0x5642fd17d1e0 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f85737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f85727fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd189170 0x5642fd18b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51880/0 (socket says 172.21.15.60:51880) 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f85737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52302/0 (socket says 172.21.15.60:52302) 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 -- 172.21.15.60:0/2559861379 learned_addr learned my addr 172.21.15.60:0/2559861379 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 -- 172.21.15.60:0/2559861379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd189170 msgr2=0x5642fd18b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd189170 0x5642fd18b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 -- 172.21.15.60:0/2559861379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 msgr2=0x5642fd17cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f8572ffd640 1 -- 172.21.15.60:0/2559861379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fce45790 con 0x5642fd17d1e0 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f85737fe640 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.612+0000 7f856bfff640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 0x7f85600b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f8572ffd640 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f856c0114f0 tx=0x7f856c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52f945318aa0f9f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f85737fe640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 0x7f85600b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f856bfff640 1 -- 172.21.15.60:0/1418257617 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f855c0e4310 con 0x7f8574106f90 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f85697fa640 1 -- 172.21.15.60:0/2559861379 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c01e070 con 0x5642fd17d1e0 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f85697fa640 1 -- 172.21.15.60:0/2559861379 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856c00fad0 con 0x5642fd17d1e0 2024-07-24T21:48:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.613+0000 7f85697fa640 1 -- 172.21.15.60:0/2559861379 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c019b10 con 0x5642fd17d1e0 2024-07-24T21:48:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- 172.21.15.60:0/2559861379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 msgr2=0x5642fd188c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f85737fe640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 0x7f85600b6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8564006010 tx=0x7f85640058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f856c0114f0 tx=0x7f856c0119b0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- 172.21.15.60:0/2559861379 shutdown_connections 2024-07-24T21:48:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd189170 0x5642fd18b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd17d1e0 0x5642fd188c30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- 172.21.15.60:0/2559861379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd17cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- 172.21.15.60:0/2559861379 >> 172.21.15.60:0/2559861379 conn(0x5642fd17a6f0 msgr2=0x5642fd17b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- 172.21.15.60:0/2559861379 shutdown_connections 2024-07-24T21:48:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- 172.21.15.60:0/2559861379 wait complete. 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- start start 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 Processor -- start 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd185d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5642fd01bab0 con 0x5642fd17d1e0 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642fd0699e0 con 0x5642fd17c8c0 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.614+0000 7f857c21e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642fd18be50 con 0x5642fd189170 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f857c21e740 10 client.0 _ll_register_callbacks cb 0x5642fcdf1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd185d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59496/0 (socket says 172.21.15.60:59496) 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd185d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52310/0 (socket says 172.21.15.60:52310) 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f85737fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 -- 172.21.15.60:0/3789621038 learned_addr learned my addr 172.21.15.60:0/3789621038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 -- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 msgr2=0x5642fd1862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 -- 172.21.15.60:0/3789621038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 msgr2=0x5642fd185d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd185d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 -- 172.21.15.60:0/3789621038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fcebe620 con 0x5642fd17c8c0 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f856c0092b0 tx=0x7f856c0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c2e091f142b4c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f855a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c01e040 con 0x5642fd17c8c0 2024-07-24T21:48:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f856c019810 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.615+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642fd090ce0 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f856c031050 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 10 client.25580 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642fd197600 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5642fd090ce0 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f856c02c030 con 0x5642fd17c8c0 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 10 client.25580 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 20 client.25580 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd198a00 0x5642fd19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 4 client.25580 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642fd0f41d0 con 0x5642fd198a00 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.616+0000 7f85727fc640 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd198a00 0x5642fd19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.617+0000 7f855a7fc640 10 client.25580 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.617+0000 7f85727fc640 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd198a00 0x5642fd19adf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450fb68380f159f3 server_cookie=219b98a8d618fffa in_seq=0 out_seq=0 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.618+0000 7f855a7fc640 1 -- 172.21.15.60:0/3789621038 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642fd0f41d0 con 0x5642fd198a00 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.618+0000 7f855a7fc640 10 client.25580 handle_command_reply: tid=0 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.618+0000 7f857c21e740 1 client.25580 shutdown 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 msgr2=0x5642fd1888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f856c0092b0 tx=0x7f856c0092e0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 shutdown_connections 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd198a00 0x5642fd19adf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f855a7fc640 0 client.25580 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd185d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 --2- 172.21.15.60:0/3789621038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1888b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 >> 172.21.15.60:0/3789621038 conn(0x5642fd17a6f0 msgr2=0x5642fd180290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 shutdown_connections 2024-07-24T21:48:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 1 -- 172.21.15.60:0/3789621038 wait complete. 2024-07-24T21:48:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.619+0000 7f857c21e740 20 client.25580 trim_cache size 0 max 16384 2024-07-24T21:48:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 Processor -- start 2024-07-24T21:48:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 -- start start 2024-07-24T21:48:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f85741b2c50 con 0x5642fd17d1e0 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85741b0520 con 0x5642fd17c8c0 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f857c21e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85741ae370 con 0x5642fd189170 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f85727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52326/0 (socket says 172.21.15.60:52326) 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f85727fc640 1 -- 172.21.15.60:0/270215733 learned_addr learned my addr 172.21.15.60:0/270215733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.649+0000 7f85737fe640 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f8572ffd640 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 -- 172.21.15.60:0/270215733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 msgr2=0x5642fd1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 -- 172.21.15.60:0/270215733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 msgr2=0x5642fd1845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 -- 172.21.15.60:0/270215733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fd17d5c0 con 0x5642fd17d1e0 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f85727fc640 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f855c0e2060 tx=0x7f855c0e7e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f469fe4ae4ebc6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f8568ff9640 1 -- 172.21.15.60:0/270215733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0aa740 con 0x5642fd17d1e0 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f8568ff9640 1 -- 172.21.15.60:0/270215733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c010720 con 0x5642fd17d1e0 2024-07-24T21:48:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.650+0000 7f8568ff9640 1 -- 172.21.15.60:0/270215733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0e1ce0 con 0x5642fd17d1e0 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- 172.21.15.60:0/270215733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 msgr2=0x5642fd184b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f855c0e2060 tx=0x7f855c0e7e60 comp rx=0 tx=0).stop 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- 172.21.15.60:0/270215733 shutdown_connections 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17d1e0 0x5642fd184b30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 --2- 172.21.15.60:0/270215733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17c8c0 0x5642fd1845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- 172.21.15.60:0/270215733 >> 172.21.15.60:0/270215733 conn(0x5642fd17a6f0 msgr2=0x5642fd185d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- 172.21.15.60:0/270215733 shutdown_connections 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- 172.21.15.60:0/270215733 wait complete. 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 -- start start 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.651+0000 7f857c21e740 1 Processor -- start 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17d1e0 0x5642fd18c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd18c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f85741ae9e0 con 0x5642fd17c8c0 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85741afe50 con 0x5642fd17d1e0 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85741af0b0 con 0x5642fd189170 2024-07-24T21:48:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 10 client.0 _ll_register_callbacks cb 0x7f856c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52332/0 (socket says 172.21.15.60:52332) 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 -- 172.21.15.60:0/3091472043 learned_addr learned my addr 172.21.15.60:0/3091472043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f85737fe640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd18c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f85727fc640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17d1e0 0x5642fd18c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 -- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 msgr2=0x5642fd18c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd18c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 -- 172.21.15.60:0/3091472043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17d1e0 msgr2=0x5642fd18c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17d1e0 0x5642fd18c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 -- 172.21.15.60:0/3091472043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fd18ccb0 con 0x5642fd17c8c0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f85737fe640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd18c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f8572ffd640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f856c0196b0 tx=0x7f856c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3b924da317ad3e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f855a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c01e040 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f856c005730 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.652+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642fd0efc70 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f856c005970 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642fd202420 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5642fd0efc70 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f856c02c070 con 0x5642fd17c8c0 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 20 client.16086 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd202bf0 0x5642fd204fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 4 client.16086 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:48:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642fd0ed3b0 con 0x5642fd202bf0 2024-07-24T21:48:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.653+0000 7f85727fc640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd202bf0 0x5642fd204fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.654+0000 7f855a7fc640 10 client.16086 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:30.654+0000 7f85727fc640 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd202bf0 0x5642fd204fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48806300a7fc45e server_cookie=9205b5fea1ca3864 in_seq=0 out_seq=0 2024-07-24T21:48:30.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:30 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:30.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:30 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:31.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:31 smithi060 ceph-mon[33000]: pgmap v432: 97 pgs: 97 active+clean; 14 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 824 KiB/s wr, 46 op/s 2024-07-24T21:48:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:31 smithi123 ceph-mon[37143]: pgmap v432: 97 pgs: 97 active+clean; 14 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 824 KiB/s wr, 46 op/s 2024-07-24T21:48:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:31 smithi086 ceph-mon[37128]: pgmap v432: 97 pgs: 97 active+clean; 14 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 824 KiB/s wr, 46 op/s 2024-07-24T21:48:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.853+0000 7f855a7fc640 1 -- 172.21.15.60:0/3091472043 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642fd0ed3b0 con 0x5642fd202bf0 2024-07-24T21:48:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.853+0000 7f855a7fc640 10 client.16086 handle_command_reply: tid=0 2024-07-24T21:48:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.853+0000 7f857c21e740 1 client.16086 shutdown 2024-07-24T21:48:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.853+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 msgr2=0x5642fd191620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.853+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f856c0196b0 tx=0x7f856c009da0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 shutdown_connections 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5642fd202bf0 0x5642fd204fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642fd189170 0x5642fd18c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f855a7fc640 0 client.16086 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fd17d1e0 0x5642fd18c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 --2- 172.21.15.60:0/3091472043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5642fd17c8c0 0x5642fd191620 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 >> 172.21.15.60:0/3091472043 conn(0x5642fd17a6f0 msgr2=0x5642fd1797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 shutdown_connections 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 1 -- 172.21.15.60:0/3091472043 wait complete. 2024-07-24T21:48:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f857c21e740 20 client.16086 trim_cache size 0 max 16384 2024-07-24T21:48:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:48:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:48:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:48:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 msgr2=0x7f85600b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 0x7f85600b6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8564006010 tx=0x7f85640058d0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 msgr2=0x7f8574197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e9f0 tx=0x7f855c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 shutdown_connections 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f85600b3d40 0x7f85600b6200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8574107910 0x7f857419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8574106f90 0x7f8574197fe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 --2- 172.21.15.60:0/1418257617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85741015e0 0x7f8574197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.854+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 >> 172.21.15.60:0/1418257617 conn(0x7f85740782e0 msgr2=0x7f8574105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.855+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 shutdown_connections 2024-07-24T21:48:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:31.855+0000 7f85794cb640 1 -- 172.21.15.60:0/1418257617 wait complete. 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 -- 172.21.15.60:0/1783960051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 msgr2=0x7f0ed81074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 --2- 172.21.15.60:0/1783960051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed81074a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009900 tx=0x7f0ec802a410 comp rx=0 tx=0).stop 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 -- 172.21.15.60:0/1783960051 shutdown_connections 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 --2- 172.21.15.60:0/1783960051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed81079e0 0x7f0ed810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 --2- 172.21.15.60:0/1783960051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed81074a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 --2- 172.21.15.60:0/1783960051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed8101650 0x7f0ed8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 -- 172.21.15.60:0/1783960051 >> 172.21.15.60:0/1783960051 conn(0x7f0ed80fd0c0 msgr2=0x7f0ed80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 -- 172.21.15.60:0/1783960051 shutdown_connections 2024-07-24T21:48:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.262+0000 7f0edf105640 1 -- 172.21.15.60:0/1783960051 wait complete. 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 Processor -- start 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 -- start start 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 0x7f0ed810ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed81079e0 0x7f0ed810f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8112750 con 0x7f0ed8107060 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81125d0 con 0x7f0ed81079e0 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edf105640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81128d0 con 0x7f0ed8101650 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 0x7f0ed810ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52360/0 (socket says 172.21.15.60:52360) 2024-07-24T21:48:32.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0ecffff640 1 -- 172.21.15.60:0/3929220029 learned_addr learned my addr 172.21.15.60:0/3929220029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed81079e0 0x7f0ed810f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 -- 172.21.15.60:0/3929220029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed81079e0 msgr2=0x7f0ed810f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed81079e0 0x7f0ed810f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 -- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 msgr2=0x7f0ed810f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0edcea1640 1 -- 172.21.15.60:0/3929220029 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed81a7040 con 0x7f0ed8101650 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.263+0000 7f0ecffff640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0edcea1640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 0x7f0ed810ecf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400e9f0 tx=0x7f0ed400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0ecdffb640 1 -- 172.21.15.60:0/3929220029 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400ccc0 con 0x7f0ed8101650 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0ecdffb640 1 -- 172.21.15.60:0/3929220029 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed40108e0 con 0x7f0ed8101650 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed81a7330 con 0x7f0ed8101650 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0ecdffb640 1 -- 172.21.15.60:0/3929220029 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4018880 con 0x7f0ed8101650 2024-07-24T21:48:32.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.264+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed81a7890 con 0x7f0ed8101650 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ecdffb640 1 -- 172.21.15.60:0/3929220029 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0ed4010430 con 0x7f0ed8101650 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 Processor -- start 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 -- start start 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7bcf80 0x55f33f7bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f33f7279b0 con 0x55f33f7bcf80 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f33f716eb0 con 0x55f33f7b0ff0 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.267+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f33f71aef0 con 0x55f33f7b06d0 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edcea1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7bcf80 0x55f33f7bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51948/0 (socket says 172.21.15.60:51948) 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2442977569 learned_addr learned my addr 172.21.15.60:0/2442977569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2442977569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 msgr2=0x55f33f7bca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7bca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2442977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7bcf80 msgr2=0x55f33f7bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7bcf80 0x55f33f7bf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2442977569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33f6a0320 con 0x55f33f7b06d0 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ecdffb640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 0x7f0ec00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edcea1640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 0x7f0ec00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009f90 tx=0x7f0ed00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736f86e2f044e545 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ecdffb640 1 -- 172.21.15.60:0/3929220029 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0ed40e4340 con 0x7f0ed8101650 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ebeffd640 1 -- 172.21.15.60:0/2442977569 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0004d20 con 0x55f33f7b06d0 2024-07-24T21:48:32.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ebeffd640 1 -- 172.21.15.60:0/2442977569 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0004050 con 0x55f33f7b06d0 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.268+0000 7f0ebeffd640 1 -- 172.21.15.60:0/2442977569 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0005280 con 0x55f33f7b06d0 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2442977569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 msgr2=0x55f33f7b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0edcea1640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 0x7f0ec00b6280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55f33f7af3e0 tx=0x7f0ed40e7df0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009f90 tx=0x7f0ed00118d0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2442977569 shutdown_connections 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7bcf80 0x55f33f7bf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7bca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2442977569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b06d0 0x55f33f7b0ab0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2442977569 >> 172.21.15.60:0/2442977569 conn(0x55f33f7ae500 msgr2=0x55f33f7af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2442977569 shutdown_connections 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.269+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2442977569 wait complete. 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 -- start start 2024-07-24T21:48:32.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 Processor -- start 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7bcf80 0x55f33f7ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f33f57a700 con 0x55f33f7b06d0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f33f6b95f0 con 0x55f33f7b0ff0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f33f7bfc00 con 0x55f33f7bcf80 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ee14b6740 10 client.0 _ll_register_callbacks cb 0x55f33f6d9960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52384/0 (socket says 172.21.15.60:52384) 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0edd6a2640 1 -- 172.21.15.60:0/80070419 learned_addr learned my addr 172.21.15.60:0/80070419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0ecffff640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.270+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7bcf80 0x55f33f7ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 -- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7bcf80 msgr2=0x55f33f7ba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7bcf80 0x55f33f7ba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 -- 172.21.15.60:0/80070419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 msgr2=0x55f33f7b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 -- 172.21.15.60:0/80070419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33f6ac680 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009250 tx=0x7f0ed0004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a86652850c1ef7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0eb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002b020 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0ed0005040 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed0004050 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ed0036050 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0ee14b6740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f33f477650 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.271+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0004050 con 0x55f33f7b06d0 2024-07-24T21:48:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed000f6a0 con 0x55f33f7b06d0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 10 client.16098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 20 client.16098 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f7ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 4 client.16098 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f33f729900 con 0x55f33f7cc9a0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ecffff640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f7ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0ecffff640 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f7ced90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c79d9b2f5a0dbc9 server_cookie=fb0fafbb1979925d in_seq=0 out_seq=0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.272+0000 7f0eb7fff640 10 client.16098 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0eb7fff640 1 -- 172.21.15.60:0/80070419 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f33f729900 con 0x55f33f7cc9a0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0eb7fff640 10 client.16098 handle_command_reply: tid=0 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 client.16098 shutdown 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 msgr2=0x55f33f7bca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009250 tx=0x7f0ed0004820 comp rx=0 tx=0).stop 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 shutdown_connections 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f7ced90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7bcf80 0x55f33f7ba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0eb7fff640 0 client.16098 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7b0ff0 0x55f33f7b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/80070419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7bca30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 >> 172.21.15.60:0/80070419 conn(0x55f33f7ae500 msgr2=0x55f33f7b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.273+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 shutdown_connections 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.274+0000 7f0ee14b6740 1 -- 172.21.15.60:0/80070419 wait complete. 2024-07-24T21:48:32.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.274+0000 7f0ee14b6740 20 client.16098 trim_cache size 0 max 16384 2024-07-24T21:48:32.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 Processor -- start 2024-07-24T21:48:32.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 -- start start 2024-07-24T21:48:32.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81b2bb0 con 0x55f33f7b06d0 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81b0480 con 0x55f33f7bcf80 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ae2d0 con 0x55f33f7b0ff0 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0edd6a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52386/0 (socket says 172.21.15.60:52386) 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2557079838 learned_addr learned my addr 172.21.15.60:0/2557079838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0ecffff640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.291+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2557079838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 msgr2=0x55f33f7d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2557079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 msgr2=0x55f33f7b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 -- 172.21.15.60:0/2557079838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33f7ba9c0 con 0x55f33f7b06d0 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0ecffff640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009560 tx=0x7f0ed0004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ab13b3ba8b310c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0ebe7fc640 1 -- 172.21.15.60:0/2557079838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002b040 con 0x55f33f7b06d0 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0ebe7fc640 1 -- 172.21.15.60:0/2557079838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0003c40 con 0x55f33f7b06d0 2024-07-24T21:48:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.292+0000 7f0ebe7fc640 1 -- 172.21.15.60:0/2557079838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00277f0 con 0x55f33f7b06d0 2024-07-24T21:48:32.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2557079838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 msgr2=0x55f33f7d4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009560 tx=0x7f0ed0004b10 comp rx=0 tx=0).stop 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2557079838 shutdown_connections 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/2557079838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7d4520 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2557079838 >> 172.21.15.60:0/2557079838 conn(0x55f33f7ae500 msgr2=0x55f33f7bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2557079838 shutdown_connections 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.293+0000 7f0ee14b6740 1 -- 172.21.15.60:0/2557079838 wait complete. 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 -- start start 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 Processor -- start 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ae940 con 0x55f33f7b06d0 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81afdb0 con 0x55f33f7bcf80 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81af010 con 0x55f33f7b0ff0 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ee14b6740 10 client.0 _ll_register_callbacks cb 0x7f0ed00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.294+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51976/0 (socket says 172.21.15.60:51976) 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 -- 172.21.15.60:0/3443699202 learned_addr learned my addr 172.21.15.60:0/3443699202 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:32.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 -- 172.21.15.60:0/3443699202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 msgr2=0x55f33f7c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 -- 172.21.15.60:0/3443699202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 msgr2=0x55f33f7c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0ecffff640 1 -- 172.21.15.60:0/3443699202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33f7c5ac0 con 0x55f33f7b0ff0 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.295+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:48:32.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0ecffff640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec80042a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9e5f2e667fb04b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0eb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8035040 con 0x55f33f7b0ff0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0ec8005220 con 0x55f33f7b0ff0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed0003c40 con 0x55f33f7b0ff0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ec80044d0 con 0x55f33f7b0ff0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0ee14b6740 10 client.25609 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.296+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f33f837b60 con 0x55f33f7b0ff0 2024-07-24T21:48:32.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0003c40 con 0x55f33f7b0ff0 2024-07-24T21:48:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ec8004a50 con 0x55f33f7b0ff0 2024-07-24T21:48:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 10 client.25609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:48:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 20 client.25609 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f83a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 4 client.25609 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f33f7253a0 con 0x55f33f7cc9a0 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f83a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.297+0000 7f0eb7fff640 10 client.25609 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0edd6a2640 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f83a300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7503b550b8dc9cb0 server_cookie=1b0cadde4ea37b37 in_seq=0 out_seq=0 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0eb7fff640 1 -- 172.21.15.60:0/3443699202 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f33f7253a0 con 0x55f33f7cc9a0 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0eb7fff640 10 client.25609 handle_command_reply: tid=0 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0ee14b6740 1 client.25609 shutdown 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 msgr2=0x55f33f7c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec80042a0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 shutdown_connections 2024-07-24T21:48:32.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.298+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f33f7cc9a0 0x55f33f83a300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f33f7bcf80 0x55f33f7c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f33f7b0ff0 0x55f33f7c53d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 --2- 172.21.15.60:0/3443699202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33f7b06d0 0x55f33f7c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 >> 172.21.15.60:0/3443699202 conn(0x55f33f7ae500 msgr2=0x55f33f7b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0eb7fff640 0 client.25609 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 shutdown_connections 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 1 -- 172.21.15.60:0/3443699202 wait complete. 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.299+0000 7f0ee14b6740 20 client.25609 trim_cache size 0 max 16384 2024-07-24T21:48:32.305 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 msgr2=0x7f0ec00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 0x7f0ec00b6280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55f33f7af3e0 tx=0x7f0ed40e7df0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 msgr2=0x7f0ed810ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 0x7f0ed810ecf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400e9f0 tx=0x7f0ed400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 shutdown_connections 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ec00b3dc0 0x7f0ec00b6280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed81079e0 0x7f0ed810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ed8107060 0x7f0ed810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 --2- 172.21.15.60:0/3929220029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed8101650 0x7f0ed810ecf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 >> 172.21.15.60:0/3929220029 conn(0x7f0ed80fd0c0 msgr2=0x7f0ed80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 shutdown_connections 2024-07-24T21:48:32.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:32.300+0000 7f0edf105640 1 -- 172.21.15.60:0/3929220029 wait complete. 2024-07-24T21:48:33.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:48:33.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:33 smithi060 ceph-mon[33000]: pgmap v433: 97 pgs: 97 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-07-24T21:48:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:33.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:33 smithi123 ceph-mon[37143]: pgmap v433: 97 pgs: 97 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-07-24T21:48:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:33 smithi086 ceph-mon[37128]: pgmap v433: 97 pgs: 97 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-07-24T21:48:34.707 INFO:tasks.workunit.client.0.smithi060.stdout: 17 82572 1090 32148 696 18887 692 2024-07-24T21:48:35.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 -- 172.21.15.60:0/1830622718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 msgr2=0x7f69c8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/1830622718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8101a30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009900 tx=0x7f69b802a410 comp rx=0 tx=0).stop 2024-07-24T21:48:35.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 -- 172.21.15.60:0/1830622718 shutdown_connections 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/1830622718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 0x7f69c810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/1830622718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 0x7f69c81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/1830622718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8101a30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 -- 172.21.15.60:0/1830622718 >> 172.21.15.60:0/1830622718 conn(0x7f69c80fd0c0 msgr2=0x7f69c80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 -- 172.21.15.60:0/1830622718 shutdown_connections 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 -- 172.21.15.60:0/1830622718 wait complete. 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.391+0000 7f69ceb8f640 1 Processor -- start 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 -- start start 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 0x7f69c8103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 0x7f69c8103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81115f0 con 0x7f69c8101650 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8111470 con 0x7f69c81079e0 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69ceb8f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8111770 con 0x7f69c8107060 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52420/0 (socket says 172.21.15.60:52420) 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 -- 172.21.15.60:0/3416225485 learned_addr learned my addr 172.21.15.60:0/3416225485 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69c7fff640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 0x7f69c8103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cd12c640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 0x7f69c8103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 -- 172.21.15.60:0/3416225485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 msgr2=0x7f69c8103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 0x7f69c8103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 -- 172.21.15.60:0/3416225485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 msgr2=0x7f69c8103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 0x7f69c8103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.392+0000 7f69cc92b640 1 -- 172.21.15.60:0/3416225485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c81a70f0 con 0x7f69c8101650 2024-07-24T21:48:35.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f69b8005940 tx=0x7f69b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69c5ffb640 1 -- 172.21.15.60:0/3416225485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b803f030 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69c5ffb640 1 -- 172.21.15.60:0/3416225485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b8003aa0 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c81a7380 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69c5ffb640 1 -- 172.21.15.60:0/3416225485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8035070 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.393+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c81a78b0 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 Processor -- start 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 -- start start 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c5ffb640 1 -- 172.21.15.60:0/3416225485 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f69b8042030 con 0x7f69c8101650 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 0x55aa773efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa7735a9b0 con 0x55aa773e3ff0 2024-07-24T21:48:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa77349eb0 con 0x55aa773eff80 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69d0f40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa7734def0 con 0x55aa773e36d0 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 0x55aa773efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59592/0 (socket says 172.21.15.60:59592) 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 -- 172.21.15.60:0/705257536 learned_addr learned my addr 172.21.15.60:0/705257536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 0x55aa773efa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52422/0 (socket says 172.21.15.60:52422) 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69cd12c640 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 -- 172.21.15.60:0/705257536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 msgr2=0x55aa773e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 -- 172.21.15.60:0/705257536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 msgr2=0x55aa773efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 0x55aa773efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.396+0000 7f69c7fff640 1 -- 172.21.15.60:0/705257536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa772d3320 con 0x55aa773eff80 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69c7fff640 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00bcb0 tx=0x7f69bc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4bf390499abe65 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69aeffd640 1 -- 172.21.15.60:0/705257536 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004030 con 0x55aa773eff80 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69aeffd640 1 -- 172.21.15.60:0/705257536 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69bc0041d0 con 0x55aa773eff80 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69aeffd640 1 -- 172.21.15.60:0/705257536 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0117b0 con 0x55aa773eff80 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69c5ffb640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 0x7f69b00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69cd12c640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 0x7f69b00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.397+0000 7f69c5ffb640 1 -- 172.21.15.60:0/3416225485 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f69b8104420 con 0x7f69c8101650 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- 172.21.15.60:0/705257536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 msgr2=0x55aa773f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00bcb0 tx=0x7f69bc0107a0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69cd12c640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 0x7f69b00b6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f69c8104fd0 tx=0x7f69c0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- 172.21.15.60:0/705257536 shutdown_connections 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773eff80 0x55aa773f2370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773e3ff0 0x55aa773efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 --2- 172.21.15.60:0/705257536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- 172.21.15.60:0/705257536 >> 172.21.15.60:0/705257536 conn(0x55aa773e1500 msgr2=0x55aa773e26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- 172.21.15.60:0/705257536 shutdown_connections 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- 172.21.15.60:0/705257536 wait complete. 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 -- start start 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.398+0000 7f69d0f40740 1 Processor -- start 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 0x55aa773ece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa773ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa771ad700 con 0x55aa773eff80 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa772ec5f0 con 0x55aa773e3ff0 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa773f2c00 con 0x55aa773e36d0 2024-07-24T21:48:35.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69d0f40740 10 client.0 _ll_register_callbacks cb 0x55aa7730c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 0x55aa773ece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 0x55aa773ece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59598/0 (socket says 172.21.15.60:59598) 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69c7fff640 1 -- 172.21.15.60:0/2595329769 learned_addr learned my addr 172.21.15.60:0/2595329769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa773ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 -- 172.21.15.60:0/2595329769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 msgr2=0x55aa773ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 -- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 msgr2=0x55aa773ece40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 0x55aa773ece40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.399+0000 7f69cd12c640 1 -- 172.21.15.60:0/2595329769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa772df680 con 0x55aa773eff80 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa773ed5d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69c0005fd0 tx=0x7f69c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b080a4dcd9afd0e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c002d060 con 0x55aa773eff80 2024-07-24T21:48:35.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f69c0018030 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc0041d0 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69c00207c0 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69d0f40740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa770aa650 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.400+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69bc0041d0 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f69c0028a20 con 0x55aa773eff80 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 20 client.16116 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa773ffa40 0x55aa77401ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 4 client.16116 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa7735c900 con 0x55aa773ffa40 2024-07-24T21:48:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa773ffa40 0x55aa77401ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa773ffa40 0x55aa77401ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7355b9ec352a8ac server_cookie=8f2733a8f0c6c83c in_seq=0 out_seq=0 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.401+0000 7f69a7fff640 10 client.16116 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69a7fff640 1 -- 172.21.15.60:0/2595329769 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa7735c900 con 0x55aa773ffa40 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69a7fff640 10 client.16116 handle_command_reply: tid=0 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 client.16116 shutdown 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 msgr2=0x55aa773ed5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa773ed5d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f69c0005fd0 tx=0x7f69c0004ab0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 shutdown_connections 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa773ffa40 0x55aa77401ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa773ed5d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e3ff0 0x55aa773ece40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69a7fff640 0 client.16116 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2595329769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e36d0 0x55aa773ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.403+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 >> 172.21.15.60:0/2595329769 conn(0x55aa773e1500 msgr2=0x55aa773e70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.404+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 shutdown_connections 2024-07-24T21:48:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.404+0000 7f69d0f40740 1 -- 172.21.15.60:0/2595329769 wait complete. 2024-07-24T21:48:35.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.404+0000 7f69d0f40740 20 client.16116 trim_cache size 0 max 16384 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 Processor -- start 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 -- start start 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81b2cf0 con 0x55aa773eff80 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81b05c0 con 0x55aa773e36d0 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69d0f40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81ae410 con 0x55aa773e3ff0 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69cd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59606/0 (socket says 172.21.15.60:59606) 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69cd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52438/0 (socket says 172.21.15.60:52438) 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69cc92b640 1 -- 172.21.15.60:0/3857730005 learned_addr learned my addr 172.21.15.60:0/3857730005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.418+0000 7f69c7fff640 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 -- 172.21.15.60:0/3857730005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 msgr2=0x55aa773e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 -- 172.21.15.60:0/3857730005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 msgr2=0x55aa77407770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 -- 172.21.15.60:0/3857730005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa773dce20 con 0x55aa773e36d0 2024-07-24T21:48:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cd12c640 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69cc92b640 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69b8102170 tx=0x7f69b810a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb596928e8e41c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69ae7fc640 1 -- 172.21.15.60:0/3857730005 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8049040 con 0x55aa773e36d0 2024-07-24T21:48:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69ae7fc640 1 -- 172.21.15.60:0/3857730005 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b80b2040 con 0x55aa773e36d0 2024-07-24T21:48:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.419+0000 7f69ae7fc640 1 -- 172.21.15.60:0/3857730005 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80d40e0 con 0x55aa773e36d0 2024-07-24T21:48:35.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- 172.21.15.60:0/3857730005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 msgr2=0x55aa773e2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69b8102170 tx=0x7f69b810a550 comp rx=0 tx=0).stop 2024-07-24T21:48:35.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- 172.21.15.60:0/3857730005 shutdown_connections 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77407770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 --2- 172.21.15.60:0/3857730005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773e2990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- 172.21.15.60:0/3857730005 >> 172.21.15.60:0/3857730005 conn(0x55aa773e1500 msgr2=0x55aa773e2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- 172.21.15.60:0/3857730005 shutdown_connections 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- 172.21.15.60:0/3857730005 wait complete. 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 -- start start 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.420+0000 7f69d0f40740 1 Processor -- start 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81aea80 con 0x55aa773eff80 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81afef0 con 0x55aa773e36d0 2024-07-24T21:48:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81af150 con 0x55aa773e3ff0 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69d0f40740 10 client.0 _ll_register_callbacks cb 0x7f69c0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cc92b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52450/0 (socket says 172.21.15.60:52450) 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 -- 172.21.15.60:0/2223488068 learned_addr learned my addr 172.21.15.60:0/2223488068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69c7fff640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 -- 172.21.15.60:0/2223488068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 msgr2=0x55aa773f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 -- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 msgr2=0x55aa773f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 -- 172.21.15.60:0/2223488068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa77469c80 con 0x55aa773eff80 2024-07-24T21:48:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69cd12c640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f69c0028e40 tx=0x7f69c0021c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a019a6376858f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.421+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0040060 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b80b2040 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f69c0005b70 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69c001da80 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 10 client.16122 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa7746add0 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b80b2040 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f69c003a030 con 0x55aa773eff80 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 10 client.16122 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 20 client.16122 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa7746b4c0 0x55aa7746d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 4 client.16122 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.422+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa773583a0 con 0x55aa7746b4c0 2024-07-24T21:48:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.423+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa7746b4c0 0x55aa7746d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.423+0000 7f69a7fff640 10 client.16122 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.423+0000 7f69cc92b640 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa7746b4c0 0x55aa7746d8e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3476f0f407e829a server_cookie=fb6d054e773e883a in_seq=0 out_seq=0 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69a7fff640 1 -- 172.21.15.60:0/2223488068 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa773583a0 con 0x55aa7746b4c0 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69a7fff640 10 client.16122 handle_command_reply: tid=0 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 client.16122 shutdown 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 msgr2=0x55aa77469740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f69c0028e40 tx=0x7f69c0021c20 comp rx=0 tx=0).stop 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 shutdown_connections 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55aa7746b4c0 0x55aa7746d8e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa773eff80 0x55aa77469740 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa773e3ff0 0x55aa773f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69a7fff640 0 client.16122 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 --2- 172.21.15.60:0/2223488068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa773e36d0 0x55aa773f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.424+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 >> 172.21.15.60:0/2223488068 conn(0x55aa773e1500 msgr2=0x55aa773ee470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 shutdown_connections 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69d0f40740 1 -- 172.21.15.60:0/2223488068 wait complete. 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69d0f40740 20 client.16122 trim_cache size 0 max 16384 2024-07-24T21:48:35.430 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 msgr2=0x7f69b00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 0x7f69b00b6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f69c8104fd0 tx=0x7f69c0009500 comp rx=0 tx=0).stop 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 msgr2=0x7f69c8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.425+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f69b8005940 tx=0x7f69b8004270 comp rx=0 tx=0).stop 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 shutdown_connections 2024-07-24T21:48:35.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f69b00b3dc0 0x7f69b00b6280 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c81079e0 0x7f69c8103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69c8107060 0x7f69c8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 --2- 172.21.15.60:0/3416225485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c8101650 0x7f69c8102f10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 >> 172.21.15.60:0/3416225485 conn(0x7f69c80fd0c0 msgr2=0x7f69c8105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 shutdown_connections 2024-07-24T21:48:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:35.426+0000 7f69ceb8f640 1 -- 172.21.15.60:0/3416225485 wait complete. 2024-07-24T21:48:35.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:35 smithi060 ceph-mon[33000]: pgmap v434: 97 pgs: 97 active+clean; 40 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2024-07-24T21:48:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:35 smithi123 ceph-mon[37143]: pgmap v434: 97 pgs: 97 active+clean; 40 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2024-07-24T21:48:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:35 smithi086 ceph-mon[37128]: pgmap v434: 97 pgs: 97 active+clean; 40 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 217 op/s 2024-07-24T21:48:36.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:48:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:37.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:37 smithi060 ceph-mon[33000]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-07-24T21:48:37.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:37 smithi123 ceph-mon[37143]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-07-24T21:48:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:37 smithi086 ceph-mon[37128]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 356 op/s 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 -- 172.21.15.60:0/1433904133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 msgr2=0x7fc308108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 --2- 172.21.15.60:0/1433904133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308108600 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009900 tx=0x7fc2fc02a3f0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 -- 172.21.15.60:0/1433904133 shutdown_connections 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 --2- 172.21.15.60:0/1433904133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc308108b40 0x7fc30810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 --2- 172.21.15.60:0/1433904133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308108600 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 --2- 172.21.15.60:0/1433904133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3081027b0 0x7fc308102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 -- 172.21.15.60:0/1433904133 >> 172.21.15.60:0/1433904133 conn(0x7fc3080fe200 msgr2=0x7fc308100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 -- 172.21.15.60:0/1433904133 shutdown_connections 2024-07-24T21:48:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.261+0000 7fc30e931640 1 -- 172.21.15.60:0/1433904133 wait complete. 2024-07-24T21:48:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 Processor -- start 2024-07-24T21:48:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 -- start start 2024-07-24T21:48:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081027b0 0x7fc308198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 0x7fc30819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc308112750 con 0x7fc3081081c0 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081125d0 con 0x7fc308108b40 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30e931640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081128d0 con 0x7fc3081027b0 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44238/0 (socket says 172.21.15.60:44238) 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc3077fe640 1 -- 172.21.15.60:0/3707260270 learned_addr learned my addr 172.21.15.60:0/3707260270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc30cece640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 0x7fc30819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.262+0000 7fc307fff640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081027b0 0x7fc308198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 -- 172.21.15.60:0/3707260270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081027b0 msgr2=0x7fc308198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081027b0 0x7fc308198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 -- 172.21.15.60:0/3707260270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 msgr2=0x7fc30819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 0x7fc30819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 -- 172.21.15.60:0/3707260270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30819ecf0 con 0x7fc3081081c0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc30cece640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 0x7fc30819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3077fe640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005940 tx=0x7fc2fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3057fa640 1 -- 172.21.15.60:0/3707260270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc03f030 con 0x7fc3081081c0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3057fa640 1 -- 172.21.15.60:0/3707260270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc003aa0 con 0x7fc3081081c0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30819ef80 con 0x7fc3081081c0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc3057fa640 1 -- 172.21.15.60:0/3707260270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc035070 con 0x7fc3081081c0 2024-07-24T21:48:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.263+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a56a0 con 0x7fc3081081c0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc3057fa640 1 -- 172.21.15.60:0/3707260270 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc2fc042030 con 0x7fc3081081c0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 Processor -- start 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 -- start start 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfc7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfda780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559b6df3e9b0 con 0x559b6dfc76d0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b6df2deb0 con 0x559b6dfc7ff0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc310ce2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b6df31ef0 con 0x559b6dfd8390 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc30cece640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfc7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc307fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc30cece640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfc7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44252/0 (socket says 172.21.15.60:44252) 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc307fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59232/0 (socket says 172.21.15.60:59232) 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc30cece640 1 -- 172.21.15.60:0/3066798867 learned_addr learned my addr 172.21.15.60:0/3066798867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.266+0000 7fc3077fe640 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfda780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc3057fa640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 0x7fc2f00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 -- 172.21.15.60:0/3066798867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 msgr2=0x559b6dfda780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfda780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 -- 172.21.15.60:0/3066798867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 msgr2=0x559b6dfc7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfc7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 -- 172.21.15.60:0/3066798867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b6deb7320 con 0x559b6dfc7ff0 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc30cece640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 0x7fc2f00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc3057fa640 1 -- 172.21.15.60:0/3707260270 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc2fc104420 con 0x7fc3081081c0 2024-07-24T21:48:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc307fff640 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009c20 tx=0x7fc2f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c0a9c0e290e4ce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc2ee7fc640 1 -- 172.21.15.60:0/3066798867 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004050 con 0x559b6dfc7ff0 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc2ee7fc640 1 -- 172.21.15.60:0/3066798867 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f80054d0 con 0x559b6dfc7ff0 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.267+0000 7fc2ee7fc640 1 -- 172.21.15.60:0/3066798867 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004ca0 con 0x559b6dfc7ff0 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- 172.21.15.60:0/3066798867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 msgr2=0x559b6dfd7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc30cece640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 0x7fc2f00b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc30000f880 tx=0x7fc300004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009c20 tx=0x7fc2f8011810 comp rx=0 tx=0).stop 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- 172.21.15.60:0/3066798867 shutdown_connections 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfda780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd7e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3066798867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfc7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- 172.21.15.60:0/3066798867 >> 172.21.15.60:0/3066798867 conn(0x559b6dfc5500 msgr2=0x559b6dfc66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- 172.21.15.60:0/3066798867 shutdown_connections 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- 172.21.15.60:0/3066798867 wait complete. 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.268+0000 7fc310ce2740 1 -- start start 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 Processor -- start 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559b6dd91700 con 0x559b6dfc76d0 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b6ded05f0 con 0x559b6dfd8390 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b6dfdb040 con 0x559b6dfc7ff0 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc310ce2740 10 client.0 _ll_register_callbacks cb 0x559b6def0960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc307fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc30cece640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc307fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44264/0 (socket says 172.21.15.60:44264) 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc30cece640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59246/0 (socket says 172.21.15.60:59246) 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc307fff640 1 -- 172.21.15.60:0/3067515778 learned_addr learned my addr 172.21.15.60:0/3067515778 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc30cece640 1 -- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 msgr2=0x559b6dfd5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.269+0000 7fc30cece640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc30cece640 1 -- 172.21.15.60:0/3067515778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 msgr2=0x559b6dfd7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc30cece640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc30cece640 1 -- 172.21.15.60:0/3067515778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6dec3680 con 0x559b6dfd8390 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc307fff640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc30cece640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc300007fd0 tx=0x7fc300004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deef673cf30e97a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc2e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300003d80 con 0x559b6dfd8390 2024-07-24T21:48:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f80054d0 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc300004050 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc300004250 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc310ce2740 10 client.25631 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b6dc8e650 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.270+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f80054d0 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc30001b930 con 0x559b6dfd8390 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 10 client.25631 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 20 client.25631 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6dfea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 4 client.25631 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b6df40900 con 0x559b6dfe7e60 2024-07-24T21:48:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc307fff640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6dfea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc307fff640 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6dfea2c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e37bd68a20b9721 server_cookie=a6e47188a4246356 in_seq=0 out_seq=0 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.271+0000 7fc2e77fe640 10 client.25631 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.272+0000 7fc2e77fe640 1 -- 172.21.15.60:0/3067515778 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b6df40900 con 0x559b6dfe7e60 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.272+0000 7fc2e77fe640 10 client.25631 handle_command_reply: tid=0 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.272+0000 7fc310ce2740 1 client.25631 shutdown 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.272+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 msgr2=0x559b6dfd59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.272+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc300007fd0 tx=0x7fc300004ab0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 shutdown_connections 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6dfea2c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfd8390 0x559b6dfd59c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc2e77fe640 0 client.25631 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 --2- 172.21.15.60:0/3067515778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc76d0 0x559b6dfd7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 >> 172.21.15.60:0/3067515778 conn(0x559b6dfc5500 msgr2=0x559b6dfcb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 shutdown_connections 2024-07-24T21:48:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 1 -- 172.21.15.60:0/3067515778 wait complete. 2024-07-24T21:48:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.273+0000 7fc310ce2740 20 client.25631 trim_cache size 0 max 16384 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 Processor -- start 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 -- start start 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfd2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfd3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081ae810 con 0x559b6dfc7ff0 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081ac0e0 con 0x559b6dfc76d0 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.287+0000 7fc310ce2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081a9f30 con 0x559b6dfd8390 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44274/0 (socket says 172.21.15.60:44274) 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 -- 172.21.15.60:0/1041101724 learned_addr learned my addr 172.21.15.60:0/1041101724 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc307fff640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfd2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc30cece640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfd3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 -- 172.21.15.60:0/1041101724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 msgr2=0x559b6dfd3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfd3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 -- 172.21.15.60:0/1041101724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 msgr2=0x559b6dfd2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfd2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 -- 172.21.15.60:0/1041101724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b6dfc5290 con 0x559b6dfc7ff0 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc307fff640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfd2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc3077fe640 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cade0 tx=0x7fc2fc03b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acc789700d0cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc2edffb640 1 -- 172.21.15.60:0/1041101724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc049070 con 0x559b6dfc7ff0 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.288+0000 7fc2edffb640 1 -- 172.21.15.60:0/1041101724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc0b2040 con 0x559b6dfc7ff0 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc2edffb640 1 -- 172.21.15.60:0/1041101724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0cb0e0 con 0x559b6dfc7ff0 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 -- 172.21.15.60:0/1041101724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 msgr2=0x559b6dfd3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cade0 tx=0x7fc2fc03b520 comp rx=0 tx=0).stop 2024-07-24T21:48:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 -- 172.21.15.60:0/1041101724 shutdown_connections 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfd8390 0x559b6dfd3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfc7ff0 0x559b6dfd3ab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 --2- 172.21.15.60:0/1041101724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfd2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 -- 172.21.15.60:0/1041101724 >> 172.21.15.60:0/1041101724 conn(0x559b6dfc5500 msgr2=0x559b6dfd9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 -- 172.21.15.60:0/1041101724 shutdown_connections 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.289+0000 7fc310ce2740 1 -- 172.21.15.60:0/1041101724 wait complete. 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 -- start start 2024-07-24T21:48:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 Processor -- start 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfe04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 0x559b6dfef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081aa5a0 con 0x559b6dfd8390 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081aba10 con 0x559b6dfc76d0 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081aac70 con 0x559b6dfc7ff0 2024-07-24T21:48:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc310ce2740 10 client.0 _ll_register_callbacks cb 0x7fc30000ff20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc30cece640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 0x559b6dfef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42414/0 (socket says 172.21.15.60:42414) 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc3077fe640 1 -- 172.21.15.60:0/2720313591 learned_addr learned my addr 172.21.15.60:0/2720313591 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.290+0000 7fc307fff640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfe04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 -- 172.21.15.60:0/2720313591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 msgr2=0x559b6dfe04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfe04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 -- 172.21.15.60:0/2720313591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 msgr2=0x559b6dfef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 0x559b6dfef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 -- 172.21.15.60:0/2720313591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6dfef9e0 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc30cece640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 0x559b6dfef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc3077fe640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cb070 tx=0x7fc2fc10a0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5eabd7bb5053e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc2e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc049040 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2fc0b2040 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc2fc0d3c50 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc0ca850 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc310ce2740 10 client.25642 fetch_fsmap learned FSMap version 27 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.291+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b6e052710 con 0x559b6dfc7ff0 2024-07-24T21:48:38.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc0b2040 con 0x559b6dfc7ff0 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2fc0ca340 con 0x559b6dfc7ff0 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 10 client.25642 fetch_fsmap finished waiting for FSMap version 27 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 20 client.25642 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6e054e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 4 client.25642 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b6df3c3a0 con 0x559b6dfe7e60 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc307fff640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6e054e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc307fff640 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6e054e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7d0cd6c5bb3c2a server_cookie=cd78f7344b7ab5ee in_seq=0 out_seq=0 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.292+0000 7fc2e77fe640 10 client.25642 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc2e77fe640 1 -- 172.21.15.60:0/2720313591 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b6df3c3a0 con 0x559b6dfe7e60 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc2e77fe640 10 client.25642 handle_command_reply: tid=0 2024-07-24T21:48:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 client.25642 shutdown 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 msgr2=0x559b6dfe0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cb070 tx=0x7fc2fc10a0c0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 shutdown_connections 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559b6dfe7e60 0x559b6e054e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b6dfd8390 0x559b6dfef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b6dfc7ff0 0x559b6dfe0a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc2e77fe640 0 client.25642 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 --2- 172.21.15.60:0/2720313591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b6dfc76d0 0x559b6dfe04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 >> 172.21.15.60:0/2720313591 conn(0x559b6dfc5500 msgr2=0x559b6dfd36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 shutdown_connections 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 1 -- 172.21.15.60:0/2720313591 wait complete. 2024-07-24T21:48:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.293+0000 7fc310ce2740 20 client.25642 trim_cache size 0 max 16384 2024-07-24T21:48:38.302 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:48:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 msgr2=0x7fc2f00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 0x7fc2f00b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc30000f880 tx=0x7fc300004a40 comp rx=0 tx=0).stop 2024-07-24T21:48:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 msgr2=0x7fc308199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005940 tx=0x7fc2fc004270 comp rx=0 tx=0).stop 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 shutdown_connections 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2f00b3dc0 0x7fc2f00b6280 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc308108b40 0x7fc30819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3081081c0 0x7fc308199180 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 --2- 172.21.15.60:0/3707260270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081027b0 0x7fc308198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 >> 172.21.15.60:0/3707260270 conn(0x7fc3080fe200 msgr2=0x7fc3081005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 shutdown_connections 2024-07-24T21:48:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:38.294+0000 7fc30e931640 1 -- 172.21.15.60:0/3707260270 wait complete. 2024-07-24T21:48:39.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:39 smithi060 ceph-mon[33000]: pgmap v436: 97 pgs: 97 active+clean; 47 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-07-24T21:48:39.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:39 smithi123 ceph-mon[37143]: pgmap v436: 97 pgs: 97 active+clean; 47 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-07-24T21:48:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:39 smithi086 ceph-mon[37128]: pgmap v436: 97 pgs: 97 active+clean; 47 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-07-24T21:48:40.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:48:40.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:48:40.428 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:40.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:40 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:40.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:40 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:40.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:40 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:41 smithi123 ceph-mon[37143]: pgmap v437: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 391 op/s 2024-07-24T21:48:41.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:41 smithi086 ceph-mon[37128]: pgmap v437: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 391 op/s 2024-07-24T21:48:41.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:41 smithi060 ceph-mon[33000]: pgmap v437: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 391 op/s 2024-07-24T21:48:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.447+0000 7f1f12a98640 1 -- 172.21.15.60:0/1473513986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c10a460 msgr2=0x7f1f0c1172c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.447+0000 7f1f12a98640 1 --2- 172.21.15.60:0/1473513986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c10a460 0x7f1f0c1172c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009900 tx=0x7f1f0802a440 comp rx=0 tx=0).stop 2024-07-24T21:48:42.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.453+0000 7f1f12a98640 1 -- 172.21.15.60:0/1473513986 shutdown_connections 2024-07-24T21:48:42.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.453+0000 7f1f12a98640 1 --2- 172.21.15.60:0/1473513986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 0x7f1f0c119bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.453+0000 7f1f12a98640 1 --2- 172.21.15.60:0/1473513986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c10a460 0x7f1f0c1172c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.453+0000 7f1f12a98640 1 --2- 172.21.15.60:0/1473513986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0c109b40 0x7f1f0c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.453+0000 7f1f12a98640 1 -- 172.21.15.60:0/1473513986 >> 172.21.15.60:0/1473513986 conn(0x7f1f0c06c4f0 msgr2=0x7f1f0c06c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.458+0000 7f1f12a98640 1 -- 172.21.15.60:0/1473513986 shutdown_connections 2024-07-24T21:48:42.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.458+0000 7f1f12a98640 1 -- 172.21.15.60:0/1473513986 wait complete. 2024-07-24T21:48:42.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 Processor -- start 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 -- start start 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0c10a460 0x7f1f0c1a8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 0x7f1f0c1ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c113730 con 0x7f1f0c109b40 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c1135b0 con 0x7f1f0c10a460 2024-07-24T21:48:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.459+0000 7f1f12a98640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c1138b0 con 0x7f1f0c117800 2024-07-24T21:48:42.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44302/0 (socket says 172.21.15.60:44302) 2024-07-24T21:48:42.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 -- 172.21.15.60:0/3996854537 learned_addr learned my addr 172.21.15.60:0/3996854537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f12297640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 0x7f1f0c1ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 -- 172.21.15.60:0/3996854537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 msgr2=0x7f1f0c1ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 0x7f1f0c1ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 -- 172.21.15.60:0/3996854537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0c10a460 msgr2=0x7f1f0c1a8370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.461+0000 7f1f11a96640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0c10a460 0x7f1f0c1a8370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.462+0000 7f1f11a96640 1 -- 172.21.15.60:0/3996854537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c1ad8a0 con 0x7f1f0c109b40 2024-07-24T21:48:42.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.465+0000 7f1f11a96640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e9f0 tx=0x7f1f0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.466+0000 7f1efeffd640 1 -- 172.21.15.60:0/3996854537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000cd80 con 0x7f1f0c109b40 2024-07-24T21:48:42.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.466+0000 7f1efeffd640 1 -- 172.21.15.60:0/3996854537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f000108e0 con 0x7f1f0c109b40 2024-07-24T21:48:42.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.466+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c1adb90 con 0x7f1f0c109b40 2024-07-24T21:48:42.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.474+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c1b4250 con 0x7f1f0c109b40 2024-07-24T21:48:42.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.474+0000 7f1efeffd640 1 -- 172.21.15.60:0/3996854537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00018880 con 0x7f1f0c109b40 2024-07-24T21:48:42.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.479+0000 7f1efeffd640 1 -- 172.21.15.60:0/3996854537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1f00018ae0 con 0x7f1f0c109b40 2024-07-24T21:48:42.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.479+0000 7f1f160ab740 1 Processor -- start 2024-07-24T21:48:42.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.480+0000 7f1f160ab740 1 -- start start 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.480+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f12297640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f12297640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44312/0 (socket says 172.21.15.60:44312) 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf2fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f160ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5627cf2649f0 con 0x5627cf2ed100 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f160ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627cf253ef0 con 0x5627cf2eda20 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1f160ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627cf257f30 con 0x5627cf2f99b0 2024-07-24T21:48:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1efeffd640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 0x7f1ef80b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.481+0000 7f1efeffd640 1 -- 172.21.15.60:0/3996854537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1f000e52e0 con 0x7f1f0c109b40 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.482+0000 7f1f12297640 1 -- 172.21.15.60:0/1953240578 learned_addr learned my addr 172.21.15.60:0/1953240578 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.482+0000 7f1f12297640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 0x7f1ef80b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.482+0000 7f1f11295640 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf2fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 -- 172.21.15.60:0/1953240578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 msgr2=0x5627cf2fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf2fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 -- 172.21.15.60:0/1953240578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 msgr2=0x5627cf2f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 -- 172.21.15.60:0/1953240578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627cefb4930 con 0x5627cf2ed100 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.483+0000 7f1f12297640 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1f04013850 tx=0x7f1f04013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfb4cdccefbcad6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.484+0000 7f1f12297640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 0x7f1ef80b64d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1f04019000 tx=0x7f1f0401b000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.484+0000 7f1ef7fff640 1 -- 172.21.15.60:0/1953240578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04021020 con 0x5627cf2ed100 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.484+0000 7f1ef7fff640 1 -- 172.21.15.60:0/1953240578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f04004b60 con 0x5627cf2ed100 2024-07-24T21:48:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.484+0000 7f1ef7fff640 1 -- 172.21.15.60:0/1953240578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003c10 con 0x5627cf2ed100 2024-07-24T21:48:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 -- 172.21.15.60:0/1953240578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 msgr2=0x5627cf2ed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1f04013850 tx=0x7f1f04013d10 comp rx=0 tx=0).stop 2024-07-24T21:48:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 -- 172.21.15.60:0/1953240578 shutdown_connections 2024-07-24T21:48:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf2fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 --2- 172.21.15.60:0/1953240578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2ed4e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.490+0000 7f1f160ab740 1 -- 172.21.15.60:0/1953240578 >> 172.21.15.60:0/1953240578 conn(0x5627cf2eaf30 msgr2=0x5627cf2ec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.492+0000 7f1f160ab740 1 -- 172.21.15.60:0/1953240578 shutdown_connections 2024-07-24T21:48:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.492+0000 7f1f160ab740 1 -- 172.21.15.60:0/1953240578 wait complete. 2024-07-24T21:48:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.492+0000 7f1f160ab740 1 -- start start 2024-07-24T21:48:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.493+0000 7f1f160ab740 1 Processor -- start 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.494+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.494+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.494+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.494+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59310/0 (socket says 172.21.15.60:59310) 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42450/0 (socket says 172.21.15.60:42450) 2024-07-24T21:48:42.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f11295640 1 -- 172.21.15.60:0/309467269 learned_addr learned my addr 172.21.15.60:0/309467269 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf2f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5627cf0b7890 con 0x5627cf2f99b0 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f12297640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf2f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.495+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627cf1f6390 con 0x5627cf2eda20 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5627cf2fc5d0 con 0x5627cf2ed100 2024-07-24T21:48:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f160ab740 10 client.0 _ll_register_callbacks cb 0x5627cf200500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f11295640 1 -- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 msgr2=0x5627cf2f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f11295640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f11295640 1 -- 172.21.15.60:0/309467269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 msgr2=0x5627cf2f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f11295640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf2f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.496+0000 7f1f11295640 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627cf1f0d60 con 0x5627cf2eda20 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1f11295640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802a9b0 tx=0x7f1f08009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8305c1a7d2520e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1ef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08041070 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f04004b60 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1f08031950 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.497+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f08031b70 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 10 client.25643 fetch_fsmap learned FSMap version 28 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627cf3085a0 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f04004b60 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1f0803d420 con 0x5627cf2eda20 2024-07-24T21:48:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 10 client.25643 fetch_fsmap finished waiting for FSMap version 28 2024-07-24T21:48:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 20 client.25643 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.498+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf30be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.499+0000 7f1f160ab740 4 client.25643 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.499+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627cf266940 con 0x5627cf309a70 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.499+0000 7f1f11a96640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf30be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.500+0000 7f1ef57fa640 10 client.25643 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.500+0000 7f1f11a96640 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf30be80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3c12e4d10c6aad server_cookie=55a137e9c0384613 in_seq=0 out_seq=0 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1ef57fa640 1 -- 172.21.15.60:0/309467269 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627cf266940 con 0x5627cf309a70 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1ef57fa640 10 client.25643 handle_command_reply: tid=0 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 client.25643 shutdown 2024-07-24T21:48:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 msgr2=0x5627cf2f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802a9b0 tx=0x7f1f08009610 comp rx=0 tx=0).stop 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 shutdown_connections 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf30be80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1ef57fa640 0 client.25643 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf2f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2f6ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 --2- 172.21.15.60:0/309467269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 >> 172.21.15.60:0/309467269 conn(0x5627cf2eaf30 msgr2=0x5627cf2f0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.501+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 shutdown_connections 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.502+0000 7f1f160ab740 1 -- 172.21.15.60:0/309467269 wait complete. 2024-07-24T21:48:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.502+0000 7f1f160ab740 20 client.25643 trim_cache size 0 max 16384 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.521+0000 7f1f160ab740 1 Processor -- start 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.521+0000 7f1f160ab740 1 -- start start 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.521+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.521+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf311c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f160ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0011120 con 0x5627cf2f99b0 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f160ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef000e9c0 con 0x5627cf2eda20 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f160ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef000c810 con 0x5627cf2ed100 2024-07-24T21:48:42.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f12297640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf311c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59320/0 (socket says 172.21.15.60:59320) 2024-07-24T21:48:42.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 -- 172.21.15.60:0/4129798990 learned_addr learned my addr 172.21.15.60:0/4129798990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:42.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11a96640 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 -- 172.21.15.60:0/4129798990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 msgr2=0x5627cf2f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 -- 172.21.15.60:0/4129798990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 msgr2=0x5627cf311c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf311c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 -- 172.21.15.60:0/4129798990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627cf2ed850 con 0x5627cf2eda20 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.522+0000 7f1f11295640 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f08032fe0 tx=0x7f1f0802a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2947830986f5a7b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1ef77fe640 1 -- 172.21.15.60:0/4129798990 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0805c070 con 0x5627cf2eda20 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1ef77fe640 1 -- 172.21.15.60:0/4129798990 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f08031950 con 0x5627cf2eda20 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1ef77fe640 1 -- 172.21.15.60:0/4129798990 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08053040 con 0x5627cf2eda20 2024-07-24T21:48:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 -- 172.21.15.60:0/4129798990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 msgr2=0x5627cf311750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f08032fe0 tx=0x7f1f0802a440 comp rx=0 tx=0).stop 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 -- 172.21.15.60:0/4129798990 shutdown_connections 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2f99b0 0x5627cf311c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf311750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 --2- 172.21.15.60:0/4129798990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2ed100 0x5627cf2f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.523+0000 7f1f160ab740 1 -- 172.21.15.60:0/4129798990 >> 172.21.15.60:0/4129798990 conn(0x5627cf2eaf30 msgr2=0x5627cf2f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.524+0000 7f1f160ab740 1 -- 172.21.15.60:0/4129798990 shutdown_connections 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.524+0000 7f1f160ab740 1 -- 172.21.15.60:0/4129798990 wait complete. 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.524+0000 7f1f160ab740 1 -- start start 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.524+0000 7f1f160ab740 1 Processor -- start 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf373720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f11a96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44344/0 (socket says 172.21.15.60:44344) 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f11295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2fdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59330/0 (socket says 172.21.15.60:59330) 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f11a96640 1 -- 172.21.15.60:0/751592756 learned_addr learned my addr 172.21.15.60:0/751592756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef000ce80 con 0x5627cf2ed100 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef000e2f0 con 0x5627cf2eda20 2024-07-24T21:48:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef000d550 con 0x5627cf2f99b0 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f12297640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf373720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f12297640 1 -- 172.21.15.60:0/751592756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 msgr2=0x5627cf2fdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f12297640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2fdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.525+0000 7f1f12297640 1 -- 172.21.15.60:0/751592756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 msgr2=0x5627cf2fdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f12297640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f160ab740 10 client.0 _ll_register_callbacks cb 0x7f1f080036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f12297640 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627cf373c60 con 0x5627cf2f99b0 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f11a96640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f08031950 con 0x5627cf2f99b0 2024-07-24T21:48:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f12297640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf373720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f04010740 tx=0x7f1f04004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c546098a5d23babb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1ef57fa640 10 client.25657 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1ef57fa640 1 -- 172.21.15.60:0/751592756 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04021050 con 0x5627cf2f99b0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1ef57fa640 1 -- 172.21.15.60:0/751592756 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1f0400abf0 con 0x5627cf2f99b0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1ef57fa640 1 -- 172.21.15.60:0/751592756 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f04004e60 con 0x5627cf2f99b0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f160ab740 10 client.25657 fetch_fsmap learned FSMap version 28 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.526+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5627cf374cb0 con 0x5627cf2f99b0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1ef57fa640 1 -- 172.21.15.60:0/751592756 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1f0401d480 con 0x5627cf2f99b0 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 10 client.25657 fetch_fsmap finished waiting for FSMap version 28 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 20 client.25657 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf377230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 4 client.25657 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:48:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5627cf2f1720 con 0x5627cf309a70 2024-07-24T21:48:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f11a96640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf377230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1ef57fa640 10 client.25657 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.527+0000 7f1f11a96640 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf377230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af09536a5e80d40 server_cookie=e1a2c11601ddcbf1 in_seq=0 out_seq=0 2024-07-24T21:48:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1ef57fa640 1 -- 172.21.15.60:0/751592756 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5627cf2f1720 con 0x5627cf309a70 2024-07-24T21:48:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1ef57fa640 10 client.25657 handle_command_reply: tid=0 2024-07-24T21:48:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 client.25657 shutdown 2024-07-24T21:48:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 msgr2=0x5627cf373720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf373720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f04010740 tx=0x7f1f04004d70 comp rx=0 tx=0).stop 2024-07-24T21:48:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 shutdown_connections 2024-07-24T21:48:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5627cf309a70 0x5627cf377230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5627cf2f99b0 0x5627cf373720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cf2eda20 0x5627cf2fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1ef57fa640 0 client.25657 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 --2- 172.21.15.60:0/751592756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5627cf2ed100 0x5627cf2fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.528+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 >> 172.21.15.60:0/751592756 conn(0x5627cf2eaf30 msgr2=0x5627cf2e97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 shutdown_connections 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f160ab740 1 -- 172.21.15.60:0/751592756 wait complete. 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f160ab740 20 client.25657 trim_cache size 0 max 16384 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:48:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "39b21460-e888-4b72-9c97-0fba157cbde0", 2024-07-24T21:48:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:48:42.539 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 msgr2=0x7f1ef80b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 0x7f1ef80b64d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1f04019000 tx=0x7f1f0401b000 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.529+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 msgr2=0x7f1f0c1a7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e9f0 tx=0x7f1f0000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 shutdown_connections 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ef80b4010 0x7f1ef80b64d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0c117800 0x7f1f0c1ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0c10a460 0x7f1f0c1a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 --2- 172.21.15.60:0/3996854537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c109b40 0x7f1f0c1a7e30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 >> 172.21.15.60:0/3996854537 conn(0x7f1f0c06c4f0 msgr2=0x7f1f0c117c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.530+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 shutdown_connections 2024-07-24T21:48:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:42.531+0000 7f1f12a98640 1 -- 172.21.15.60:0/3996854537 wait complete. 2024-07-24T21:48:43.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: pgmap v438: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 394 op/s 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:43.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:43.749 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: pgmap v438: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 394 op/s 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: pgmap v438: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 394 op/s 2024-07-24T21:48:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:48:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:44 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:44.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:44 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:44 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:44.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:44 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.707 INFO:tasks.workunit.client.0.smithi060.stdout: 29 483457 639 284611 607 157727 1342 2024-07-24T21:48:44.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:44 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:44.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:44 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:44 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:44.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:44 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:44 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:44 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:44 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:44 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 -- 172.21.15.60:0/537121731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d8107910 msgr2=0x7f65d810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 --2- 172.21.15.60:0/537121731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d8107910 0x7f65d810e110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009ca0 tx=0x7f65d002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 -- 172.21.15.60:0/537121731 shutdown_connections 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 --2- 172.21.15.60:0/537121731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d8107910 0x7f65d810e110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 --2- 172.21.15.60:0/537121731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8106f90 0x7f65d81073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 --2- 172.21.15.60:0/537121731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d81015e0 0x7f65d81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 -- 172.21.15.60:0/537121731 >> 172.21.15.60:0/537121731 conn(0x7f65d80782e0 msgr2=0x7f65d80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 -- 172.21.15.60:0/537121731 shutdown_connections 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.951+0000 7f65ddee9640 1 -- 172.21.15.60:0/537121731 wait complete. 2024-07-24T21:48:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 Processor -- start 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 -- start start 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 0x7f65d81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 0x7f65d8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107910 0x7f65d819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111560 con 0x7f65d8106f90 2024-07-24T21:48:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81113e0 con 0x7f65d81015e0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65ddee9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81116e0 con 0x7f65d8107910 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 0x7f65d8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 0x7f65d81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 0x7f65d8199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44370/0 (socket says 172.21.15.60:44370) 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65d6ffd640 1 -- 172.21.15.60:0/164447929 learned_addr learned my addr 172.21.15.60:0/164447929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.952+0000 7f65d7fff640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107910 0x7f65d819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 -- 172.21.15.60:0/164447929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107910 msgr2=0x7f65d819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107910 0x7f65d819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 -- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 msgr2=0x7f65d8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 0x7f65d8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 -- 172.21.15.60:0/164447929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a4dd0 con 0x7f65d81015e0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d77fe640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 0x7f65d81999d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c800e9f0 tx=0x7f65c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d4ff9640 1 -- 172.21.15.60:0/164447929 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800ccc0 con 0x7f65d81015e0 2024-07-24T21:48:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d4ff9640 1 -- 172.21.15.60:0/164447929 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c80108e0 con 0x7f65d81015e0 2024-07-24T21:48:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a50c0 con 0x7f65d81015e0 2024-07-24T21:48:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65d4ff9640 1 -- 172.21.15.60:0/164447929 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8018880 con 0x7f65d81015e0 2024-07-24T21:48:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.953+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a5610 con 0x7f65d81015e0 2024-07-24T21:48:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 Processor -- start 2024-07-24T21:48:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 -- start start 2024-07-24T21:48:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b191438280 con 0x55b1914c31e0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b191427780 con 0x55b1914c28c0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65e0c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b19142b7c0 con 0x55b1914cf170 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d4ff9640 1 -- 172.21.15.60:0/164447929 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f65c8010430 con 0x7f65d81015e0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59352/0 (socket says 172.21.15.60:59352) 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 -- 172.21.15.60:0/877117425 learned_addr learned my addr 172.21.15.60:0/877117425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d77fe640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 -- 172.21.15.60:0/877117425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 msgr2=0x55b1914d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 -- 172.21.15.60:0/877117425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 msgr2=0x55b1914cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d7fff640 1 -- 172.21.15.60:0/877117425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19118b790 con 0x55b1914c28c0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.956+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65d7fff640 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65d0005cc0 tx=0x7f65d0009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5644d645a98d2153 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65bdffb640 1 -- 172.21.15.60:0/877117425 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0031cf0 con 0x55b1914c28c0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65bdffb640 1 -- 172.21.15.60:0/877117425 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d0036040 con 0x55b1914c28c0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65bdffb640 1 -- 172.21.15.60:0/877117425 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d00042a0 con 0x55b1914c28c0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65d4ff9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 0x7f65c00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65d77fe640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 0x7f65c00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.957+0000 7f65d4ff9640 1 -- 172.21.15.60:0/164447929 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f65c80e4430 con 0x7f65d81015e0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- 172.21.15.60:0/877117425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 msgr2=0x55b1914c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65d77fe640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 0x7f65c00b6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65c80e1c30 tx=0x7f65c8009700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65d0005cc0 tx=0x7f65d0009360 comp rx=0 tx=0).stop 2024-07-24T21:48:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- 172.21.15.60:0/877117425 shutdown_connections 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 --2- 172.21.15.60:0/877117425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914c2ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- 172.21.15.60:0/877117425 >> 172.21.15.60:0/877117425 conn(0x55b1914c06f0 msgr2=0x55b1914c1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- 172.21.15.60:0/877117425 shutdown_connections 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- 172.21.15.60:0/877117425 wait complete. 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.958+0000 7f65e0c43740 1 -- start start 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 Processor -- start 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b191361ab0 con 0x55b1914c31e0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1913af9e0 con 0x55b1914c28c0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b1914d1df0 con 0x55b1914cf170 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65e0c43740 10 client.0 _ll_register_callbacks cb 0x55b191137a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59366/0 (socket says 172.21.15.60:59366) 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65d7fff640 1 -- 172.21.15.60:0/4217549816 learned_addr learned my addr 172.21.15.60:0/4217549816 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.959+0000 7f65d7fff640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 -- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 msgr2=0x55b1914cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 -- 172.21.15.60:0/4217549816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 msgr2=0x55b1914cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 -- 172.21.15.60:0/4217549816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b191204620 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65d7fff640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65d0003cd0 tx=0x7f65d0005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccdc1471d63e18e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0031cf0 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f65d0031450 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1913d6ce0 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65d0031650 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65e0c43740 10 client.25661 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1914dd5a0 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.960+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b1913d6ce0 con 0x55b1914c28c0 2024-07-24T21:48:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65d002f660 con 0x55b1914c28c0 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 10 client.25661 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 20 client.25661 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b1914de8f0 0x55b1914e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 4 client.25661 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b19143a1d0 con 0x55b1914de8f0 2024-07-24T21:48:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b1914de8f0 0x55b1914e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65b6ffd640 10 client.25661 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.961+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b1914de8f0 0x55b1914e0ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea251d26acdd334 server_cookie=21da828599f4b11d in_seq=0 out_seq=0 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65b6ffd640 1 -- 172.21.15.60:0/4217549816 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b19143a1d0 con 0x55b1914de8f0 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65b6ffd640 10 client.25661 handle_command_reply: tid=0 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 client.25661 shutdown 2024-07-24T21:48:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 msgr2=0x55b1914ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65d0003cd0 tx=0x7f65d0005f70 comp rx=0 tx=0).stop 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 shutdown_connections 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b1914de8f0 0x55b1914e0ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914cf170 0x55b1914cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 --2- 172.21.15.60:0/4217549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914ce8e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65b6ffd640 0 client.25661 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.962+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 >> 172.21.15.60:0/4217549816 conn(0x55b1914c06f0 msgr2=0x55b1914c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.963+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 shutdown_connections 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.963+0000 7f65e0c43740 1 -- 172.21.15.60:0/4217549816 wait complete. 2024-07-24T21:48:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.963+0000 7f65e0c43740 20 client.25661 trim_cache size 0 max 16384 2024-07-24T21:48:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 Processor -- start 2024-07-24T21:48:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 -- start start 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c31e0 0x55b1914e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b0980 con 0x55b1914cf170 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81ae1f0 con 0x55b1914c28c0 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65e0c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81ac040 con 0x55b1914c31e0 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44388/0 (socket says 172.21.15.60:44388) 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 -- 172.21.15.60:0/3423618973 learned_addr learned my addr 172.21.15.60:0/3423618973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c31e0 0x55b1914e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 -- 172.21.15.60:0/3423618973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c31e0 msgr2=0x55b1914e62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c31e0 0x55b1914e62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 -- 172.21.15.60:0/3423618973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 msgr2=0x55b1914daf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.980+0000 7f65d7fff640 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914daf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.981+0000 7f65d7fff640 1 -- 172.21.15.60:0/3423618973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1914c3010 con 0x55b1914cf170 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.981+0000 7f65d7fff640 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f65d002fc20 tx=0x7f65d0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a8634bdb7d9f3e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.981+0000 7f65bd7fa640 1 -- 172.21.15.60:0/3423618973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0061070 con 0x55b1914cf170 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.981+0000 7f65bd7fa640 1 -- 172.21.15.60:0/3423618973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d0062040 con 0x55b1914cf170 2024-07-24T21:48:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.981+0000 7f65bd7fa640 1 -- 172.21.15.60:0/3423618973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0004250 con 0x55b1914cf170 2024-07-24T21:48:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- 172.21.15.60:0/3423618973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 msgr2=0x55b1914e6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f65d002fc20 tx=0x7f65d0009360 comp rx=0 tx=0).stop 2024-07-24T21:48:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- 172.21.15.60:0/3423618973 shutdown_connections 2024-07-24T21:48:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914cf170 0x55b1914e6820 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c31e0 0x55b1914e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 --2- 172.21.15.60:0/3423618973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914c28c0 0x55b1914daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- 172.21.15.60:0/3423618973 >> 172.21.15.60:0/3423618973 conn(0x55b1914c06f0 msgr2=0x55b1914c7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- 172.21.15.60:0/3423618973 shutdown_connections 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- 172.21.15.60:0/3423618973 wait complete. 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.982+0000 7f65e0c43740 1 -- start start 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 Processor -- start 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c28c0 0x55b1914d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914cf170 0x55b1914c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81ac6b0 con 0x55b1914c31e0 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81adb20 con 0x55b1914cf170 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81acd80 con 0x55b1914c28c0 2024-07-24T21:48:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65e0c43740 10 client.0 _ll_register_callbacks cb 0x7f65c80e7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44396/0 (socket says 172.21.15.60:44396) 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65d6ffd640 1 -- 172.21.15.60:0/722968313 learned_addr learned my addr 172.21.15.60:0/722968313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.983+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914cf170 0x55b1914c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d7fff640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c28c0 0x55b1914d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 -- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c28c0 msgr2=0x55b1914d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c28c0 0x55b1914d7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 -- 172.21.15.60:0/722968313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914cf170 msgr2=0x55b1914c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914cf170 0x55b1914c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 -- 172.21.15.60:0/722968313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1914d7d00 con 0x55b1914c31e0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65d6ffd640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55b1914e1280 tx=0x7f65cc0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd6b6144b3e322b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc005250 con 0x55b1914c31e0 2024-07-24T21:48:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f65cc0053f0 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.984+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d0062040 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65cc02d050 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 10 client.16176 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b191548930 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d0062040 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65cc01d7a0 con 0x55b1914c31e0 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 10 client.16176 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 20 client.16176 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b191549140 0x55b19154b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 4 client.16176 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.985+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b191435c70 con 0x55b191549140 2024-07-24T21:48:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.986+0000 7f65d7fff640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b191549140 0x55b19154b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.986+0000 7f65d7fff640 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b191549140 0x55b19154b560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc9b208b26a2cf0 server_cookie=d9a4f7fe549a5ace in_seq=0 out_seq=0 2024-07-24T21:48:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:44.986+0000 7f65b6ffd640 10 client.16176 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65b6ffd640 1 -- 172.21.15.60:0/722968313 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b191435c70 con 0x55b191549140 2024-07-24T21:48:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65b6ffd640 10 client.16176 handle_command_reply: tid=0 2024-07-24T21:48:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 client.16176 shutdown 2024-07-24T21:48:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 msgr2=0x55b1914d75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55b1914e1280 tx=0x7f65cc0138f0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 shutdown_connections 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b191549140 0x55b19154b560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65b6ffd640 0 client.16176 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1914cf170 0x55b1914c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.007+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b1914c31e0 0x55b1914d75c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65e0c43740 1 --2- 172.21.15.60:0/722968313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b1914c28c0 0x55b1914d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 >> 172.21.15.60:0/722968313 conn(0x55b1914c06f0 msgr2=0x55b1914c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 shutdown_connections 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65e0c43740 1 -- 172.21.15.60:0/722968313 wait complete. 2024-07-24T21:48:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65e0c43740 20 client.16176 trim_cache size 0 max 16384 2024-07-24T21:48:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:48:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:48:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:48:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 msgr2=0x7f65c00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 0x7f65c00b6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65c80e1c30 tx=0x7f65c8009700 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 msgr2=0x7f65d81999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.008+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 0x7f65d81999d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c800e9f0 tx=0x7f65c800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 shutdown_connections 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65c00b3d40 0x7f65c00b6200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d8107910 0x7f65d819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d8106f90 0x7f65d8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 --2- 172.21.15.60:0/164447929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65d81015e0 0x7f65d81999d0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c800e9f0 tx=0x7f65c800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 >> 172.21.15.60:0/164447929 conn(0x7f65d80782e0 msgr2=0x7f65d8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 shutdown_connections 2024-07-24T21:48:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:45.009+0000 7f65ddee9640 1 -- 172.21.15.60:0/164447929 wait complete. 2024-07-24T21:48:45.496 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:45 smithi060 ceph-mon[33000]: pgmap v439: 97 pgs: 97 active+clean; 61 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-07-24T21:48:45.496 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:45 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.494+0000 7f79ed88a640 1 -- 172.21.15.60:0/4200219907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 msgr2=0x7f79e8103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.494+0000 7f79ed88a640 1 --2- 172.21.15.60:0/4200219907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 0x7f79e8103a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79d800fb10 tx=0x7f79d802e570 comp rx=0 tx=0).stop 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.495+0000 7f79ed88a640 1 -- 172.21.15.60:0/4200219907 shutdown_connections 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.495+0000 7f79ed88a640 1 --2- 172.21.15.60:0/4200219907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 0x7f79e810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.495+0000 7f79ed88a640 1 --2- 172.21.15.60:0/4200219907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.495+0000 7f79ed88a640 1 --2- 172.21.15.60:0/4200219907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 0x7f79e8103a40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.495+0000 7f79ed88a640 1 -- 172.21.15.60:0/4200219907 >> 172.21.15.60:0/4200219907 conn(0x7f79e80fd0c0 msgr2=0x7f79e80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.499+0000 7f79ed88a640 1 -- 172.21.15.60:0/4200219907 shutdown_connections 2024-07-24T21:48:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.504+0000 7f79ed88a640 1 -- 172.21.15.60:0/4200219907 wait complete. 2024-07-24T21:48:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.505+0000 7f79ed88a640 1 Processor -- start 2024-07-24T21:48:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.505+0000 7f79ed88a640 1 -- start start 2024-07-24T21:48:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79ed88a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 0x7f79e819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79ed88a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79ed88a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 0x7f79e81a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79ed88a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8110570 con 0x7f79e8103f80 2024-07-24T21:48:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44402/0 (socket says 172.21.15.60:44402) 2024-07-24T21:48:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79e67fc640 1 -- 172.21.15.60:0/1829466087 learned_addr learned my addr 172.21.15.60:0/1829466087 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.506+0000 7f79e77fe640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 0x7f79e81a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.507+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81103f0 con 0x7f79e8103660 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.507+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81106f0 con 0x7f79e810cac0 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 -- 172.21.15.60:0/1829466087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 msgr2=0x7f79e81a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 0x7f79e81a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 -- 172.21.15.60:0/1829466087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 msgr2=0x7f79e819f250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 0x7f79e819f250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 -- 172.21.15.60:0/1829466087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e81a5300 con 0x7f79e8103f80 2024-07-24T21:48:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.508+0000 7f79e67fc640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e9c0 tx=0x7f79dc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.513+0000 7f79ec888640 1 -- 172.21.15.60:0/1829466087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00cd80 con 0x7f79e8103f80 2024-07-24T21:48:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.513+0000 7f79ec888640 1 -- 172.21.15.60:0/1829466087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc0096c0 con 0x7f79e8103f80 2024-07-24T21:48:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.513+0000 7f79ec888640 1 -- 172.21.15.60:0/1829466087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc018840 con 0x7f79e8103f80 2024-07-24T21:48:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.513+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e81a55f0 con 0x7f79e8103f80 2024-07-24T21:48:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.513+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e81abcb0 con 0x7f79e8103f80 2024-07-24T21:48:45.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.527+0000 7f79efc3b740 1 Processor -- start 2024-07-24T21:48:45.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.527+0000 7f79ec888640 1 -- 172.21.15.60:0/1829466087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f79dc01f020 con 0x7f79e8103f80 2024-07-24T21:48:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 -- start start 2024-07-24T21:48:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae629e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ae6294d960 con 0x55ae629d6680 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae6293ce60 con 0x55ae629d6fa0 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.532+0000 7f79efc3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae62940ea0 con 0x55ae629e2f30 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42548/0 (socket says 172.21.15.60:42548) 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 -- 172.21.15.60:0/862468592 learned_addr learned my addr 172.21.15.60:0/862468592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 -- 172.21.15.60:0/862468592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 msgr2=0x55ae629e29f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae629e29f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 -- 172.21.15.60:0/862468592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 msgr2=0x55ae629d6a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629d6a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.533+0000 7f79e67fc640 1 -- 172.21.15.60:0/862468592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae628c62d0 con 0x55ae629e2f30 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.534+0000 7f79e67fc640 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0e1410 tx=0x7f79dc0e7f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=342effada26c697a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.535+0000 7f79ec888640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 0x7f79d00b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.535+0000 7f79ec888640 1 -- 172.21.15.60:0/1829466087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79dc0e3de0 con 0x7f79e8103f80 2024-07-24T21:48:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.536+0000 7f79cdffb640 1 -- 172.21.15.60:0/862468592 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0f5640 con 0x55ae629e2f30 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.536+0000 7f79cdffb640 1 -- 172.21.15.60:0/862468592 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc0ece20 con 0x55ae629e2f30 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.537+0000 7f79cdffb640 1 -- 172.21.15.60:0/862468592 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0f4770 con 0x55ae629e2f30 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.538+0000 7f79efc3b740 1 -- 172.21.15.60:0/862468592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 msgr2=0x55ae629e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.538+0000 7f79efc3b740 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0e1410 tx=0x7f79dc0e7f90 comp rx=0 tx=0).stop 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.539+0000 7f79e77fe640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 0x7f79d00b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79e77fe640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 0x7f79d00b6410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79efc3b740 1 -- 172.21.15.60:0/862468592 shutdown_connections 2024-07-24T21:48:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79efc3b740 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae629e5320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79efc3b740 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae629e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79efc3b740 1 --2- 172.21.15.60:0/862468592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.540+0000 7f79efc3b740 1 -- 172.21.15.60:0/862468592 >> 172.21.15.60:0/862468592 conn(0x55ae629d44b0 msgr2=0x55ae629d5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.547+0000 7f79efc3b740 1 -- 172.21.15.60:0/862468592 shutdown_connections 2024-07-24T21:48:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.551+0000 7f79efc3b740 1 -- 172.21.15.60:0/862468592 wait complete. 2024-07-24T21:48:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.551+0000 7f79efc3b740 1 -- start start 2024-07-24T21:48:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.551+0000 7f79efc3b740 1 Processor -- start 2024-07-24T21:48:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629d6fa0 0x55ae629e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ae627a0700 con 0x55ae629d6680 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae628df5a0 con 0x55ae629e2f30 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae629f6bc0 con 0x55ae629d6fa0 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.553+0000 7f79efc3b740 10 client.0 _ll_register_callbacks cb 0x55ae628ff910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.554+0000 7f79e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.554+0000 7f79e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.555+0000 7f79e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59418/0 (socket says 172.21.15.60:59418) 2024-07-24T21:48:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.555+0000 7f79e77fe640 1 -- 172.21.15.60:0/3785545839 learned_addr learned my addr 172.21.15.60:0/3785545839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e67fc640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629d6fa0 0x55ae629e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e77fe640 1 -- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629d6fa0 msgr2=0x55ae629e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e77fe640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629d6fa0 0x55ae629e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e77fe640 1 -- 172.21.15.60:0/3785545839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 msgr2=0x55ae629dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e77fe640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.556+0000 7f79e77fe640 1 -- 172.21.15.60:0/3785545839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae628d2630 con 0x55ae629e2f30 2024-07-24T21:48:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.558+0000 7f79e77fe640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79e0003fd0 tx=0x7f79e0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a920dbecb7e4cb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.558+0000 7f79c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:48:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.558+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e000ed40 con 0x55ae629e2f30 2024-07-24T21:48:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.559+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79e001e420 con 0x55ae629e2f30 2024-07-24T21:48:45.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.559+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc0ece20 con 0x55ae629e2f30 2024-07-24T21:48:45.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.563+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79e0004810 con 0x55ae629e2f30 2024-07-24T21:48:45.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.564+0000 7f79efc3b740 10 client.25673 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:45.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.564+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae6269d650 con 0x55ae629e2f30 2024-07-24T21:48:45.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.566+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc0ece20 con 0x55ae629e2f30 2024-07-24T21:48:45.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.566+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79e0030770 con 0x55ae629e2f30 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 10 client.25673 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 20 client.25673 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae62a03b20 0x55ae62a05f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 4 client.25673 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.568+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae6294f8b0 con 0x55ae62a03b20 2024-07-24T21:48:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.573+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae62a03b20 0x55ae62a05f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.575+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae62a03b20 0x55ae62a05f60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2305ea432696a797 server_cookie=d26d1eb62a456af1 in_seq=0 out_seq=0 2024-07-24T21:48:45.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.577+0000 7f79c6ffd640 10 client.25673 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.580+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3785545839 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae6294f8b0 con 0x55ae62a03b20 2024-07-24T21:48:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.580+0000 7f79c6ffd640 10 client.25673 handle_command_reply: tid=0 2024-07-24T21:48:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.581+0000 7f79efc3b740 1 client.25673 shutdown 2024-07-24T21:48:45.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.583+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 msgr2=0x55ae629e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.583+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79e0003fd0 tx=0x7f79e0004ab0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 shutdown_connections 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae62a03b20 0x55ae62a05f60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629e2f30 0x55ae629e0970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629d6fa0 0x55ae629e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3785545839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 >> 172.21.15.60:0/3785545839 conn(0x55ae629d44b0 msgr2=0x55ae629da050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.585+0000 7f79c6ffd640 0 client.25673 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.587+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 shutdown_connections 2024-07-24T21:48:45.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.587+0000 7f79efc3b740 1 -- 172.21.15.60:0/3785545839 wait complete. 2024-07-24T21:48:45.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.587+0000 7f79efc3b740 20 client.25673 trim_cache size 0 max 16384 2024-07-24T21:48:45.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.632+0000 7f79efc3b740 1 Processor -- start 2024-07-24T21:48:45.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.632+0000 7f79efc3b740 1 -- start start 2024-07-24T21:48:45.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.632+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.632+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a0bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59430/0 (socket says 172.21.15.60:59430) 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e67fc640 1 -- 172.21.15.60:0/2172806049 learned_addr learned my addr 172.21.15.60:0/2172806049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e77fe640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a0bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79efc3b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4011220 con 0x55ae629d6680 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79d400ea90 con 0x55ae629d6fa0 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79d400c8e0 con 0x55ae629e2f30 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 -- 172.21.15.60:0/2172806049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 msgr2=0x55ae62a0bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a0bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 -- 172.21.15.60:0/2172806049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 msgr2=0x55ae62a0b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:45.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.633+0000 7f79e6ffd640 1 -- 172.21.15.60:0/2172806049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae629d4eb0 con 0x55ae629d6680 2024-07-24T21:48:45.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.634+0000 7f79e67fc640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:45.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.634+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629da7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f79d8037040 tx=0x7f79d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b5782d4ef4bdfe server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.636+0000 7f79cd7fa640 1 -- 172.21.15.60:0/2172806049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d800e670 con 0x55ae629d6680 2024-07-24T21:48:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.636+0000 7f79cd7fa640 1 -- 172.21.15.60:0/2172806049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d800e960 con 0x55ae629d6680 2024-07-24T21:48:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.637+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 msgr2=0x55ae629da7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.637+0000 7f79efc3b740 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629da7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f79d8037040 tx=0x7f79d8004ab0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.639+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 shutdown_connections 2024-07-24T21:48:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.639+0000 7f79efc3b740 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a0bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.639+0000 7f79efc3b740 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6fa0 0x55ae62a0b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.639+0000 7f79efc3b740 1 --2- 172.21.15.60:0/2172806049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6680 0x55ae629da7d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.639+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 >> 172.21.15.60:0/2172806049 conn(0x55ae629d44b0 msgr2=0x55ae629e1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.642+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 shutdown_connections 2024-07-24T21:48:45.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.642+0000 7f79efc3b740 1 -- 172.21.15.60:0/2172806049 wait complete. 2024-07-24T21:48:45.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.643+0000 7f79efc3b740 1 -- start start 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.643+0000 7f79efc3b740 1 Processor -- start 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.643+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6680 0x55ae629f7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 0x55ae629f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79d400cf50 con 0x55ae629d6fa0 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79d400e3c0 con 0x55ae629d6680 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79d400d620 con 0x55ae629e2f30 2024-07-24T21:48:45.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79efc3b740 10 client.0 _ll_register_callbacks cb 0x7f79e0009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79e67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 0x55ae629f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79e77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79e77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a6c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42584/0 (socket says 172.21.15.60:42584) 2024-07-24T21:48:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.644+0000 7f79e77fe640 1 -- 172.21.15.60:0/3076753294 learned_addr learned my addr 172.21.15.60:0/3076753294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:45.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.645+0000 7f79e67fc640 1 -- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 msgr2=0x55ae62a6c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:45.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.645+0000 7f79e67fc640 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a6c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.645+0000 7f79e67fc640 1 -- 172.21.15.60:0/3076753294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6680 msgr2=0x55ae629f7670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:48:45.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.645+0000 7f79e67fc640 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6680 0x55ae629f7670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.645+0000 7f79e67fc640 1 -- 172.21.15.60:0/3076753294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae62a6ce60 con 0x55ae629d6fa0 2024-07-24T21:48:45.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.646+0000 7f79e67fc640 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 0x55ae629f7bb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0e78f0 tx=0x7f79dc0e9e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa3d8cffd529203 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.648+0000 7f79c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.648+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0a6850 con 0x55ae629d6fa0 2024-07-24T21:48:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.648+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79dc0aabf0 con 0x55ae629d6fa0 2024-07-24T21:48:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.650+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae6294b350 con 0x55ae629d6fa0 2024-07-24T21:48:45.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.652+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79dc0a6120 con 0x55ae629d6fa0 2024-07-24T21:48:45.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.653+0000 7f79efc3b740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:45.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.653+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae629e5ac0 con 0x55ae629d6fa0 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ae6294b350 con 0x55ae629d6fa0 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79dc0f3070 con 0x55ae629d6fa0 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 20 client.16191 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae629e5cf0 0x55ae629e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:45.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 4 client.16191 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:48:45.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.656+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae62948a90 con 0x55ae629e5cf0 2024-07-24T21:48:45.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.658+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae629e5cf0 0x55ae629e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:45.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.669+0000 7f79e6ffd640 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae629e5cf0 0x55ae629e80e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e1eacca677dbfa server_cookie=65d17bb6a0ddd6f0 in_seq=0 out_seq=0 2024-07-24T21:48:45.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.670+0000 7f79c6ffd640 10 client.16191 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.673+0000 7f79c6ffd640 1 -- 172.21.15.60:0/3076753294 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae62948a90 con 0x55ae629e5cf0 2024-07-24T21:48:45.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.673+0000 7f79c6ffd640 10 client.16191 handle_command_reply: tid=0 2024-07-24T21:48:45.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.676+0000 7f79efc3b740 1 client.16191 shutdown 2024-07-24T21:48:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 msgr2=0x55ae629f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 0x55ae629f7bb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0e78f0 tx=0x7f79dc0e9e40 comp rx=0 tx=0).stop 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 shutdown_connections 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae629e5cf0 0x55ae629e80e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae629e2f30 0x55ae62a6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae629d6fa0 0x55ae629f7bb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 --2- 172.21.15.60:0/3076753294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae629d6680 0x55ae629f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.682+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 >> 172.21.15.60:0/3076753294 conn(0x55ae629d44b0 msgr2=0x55ae629d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.683+0000 7f79c6ffd640 0 client.16191 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.685+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 shutdown_connections 2024-07-24T21:48:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.685+0000 7f79efc3b740 1 -- 172.21.15.60:0/3076753294 wait complete. 2024-07-24T21:48:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.685+0000 7f79efc3b740 20 client.16191 trim_cache size 0 max 16384 2024-07-24T21:48:45.690 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:48:45.690 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:48:45.690 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:48:45.690 INFO:teuthology.orchestra.run.smithi060.stdout: "39b21460-e888-4b72-9c97-0fba157cbde0": { 2024-07-24T21:48:45.690 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:48:45.691 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "39b21460-e888-4b72-9c97-0fba157cbde0", 2024-07-24T21:48:45.691 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:48:45.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:45.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:48:45.691 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:48:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.692+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 msgr2=0x7f79d00b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.692+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 0x7f79d00b6410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e0009500 comp rx=0 tx=0).stop 2024-07-24T21:48:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.692+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 msgr2=0x7f79e819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.692+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e9c0 tx=0x7f79dc00edc0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 shutdown_connections 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79d00b3f50 0x7f79d00b6410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79e810cac0 0x7f79e81a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e8103f80 0x7f79e819f790 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 --2- 172.21.15.60:0/1829466087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e8103660 0x7f79e819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.693+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 >> 172.21.15.60:0/1829466087 conn(0x7f79e80fd0c0 msgr2=0x7f79e810ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.694+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 shutdown_connections 2024-07-24T21:48:45.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:48:45.695+0000 7f79ed88a640 1 -- 172.21.15.60:0/1829466087 wait complete. 2024-07-24T21:48:45.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:45 smithi123 ceph-mon[37143]: pgmap v439: 97 pgs: 97 active+clean; 61 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-07-24T21:48:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:45 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:45 smithi086 ceph-mon[37128]: pgmap v439: 97 pgs: 97 active+clean; 61 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-07-24T21:48:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:45 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:48:46.573 INFO:tasks.cephfs.filesystem:scrub status for tag:39b21460-e888-4b72-9c97-0fba157cbde0 - {'path': '/', 'tag': '39b21460-e888-4b72-9c97-0fba157cbde0', 'options': 'recursive,force'} 2024-07-24T21:48:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:47 smithi060 ceph-mon[33000]: pgmap v440: 97 pgs: 97 active+clean; 61 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-07-24T21:48:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:47 smithi123 ceph-mon[37143]: pgmap v440: 97 pgs: 97 active+clean; 61 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-07-24T21:48:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:47 smithi086 ceph-mon[37128]: pgmap v440: 97 pgs: 97 active+clean; 61 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-07-24T21:48:49.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:49 smithi123 ceph-mon[37143]: pgmap v441: 97 pgs: 97 active+clean; 58 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-07-24T21:48:49.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:49 smithi086 ceph-mon[37128]: pgmap v441: 97 pgs: 97 active+clean; 58 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-07-24T21:48:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:49 smithi060 ceph-mon[33000]: pgmap v441: 97 pgs: 97 active+clean; 58 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:50.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:50.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:50.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:50.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:51.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:51 smithi123 ceph-mon[37143]: pgmap v442: 97 pgs: 97 active+clean; 60 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-07-24T21:48:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:51 smithi086 ceph-mon[37128]: pgmap v442: 97 pgs: 97 active+clean; 60 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-07-24T21:48:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:51 smithi060 ceph-mon[33000]: pgmap v442: 97 pgs: 97 active+clean; 60 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-07-24T21:48:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: pgmap v443: 97 pgs: 97 active+clean; 70 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:48:53.757 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: pgmap v443: 97 pgs: 97 active+clean; 70 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:48:53.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: pgmap v443: 97 pgs: 97 active+clean; 70 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:53.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:48:53.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:54.707 INFO:tasks.workunit.client.0.smithi060.stdout: 42 609858 772 374060 569 273731 1622 2024-07-24T21:48:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:55 smithi123 ceph-mon[37143]: pgmap v444: 97 pgs: 97 active+clean; 70 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-07-24T21:48:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:55 smithi086 ceph-mon[37128]: pgmap v444: 97 pgs: 97 active+clean; 70 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-07-24T21:48:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:55 smithi060 ceph-mon[33000]: pgmap v444: 97 pgs: 97 active+clean; 70 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-07-24T21:48:56.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:56.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:56.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:48:57.705 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:57 smithi060 ceph-mon[33000]: pgmap v445: 97 pgs: 97 active+clean; 73 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-07-24T21:48:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:57 smithi123 ceph-mon[37143]: pgmap v445: 97 pgs: 97 active+clean; 73 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-07-24T21:48:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:57 smithi086 ceph-mon[37128]: pgmap v445: 97 pgs: 97 active+clean; 73 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-07-24T21:48:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- 172.21.15.60:0/2735246163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5494101a80 msgr2=0x7f549410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 --2- 172.21.15.60:0/2735246163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5494101a80 0x7f549410e7d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5484009900 tx=0x7f548402a480 comp rx=0 tx=0).stop 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- 172.21.15.60:0/2735246163 shutdown_connections 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 --2- 172.21.15.60:0/2735246163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549410ed10 0x7f5494111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 --2- 172.21.15.60:0/2735246163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5494101a80 0x7f549410e7d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 --2- 172.21.15.60:0/2735246163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101160 0x7f5494101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- 172.21.15.60:0/2735246163 >> 172.21.15.60:0/2735246163 conn(0x7f54940fd030 msgr2=0x7f54940ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- 172.21.15.60:0/2735246163 shutdown_connections 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- 172.21.15.60:0/2735246163 wait complete. 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 Processor -- start 2024-07-24T21:48:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.103+0000 7f549bd53640 1 -- start start 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101a80 0x7f549419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 0x7f54941a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f54941148c0 con 0x7f549410ed10 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5494114740 con 0x7f5494101160 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549bd53640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5494114a40 con 0x7f5494101a80 2024-07-24T21:48:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 0x7f54941a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:58814/0 (socket says 172.21.15.60:58814) 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 -- 172.21.15.60:0/2167943597 learned_addr learned my addr 172.21.15.60:0/2167943597 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f54992ee640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101a80 0x7f549419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 -- 172.21.15.60:0/2167943597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101a80 msgr2=0x7f549419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101a80 0x7f549419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 -- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 msgr2=0x7f54941a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 0x7f54941a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 -- 172.21.15.60:0/2167943597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54941a0ea0 con 0x7f5494101160 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f549a2f0640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 0x7f54941a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.104+0000 7f5499aef640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.105+0000 7f548affd640 1 -- 172.21.15.60:0/2167943597 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000cd80 con 0x7f5494101160 2024-07-24T21:48:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.105+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54941a1190 con 0x7f5494101160 2024-07-24T21:48:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.105+0000 7f548affd640 1 -- 172.21.15.60:0/2167943597 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54900054d0 con 0x7f5494101160 2024-07-24T21:48:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.105+0000 7f548affd640 1 -- 172.21.15.60:0/2167943597 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490010690 con 0x7f5494101160 2024-07-24T21:48:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.105+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54941a7850 con 0x7f5494101160 2024-07-24T21:48:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.107+0000 7f549eb82740 1 Processor -- start 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 -- start start 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f548affd640 1 -- 172.21.15.60:0/2167943597 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5490004e70 con 0x7f5494101160 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fd1e0 0x563a9f608c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563a9f572280 con 0x563a9f5fc8c0 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a9f561780 con 0x563a9f5fd1e0 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549eb82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a9f5657c0 con 0x563a9f609170 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f5499aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fd1e0 0x563a9f608c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57496/0 (socket says 172.21.15.60:57496) 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 -- 172.21.15.60:0/2130558389 learned_addr learned my addr 172.21.15.60:0/2130558389 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f54992ee640 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 -- 172.21.15.60:0/2130558389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 msgr2=0x563a9f60b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 -- 172.21.15.60:0/2130558389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fd1e0 msgr2=0x563a9f608c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fd1e0 0x563a9f608c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.108+0000 7f549a2f0640 1 -- 172.21.15.60:0/2130558389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9f2c5790 con 0x563a9f5fc8c0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f549a2f0640 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f548c0117e0 tx=0x7f548c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb56a01afafbecc0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f548affd640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 0x7f54800b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f547ffff640 1 -- 172.21.15.60:0/2130558389 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c011310 con 0x563a9f5fc8c0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f547ffff640 1 -- 172.21.15.60:0/2130558389 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c003e20 con 0x563a9f5fc8c0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f547ffff640 1 -- 172.21.15.60:0/2130558389 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c004480 con 0x563a9f5fc8c0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f5499aef640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 0x7f54800b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.109+0000 7f548affd640 1 -- 172.21.15.60:0/2167943597 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f54900e4410 con 0x7f5494101160 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- 172.21.15.60:0/2130558389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 msgr2=0x563a9f5fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f548c0117e0 tx=0x7f548c011ca0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f5499aef640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 0x7f54800b6200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f54900e1c10 tx=0x7f54900e7f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- 172.21.15.60:0/2130558389 shutdown_connections 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fd1e0 0x563a9f608c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 --2- 172.21.15.60:0/2130558389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fc8c0 0x563a9f5fcca0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- 172.21.15.60:0/2130558389 >> 172.21.15.60:0/2130558389 conn(0x563a9f5fa6f0 msgr2=0x563a9f5fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- 172.21.15.60:0/2130558389 shutdown_connections 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- 172.21.15.60:0/2130558389 wait complete. 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 -- start start 2024-07-24T21:48:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.110+0000 7f549eb82740 1 Processor -- start 2024-07-24T21:48:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 0x563a9f605df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f609170 0x563a9f606330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563a9f49bab0 con 0x563a9f609170 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a9f4e99e0 con 0x563a9f5fc8c0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a9f60be50 con 0x563a9f5fd1e0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549eb82740 10 client.0 _ll_register_callbacks cb 0x563a9f271a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:58834/0 (socket says 172.21.15.60:58834) 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549a2f0640 1 -- 172.21.15.60:0/3621599492 learned_addr learned my addr 172.21.15.60:0/3621599492 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 0x563a9f605df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f609170 0x563a9f606330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 -- 172.21.15.60:0/3621599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 msgr2=0x563a9f608910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 -- 172.21.15.60:0/3621599492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f609170 msgr2=0x563a9f606330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f609170 0x563a9f606330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 -- 172.21.15.60:0/3621599492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9f33e620 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f54992ee640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 0x563a9f605df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5484009230 tx=0x7f5484032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1669f86db4c27271 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.111+0000 7f547d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484035040 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f54840316a0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a9f510ce0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54840318a0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a9f617630 con 0x563a9f5fd1e0 2024-07-24T21:48:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563a9f510ce0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5484004750 con 0x563a9f5fd1e0 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 20 client.25705 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f618b90 0x563a9f61af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 4 client.25705 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a9f5741d0 con 0x563a9f618b90 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.112+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f618b90 0x563a9f61af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.113+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f618b90 0x563a9f61af80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4630e71bd0672f0b server_cookie=abe0639401d44246 in_seq=0 out_seq=0 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.113+0000 7f547d7fa640 10 client.25705 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.113+0000 7f547d7fa640 1 -- 172.21.15.60:0/3621599492 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a9f5741d0 con 0x563a9f618b90 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.113+0000 7f547d7fa640 10 client.25705 handle_command_reply: tid=0 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.113+0000 7f549eb82740 1 client.25705 shutdown 2024-07-24T21:48:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 msgr2=0x563a9f605df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 0x563a9f605df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5484009230 tx=0x7f5484032dc0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 shutdown_connections 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f618b90 0x563a9f61af80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f609170 0x563a9f606330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fd1e0 0x563a9f605df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 --2- 172.21.15.60:0/3621599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 >> 172.21.15.60:0/3621599492 conn(0x563a9f5fa6f0 msgr2=0x563a9f600220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f547d7fa640 0 client.25705 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 shutdown_connections 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 1 -- 172.21.15.60:0/3621599492 wait complete. 2024-07-24T21:48:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.114+0000 7f549eb82740 20 client.25705 trim_cache size 0 max 16384 2024-07-24T21:48:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 Processor -- start 2024-07-24T21:48:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 -- start start 2024-07-24T21:48:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fc8c0 0x563a9f614f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 0x563a9f6062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f54941b2c60 con 0x563a9f5fd1e0 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54941b04d0 con 0x563a9f609170 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.130+0000 7f549eb82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54941ae320 con 0x563a9f5fc8c0 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57512/0 (socket says 172.21.15.60:57512) 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 -- 172.21.15.60:0/3752110691 learned_addr learned my addr 172.21.15.60:0/3752110691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 0x563a9f6062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fc8c0 0x563a9f614f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 -- 172.21.15.60:0/3752110691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fc8c0 msgr2=0x563a9f614f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fc8c0 0x563a9f614f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 -- 172.21.15.60:0/3752110691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 msgr2=0x563a9f6062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 0x563a9f6062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 -- 172.21.15.60:0/3752110691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9f5fd5c0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 0x563a9f6062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f54992ee640 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5484004d10 tx=0x7f5484000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff02e5ea32a7410 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f547f7fe640 1 -- 172.21.15.60:0/3752110691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484052070 con 0x563a9f5fd1e0 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f547f7fe640 1 -- 172.21.15.60:0/3752110691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840314a0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.131+0000 7f547f7fe640 1 -- 172.21.15.60:0/3752110691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484035020 con 0x563a9f5fd1e0 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- 172.21.15.60:0/3752110691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 msgr2=0x563a9f5fbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5484004d10 tx=0x7f5484000bc0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- 172.21.15.60:0/3752110691 shutdown_connections 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f609170 0x563a9f6062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f5fbf60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 --2- 172.21.15.60:0/3752110691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f5fc8c0 0x563a9f614f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- 172.21.15.60:0/3752110691 >> 172.21.15.60:0/3752110691 conn(0x563a9f5fa6f0 msgr2=0x563a9f600be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- 172.21.15.60:0/3752110691 shutdown_connections 2024-07-24T21:48:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- 172.21.15.60:0/3752110691 wait complete. 2024-07-24T21:48:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 -- start start 2024-07-24T21:48:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.132+0000 7f549eb82740 1 Processor -- start 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f6115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f54941ae990 con 0x563a9f5fd1e0 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54941afe00 con 0x563a9f5fc8c0 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54941af060 con 0x563a9f609170 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 10 client.0 _ll_register_callbacks cb 0x7f5490005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549a2f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f6115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57520/0 (socket says 172.21.15.60:57520) 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 -- 172.21.15.60:0/3673128187 learned_addr learned my addr 172.21.15.60:0/3673128187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 -- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 msgr2=0x563a9f60c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 -- 172.21.15.60:0/3673128187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 msgr2=0x563a9f6115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f6115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 -- 172.21.15.60:0/3673128187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9f60cc20 con 0x563a9f5fd1e0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f54992ee640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5484004af0 tx=0x7f5484005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39f8b8e708bbcaa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f547d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484052040 con 0x563a9f5fd1e0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54840314a0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.133+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5484031a40 con 0x563a9f5fd1e0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5484031c40 con 0x563a9f5fd1e0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a9f682690 con 0x563a9f5fd1e0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840314a0 con 0x563a9f5fd1e0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5484032280 con 0x563a9f5fd1e0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 20 client.16209 populate_metadata read hostname 'smithi060' 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f682d00 0x563a9f685120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 4 client.16209 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a9f56fc70 con 0x563a9f682d00 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.134+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f682d00 0x563a9f685120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.135+0000 7f547d7fa640 10 client.16209 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:58.135+0000 7f549a2f0640 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f682d00 0x563a9f685120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d336da0c1f72949c server_cookie=bace326383664811 in_seq=0 out_seq=0 2024-07-24T21:48:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.472+0000 7f547d7fa640 1 -- 172.21.15.60:0/3673128187 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a9f56fc70 con 0x563a9f682d00 2024-07-24T21:48:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.472+0000 7f547d7fa640 10 client.16209 handle_command_reply: tid=0 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.472+0000 7f549eb82740 1 client.16209 shutdown 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.472+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 msgr2=0x563a9f60c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.472+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5484004af0 tx=0x7f5484005fd0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 shutdown_connections 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563a9f682d00 0x563a9f685120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f547d7fa640 0 client.16209 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a9f609170 0x563a9f60c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a9f5fd1e0 0x563a9f60c0a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 --2- 172.21.15.60:0/3673128187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a9f5fc8c0 0x563a9f6115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 >> 172.21.15.60:0/3673128187 conn(0x563a9f5fa6f0 msgr2=0x563a9f601340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 shutdown_connections 2024-07-24T21:48:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 1 -- 172.21.15.60:0/3673128187 wait complete. 2024-07-24T21:48:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549eb82740 20 client.16209 trim_cache size 0 max 16384 2024-07-24T21:48:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:48:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:48:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:48:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 msgr2=0x7f54800b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 0x7f54800b6200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f54900e1c10 tx=0x7f54900e7f60 comp rx=0 tx=0).stop 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 msgr2=0x7f549419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.473+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000edf0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 shutdown_connections 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54800b3d40 0x7f54800b6200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f549410ed10 0x7f54941a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5494101a80 0x7f549419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 --2- 172.21.15.60:0/2167943597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494101160 0x7f549419ae00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:48:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 >> 172.21.15.60:0/2167943597 conn(0x7f54940fd030 msgr2=0x7f54940ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:48:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 shutdown_connections 2024-07-24T21:48:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:48:59.474+0000 7f549bd53640 1 -- 172.21.15.60:0/2167943597 wait complete. 2024-07-24T21:48:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:48:59 smithi123 ceph-mon[37143]: pgmap v446: 97 pgs: 97 active+clean; 116 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 315 op/s 2024-07-24T21:48:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:48:59 smithi086 ceph-mon[37128]: pgmap v446: 97 pgs: 97 active+clean; 116 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 315 op/s 2024-07-24T21:48:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:48:59 smithi060 ceph-mon[33000]: pgmap v446: 97 pgs: 97 active+clean; 116 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 315 op/s 2024-07-24T21:49:01.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:01 smithi123 ceph-mon[37143]: pgmap v447: 97 pgs: 97 active+clean; 153 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 599 op/s 2024-07-24T21:49:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:01 smithi086 ceph-mon[37128]: pgmap v447: 97 pgs: 97 active+clean; 153 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 599 op/s 2024-07-24T21:49:01.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:01 smithi060 ceph-mon[33000]: pgmap v447: 97 pgs: 97 active+clean; 153 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 599 op/s 2024-07-24T21:49:03.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:03 smithi123 ceph-mon[37143]: pgmap v448: 97 pgs: 97 active+clean; 172 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-07-24T21:49:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:03 smithi086 ceph-mon[37128]: pgmap v448: 97 pgs: 97 active+clean; 172 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-07-24T21:49:03.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:03 smithi060 ceph-mon[33000]: pgmap v448: 97 pgs: 97 active+clean; 172 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-07-24T21:49:04.707 INFO:tasks.workunit.client.0.smithi060.stdout: 51 148870 527 99338 634 58519 567 2024-07-24T21:49:04.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:04 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:04.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:04 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:04.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:04 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:04 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:04.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:04 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:04.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:04 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:05.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:05 smithi123 ceph-mon[37143]: pgmap v449: 97 pgs: 97 active+clean; 179 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-07-24T21:49:05.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:05 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:05.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:05 smithi086 ceph-mon[37128]: pgmap v449: 97 pgs: 97 active+clean; 179 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-07-24T21:49:05.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:05 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:05 smithi060 ceph-mon[33000]: pgmap v449: 97 pgs: 97 active+clean; 179 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-07-24T21:49:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:05 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:07.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:07 smithi123 ceph-mon[37143]: pgmap v450: 97 pgs: 97 active+clean; 191 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 936 op/s 2024-07-24T21:49:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:07 smithi086 ceph-mon[37128]: pgmap v450: 97 pgs: 97 active+clean; 191 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 936 op/s 2024-07-24T21:49:07.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:07 smithi060 ceph-mon[33000]: pgmap v450: 97 pgs: 97 active+clean; 191 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 936 op/s 2024-07-24T21:49:09.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:09 smithi123 ceph-mon[37143]: pgmap v451: 97 pgs: 97 active+clean; 199 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 964 op/s 2024-07-24T21:49:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:09 smithi086 ceph-mon[37128]: pgmap v451: 97 pgs: 97 active+clean; 199 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 964 op/s 2024-07-24T21:49:09.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:09 smithi060 ceph-mon[33000]: pgmap v451: 97 pgs: 97 active+clean; 199 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 964 op/s 2024-07-24T21:49:11.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:11 smithi123 ceph-mon[37143]: pgmap v452: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 766 op/s 2024-07-24T21:49:11.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:11 smithi086 ceph-mon[37128]: pgmap v452: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 766 op/s 2024-07-24T21:49:11.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:11 smithi060 ceph-mon[33000]: pgmap v452: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 766 op/s 2024-07-24T21:49:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 -- 172.21.15.60:0/754102004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 msgr2=0x7f79fc10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 --2- 172.21.15.60:0/754102004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc10c490 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009900 tx=0x7f79e402a410 comp rx=0 tx=0).stop 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 -- 172.21.15.60:0/754102004 shutdown_connections 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 --2- 172.21.15.60:0/754102004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc10ca60 0x7f79fc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 --2- 172.21.15.60:0/754102004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc10c490 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 --2- 172.21.15.60:0/754102004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc103420 0x7f79fc103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 -- 172.21.15.60:0/754102004 >> 172.21.15.60:0/754102004 conn(0x7f79fc0782e0 msgr2=0x7f79fc0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 -- 172.21.15.60:0/754102004 shutdown_connections 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 -- 172.21.15.60:0/754102004 wait complete. 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.566+0000 7f7a02c65640 1 Processor -- start 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 -- start start 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc103420 0x7f79fc101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc10ca60 0x7f79fc101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1126e0 con 0x7f79fc103e20 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc112560 con 0x7f79fc10ca60 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a02c65640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc112860 con 0x7f79fc103420 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40562/0 (socket says 172.21.15.60:40562) 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 -- 172.21.15.60:0/2553636367 learned_addr learned my addr 172.21.15.60:0/2553636367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a01202640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc10ca60 0x7f79fc101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f7a00a01640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc103420 0x7f79fc101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 -- 172.21.15.60:0/2553636367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc103420 msgr2=0x7f79fc101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc103420 0x7f79fc101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 -- 172.21.15.60:0/2553636367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc10ca60 msgr2=0x7f79fc101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc10ca60 0x7f79fc101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 -- 172.21.15.60:0/2553636367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc1a7030 con 0x7f79fc103e20 2024-07-24T21:49:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79fbfff640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005ab0 tx=0x7f79e4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.567+0000 7f79f9ffb640 1 -- 172.21.15.60:0/2553636367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e403f070 con 0x7f79fc103e20 2024-07-24T21:49:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.568+0000 7f79f9ffb640 1 -- 172.21.15.60:0/2553636367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e4031d60 con 0x7f79fc103e20 2024-07-24T21:49:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.568+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc1a72c0 con 0x7f79fc103e20 2024-07-24T21:49:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.568+0000 7f79f9ffb640 1 -- 172.21.15.60:0/2553636367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4035030 con 0x7f79fc103e20 2024-07-24T21:49:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.568+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc1a7800 con 0x7f79fc103e20 2024-07-24T21:49:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.570+0000 7f7a059c2740 1 Processor -- start 2024-07-24T21:49:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.570+0000 7f79f9ffb640 1 -- 172.21.15.60:0/2553636367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f79e4042070 con 0x7f79fc103e20 2024-07-24T21:49:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.570+0000 7f7a059c2740 1 -- start start 2024-07-24T21:49:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x565380618280 con 0x5653806a31e0 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565380607780 con 0x5653806af170 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a059c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56538060b7c0 con 0x5653806a28c0 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40576/0 (socket says 172.21.15.60:40576) 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 -- 172.21.15.60:0/775048051 learned_addr learned my addr 172.21.15.60:0/775048051 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f79fbfff640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a01202640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 -- 172.21.15.60:0/775048051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 msgr2=0x5653806a2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806a2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 -- 172.21.15.60:0/775048051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 msgr2=0x5653806b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 -- 172.21.15.60:0/775048051 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538036b790 con 0x5653806a31e0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f79fbfff640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f79f9ffb640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 0x7f79e80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.571+0000 7f7a00a01640 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009be0 tx=0x7f79ec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b691e4a16f6287 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a01202640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 0x7f79e80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f79f2ffd640 1 -- 172.21.15.60:0/775048051 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec016070 con 0x5653806a31e0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f79f2ffd640 1 -- 172.21.15.60:0/775048051 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ec0054d0 con 0x5653806a31e0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f79f2ffd640 1 -- 172.21.15.60:0/775048051 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec011470 con 0x5653806a31e0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f79f9ffb640 1 -- 172.21.15.60:0/2553636367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79e4104860 con 0x7f79fc103e20 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a059c2740 1 -- 172.21.15.60:0/775048051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 msgr2=0x5653806aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a059c2740 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009be0 tx=0x7f79ec00e6c0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a01202640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 0x7f79e80b6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f79fc1033f0 tx=0x7f79f400a4d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a059c2740 1 -- 172.21.15.60:0/775048051 shutdown_connections 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.572+0000 7f7a059c2740 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aec30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 --2- 172.21.15.60:0/775048051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- 172.21.15.60:0/775048051 >> 172.21.15.60:0/775048051 conn(0x5653806a06f0 msgr2=0x5653806a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- 172.21.15.60:0/775048051 shutdown_connections 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- 172.21.15.60:0/775048051 wait complete. 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- start start 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 Processor -- start 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 0x5653806abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x565380541ab0 con 0x5653806a28c0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56538058f9e0 con 0x5653806af170 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.573+0000 7f7a059c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653806b1e50 con 0x5653806a31e0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a059c2740 10 client.0 _ll_register_callbacks cb 0x565380317a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a00a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a01202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 0x5653806abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a00a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40590/0 (socket says 172.21.15.60:40590) 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a00a01640 1 -- 172.21.15.60:0/3576169447 learned_addr learned my addr 172.21.15.60:0/3576169447 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 -- 172.21.15.60:0/3576169447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 msgr2=0x5653806ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 -- 172.21.15.60:0/3576169447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 msgr2=0x5653806ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 -- 172.21.15.60:0/3576169447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653803e4620 con 0x5653806a31e0 2024-07-24T21:49:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a01202640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a00a01640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79fbfff640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 0x5653806abd10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79e4101fb0 tx=0x7f79e402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be4ce2d05a6fb0c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4049070 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653805b6ce0 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.574+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79e403b580 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79e40cadc0 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 10 client.25717 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653806bd590 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5653805b6ce0 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79e40cb6d0 con 0x5653806a31e0 2024-07-24T21:49:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 10 client.25717 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 20 client.25717 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x5653806c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 4 client.25717 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56538061a1d0 con 0x5653806bea50 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.575+0000 7f7a00a01640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x5653806c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.576+0000 7f79dbfff640 10 client.25717 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.576+0000 7f7a00a01640 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x5653806c0ed0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f672ac21eb3ef6c2 server_cookie=7c97897d8f2abee in_seq=0 out_seq=0 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f79dbfff640 1 -- 172.21.15.60:0/3576169447 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56538061a1d0 con 0x5653806bea50 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f79dbfff640 10 client.25717 handle_command_reply: tid=0 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 client.25717 shutdown 2024-07-24T21:49:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 msgr2=0x5653806abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 0x5653806abd10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79e4101fb0 tx=0x7f79e402a410 comp rx=0 tx=0).stop 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 shutdown_connections 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x5653806c0ed0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806af170 0x5653806ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f79dbfff640 0 client.25717 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a31e0 0x5653806abd10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 --2- 172.21.15.60:0/3576169447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a28c0 0x5653806ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 >> 172.21.15.60:0/3576169447 conn(0x5653806a06f0 msgr2=0x5653806a6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 shutdown_connections 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 1 -- 172.21.15.60:0/3576169447 wait complete. 2024-07-24T21:49:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.577+0000 7f7a059c2740 20 client.25717 trim_cache size 0 max 16384 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.599+0000 7f7a059c2740 1 Processor -- start 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.599+0000 7f7a059c2740 1 -- start start 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 0x5653806aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806af170 0x5653806c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1b2bf0 con 0x5653806a31e0 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1b0460 con 0x5653806a28c0 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a059c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1ae2b0 con 0x5653806af170 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40592/0 (socket says 172.21.15.60:40592) 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a00a01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 0x5653806aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 -- 172.21.15.60:0/4107728764 learned_addr learned my addr 172.21.15.60:0/4107728764 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a01202640 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806af170 0x5653806c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 -- 172.21.15.60:0/4107728764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806af170 msgr2=0x5653806c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806af170 0x5653806c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 -- 172.21.15.60:0/4107728764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 msgr2=0x5653806aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 0x5653806aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 -- 172.21.15.60:0/4107728764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653806a3010 con 0x5653806a31e0 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f7a00a01640 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 0x5653806aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.600+0000 7f79fbfff640 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f79e40c86e0 tx=0x7f79e4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39ed56733206bdb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.601+0000 7f79f27fc640 1 -- 172.21.15.60:0/4107728764 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40e4070 con 0x5653806a31e0 2024-07-24T21:49:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.601+0000 7f79f27fc640 1 -- 172.21.15.60:0/4107728764 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e4106e50 con 0x5653806a31e0 2024-07-24T21:49:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.601+0000 7f79f27fc640 1 -- 172.21.15.60:0/4107728764 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4049070 con 0x5653806a31e0 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- 172.21.15.60:0/4107728764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 msgr2=0x5653806aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f79e40c86e0 tx=0x7f79e4031400 comp rx=0 tx=0).stop 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- 172.21.15.60:0/4107728764 shutdown_connections 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806af170 0x5653806c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806a31e0 0x5653806aaab0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 --2- 172.21.15.60:0/4107728764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a28c0 0x5653806aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- 172.21.15.60:0/4107728764 >> 172.21.15.60:0/4107728764 conn(0x5653806a06f0 msgr2=0x5653806a1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- 172.21.15.60:0/4107728764 shutdown_connections 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- 172.21.15.60:0/4107728764 wait complete. 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.602+0000 7f7a059c2740 1 -- start start 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 Processor -- start 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 0x5653806b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1ae920 con 0x5653806af170 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1afd90 con 0x5653806a31e0 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1aeff0 con 0x5653806a28c0 2024-07-24T21:49:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a059c2740 10 client.0 _ll_register_callbacks cb 0x7f79e403b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a01202640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 0x5653806b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42606/0 (socket says 172.21.15.60:42606) 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f79fbfff640 1 -- 172.21.15.60:0/1885374203 learned_addr learned my addr 172.21.15.60:0/1885374203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.603+0000 7f7a00a01640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 -- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 msgr2=0x5653806b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 -- 172.21.15.60:0/1885374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 msgr2=0x5653806b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 0x5653806b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 -- 172.21.15.60:0/1885374203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565380727760 con 0x5653806a31e0 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f7a01202640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 0x5653806b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f7a00a01640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79fbfff640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f79e402a9d0 tx=0x7f79e403b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fada41f432e5439 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40e4030 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79e40c7a60 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e4106e50 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79e4049070 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f7a059c2740 10 client.25712 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.604+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565380727760 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e403be50 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79e40c5b90 con 0x5653806a31e0 2024-07-24T21:49:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 10 client.25712 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 20 client.25712 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x56538072ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 4 client.25712 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f79e4106e50 con 0x5653806bea50 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.605+0000 7f7a00a01640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x56538072ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.606+0000 7f7a00a01640 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x56538072ac30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b977babc612d3b6c server_cookie=31579fd71a1fef0f in_seq=0 out_seq=0 2024-07-24T21:49:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:12.606+0000 7f79dbfff640 10 client.25712 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f79dbfff640 1 -- 172.21.15.60:0/1885374203 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f79e4106e50 con 0x5653806bea50 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f79dbfff640 10 client.25712 handle_command_reply: tid=0 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 client.25712 shutdown 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 msgr2=0x5653806b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f79e402a9d0 tx=0x7f79e403b7b0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 shutdown_connections 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5653806bea50 0x56538072ac30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5653806af170 0x5653806b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653806a31e0 0x5653806b5350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 --2- 172.21.15.60:0/1885374203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653806a28c0 0x5653806b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 >> 172.21.15.60:0/1885374203 conn(0x5653806a06f0 msgr2=0x5653806a6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f79dbfff640 0 client.25712 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 shutdown_connections 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.218+0000 7f7a059c2740 1 -- 172.21.15.60:0/1885374203 wait complete. 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.219+0000 7f7a059c2740 20 client.25712 trim_cache size 0 max 16384 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:49:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:49:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.219+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 msgr2=0x7f79e80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.219+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 0x7f79e80b6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f79fc1033f0 tx=0x7f79f400a4d0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.219+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 msgr2=0x7f79fc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.219+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005ab0 tx=0x7f79e4003cd0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 shutdown_connections 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79e80b3dc0 0x7f79e80b6280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc10ca60 0x7f79fc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc103e20 0x7f79fc101870 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 --2- 172.21.15.60:0/2553636367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc103420 0x7f79fc101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 >> 172.21.15.60:0/2553636367 conn(0x7f79fc0782e0 msgr2=0x7f79fc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 shutdown_connections 2024-07-24T21:49:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:13.220+0000 7f7a02c65640 1 -- 172.21.15.60:0/2553636367 wait complete. 2024-07-24T21:49:13.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:13 smithi123 ceph-mon[37143]: pgmap v453: 97 pgs: 97 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 481 op/s 2024-07-24T21:49:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:13 smithi086 ceph-mon[37128]: pgmap v453: 97 pgs: 97 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 481 op/s 2024-07-24T21:49:13.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:13 smithi060 ceph-mon[33000]: pgmap v453: 97 pgs: 97 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 481 op/s 2024-07-24T21:49:14.707 INFO:tasks.workunit.client.0.smithi060.stdout: 82 256864 1581 173620 1605 106160 756 2024-07-24T21:49:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:15 smithi123 ceph-mon[37143]: pgmap v454: 97 pgs: 97 active+clean; 211 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-07-24T21:49:15.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:15 smithi086 ceph-mon[37128]: pgmap v454: 97 pgs: 97 active+clean; 211 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-07-24T21:49:15.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:15 smithi060 ceph-mon[33000]: pgmap v454: 97 pgs: 97 active+clean; 211 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-07-24T21:49:16.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:49:16.847 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:17.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:17 smithi060 ceph-mon[33000]: pgmap v455: 97 pgs: 97 active+clean; 212 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-07-24T21:49:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:17 smithi123 ceph-mon[37143]: pgmap v455: 97 pgs: 97 active+clean; 212 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-07-24T21:49:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:17 smithi086 ceph-mon[37128]: pgmap v455: 97 pgs: 97 active+clean; 212 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 -- 172.21.15.60:0/711877148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec107060 msgr2=0x7fe6ec1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 --2- 172.21.15.60:0/711877148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec107060 0x7fe6ec1074a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009900 tx=0x7fe6e002a410 comp rx=0 tx=0).stop 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 -- 172.21.15.60:0/711877148 shutdown_connections 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 --2- 172.21.15.60:0/711877148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 --2- 172.21.15.60:0/711877148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec107060 0x7fe6ec1074a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 --2- 172.21.15.60:0/711877148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec101650 0x7fe6ec101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 -- 172.21.15.60:0/711877148 >> 172.21.15.60:0/711877148 conn(0x7fe6ec0fd0c0 msgr2=0x7fe6ec0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 -- 172.21.15.60:0/711877148 shutdown_connections 2024-07-24T21:49:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.647+0000 7fe6f3608640 1 -- 172.21.15.60:0/711877148 wait complete. 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 Processor -- start 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 -- start start 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 0x7fe6ec198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec107060 0x7fe6ec199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112780 con 0x7fe6ec101650 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112600 con 0x7fe6ec107060 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f3608640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112900 con 0x7fe6ec1079e0 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 0x7fe6ec198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f1ba5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 0x7fe6ec198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53754/0 (socket says 172.21.15.60:53754) 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f13a4640 1 -- 172.21.15.60:0/572542139 learned_addr learned my addr 172.21.15.60:0/572542139 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.648+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec107060 0x7fe6ec199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 -- 172.21.15.60:0/572542139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec107060 msgr2=0x7fe6ec199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec107060 0x7fe6ec199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 -- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 msgr2=0x7fe6ec198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 0x7fe6ec198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 -- 172.21.15.60:0/572542139 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec19ed00 con 0x7fe6ec1079e0 2024-07-24T21:49:18.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f1ba5640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec19e530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e970 tx=0x7fe6e400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6ea7fc640 1 -- 172.21.15.60:0/572542139 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ccc0 con 0x7fe6ec1079e0 2024-07-24T21:49:18.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6ea7fc640 1 -- 172.21.15.60:0/572542139 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e4005020 con 0x7fe6ec1079e0 2024-07-24T21:49:18.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec19eff0 con 0x7fe6ec1079e0 2024-07-24T21:49:18.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6ea7fc640 1 -- 172.21.15.60:0/572542139 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4004a00 con 0x7fe6ec1079e0 2024-07-24T21:49:18.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.649+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec1a56b0 con 0x7fe6ec1079e0 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.652+0000 7fe6ea7fc640 1 -- 172.21.15.60:0/572542139 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe6e4004ba0 con 0x7fe6ec1079e0 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.652+0000 7fe6f59b9740 1 Processor -- start 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.652+0000 7fe6f59b9740 1 -- start start 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.652+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622217a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x55662222a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55662218e960 con 0x556622217fa0 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55662217de60 con 0x556622228340 2024-07-24T21:49:18.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556622181ea0 con 0x556622217680 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622217a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53760/0 (socket says 172.21.15.60:53760) 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f13a4640 1 -- 172.21.15.60:0/2093521189 learned_addr learned my addr 172.21.15.60:0/2093521189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x55662222a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6ea7fc640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 0x7fe6d40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/2093521189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 msgr2=0x55662222a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x55662222a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/2093521189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 msgr2=0x556622227e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/2093521189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566221072d0 con 0x556622217680 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f1ba5640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 0x7fe6d40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6ea7fc640 1 -- 172.21.15.60:0/572542139 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6e40e3eb0 con 0x7fe6ec1079e0 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.653+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622217a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009230 tx=0x7fe6e0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09d5472afb456a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6d37fe640 1 -- 172.21.15.60:0/2093521189 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0035070 con 0x556622217680 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6f1ba5640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 0x7fe6d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4018000 tx=0x7fe6e40e41b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6d37fe640 1 -- 172.21.15.60:0/2093521189 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e00035f0 con 0x556622217680 2024-07-24T21:49:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6d37fe640 1 -- 172.21.15.60:0/2093521189 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0032e60 con 0x556622217680 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6f59b9740 1 -- 172.21.15.60:0/2093521189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 msgr2=0x556622217a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.654+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622217a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009230 tx=0x7fe6e0005b50 comp rx=0 tx=0).stop 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 -- 172.21.15.60:0/2093521189 shutdown_connections 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x55662222a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622227e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/2093521189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622217a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 -- 172.21.15.60:0/2093521189 >> 172.21.15.60:0/2093521189 conn(0x5566222154b0 msgr2=0x556622216650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 -- 172.21.15.60:0/2093521189 shutdown_connections 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 -- 172.21.15.60:0/2093521189 wait complete. 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 -- start start 2024-07-24T21:49:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.655+0000 7fe6f59b9740 1 Processor -- start 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622225340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x556622225dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556621fe1700 con 0x556622217fa0 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566221205a0 con 0x556622228340 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55662222af90 con 0x556622217680 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f59b9740 10 client.0 _ll_register_callbacks cb 0x556622140910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53774/0 (socket says 172.21.15.60:53774) 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 -- 172.21.15.60:0/4252921354 learned_addr learned my addr 172.21.15.60:0/4252921354 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622225340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x556622225dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 -- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 msgr2=0x556622225340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622225340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 -- 172.21.15.60:0/4252921354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 msgr2=0x556622225dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x556622225dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 -- 172.21.15.60:0/4252921354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556622113630 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc011c30 tx=0x7fe6dc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b787f949a208b17 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.656+0000 7fe6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc01f060 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e00035f0 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6dc003b80 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc003d80 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 10 client.16239 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556621ede650 con 0x556622217fa0 2024-07-24T21:49:18.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e00035f0 con 0x556622217fa0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6dc01a610 con 0x556622217fa0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 10 client.16239 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 20 client.16239 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556622237ee0 0x55662223a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 4 client.16239 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.657+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566221908b0 con 0x556622237ee0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.658+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556622237ee0 0x55662223a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.658+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556622237ee0 0x55662223a2d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f5324f7fa1ebc6 server_cookie=db5c23db8eff2172 in_seq=0 out_seq=0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.658+0000 7fe6d0ff9640 10 client.16239 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/4252921354 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566221908b0 con 0x556622237ee0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6d0ff9640 10 client.16239 handle_command_reply: tid=0 2024-07-24T21:49:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 client.16239 shutdown 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 msgr2=0x556622225880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc011c30 tx=0x7fe6dc00f790 comp rx=0 tx=0).stop 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 shutdown_connections 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556622237ee0 0x55662223a2d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622228340 0x556622225dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622225880 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/4252921354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622217680 0x556622225340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 >> 172.21.15.60:0/4252921354 conn(0x5566222154b0 msgr2=0x55662221b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6d0ff9640 0 client.16239 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 shutdown_connections 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 1 -- 172.21.15.60:0/4252921354 wait complete. 2024-07-24T21:49:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.659+0000 7fe6f59b9740 20 client.16239 trim_cache size 0 max 16384 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.679+0000 7fe6f59b9740 1 Processor -- start 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.679+0000 7fe6f59b9740 1 -- start start 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622214d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622227080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1ae890 con 0x556622217fa0 2024-07-24T21:49:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1ac100 con 0x556622217680 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1a9f50 con 0x556622228340 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622214d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36582/0 (socket says 172.21.15.60:36582) 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/1963797324 learned_addr learned my addr 172.21.15.60:0/1963797324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622227080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/1963797324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 msgr2=0x556622227080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622227080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/1963797324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 msgr2=0x556622214d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622214d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 -- 172.21.15.60:0/1963797324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556622236a90 con 0x556622217680 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622214d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.680+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00057c0 tx=0x7fe6e0041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3debcd9f908eab5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6d2ffd640 1 -- 172.21.15.60:0/1963797324 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e005d070 con 0x556622217680 2024-07-24T21:49:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6d2ffd640 1 -- 172.21.15.60:0/1963797324 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e0031890 con 0x556622217680 2024-07-24T21:49:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6d2ffd640 1 -- 172.21.15.60:0/1963797324 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0035040 con 0x556622217680 2024-07-24T21:49:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 -- 172.21.15.60:0/1963797324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 msgr2=0x556622222dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00057c0 tx=0x7fe6e0041040 comp rx=0 tx=0).stop 2024-07-24T21:49:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 -- 172.21.15.60:0/1963797324 shutdown_connections 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622227080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622214d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/1963797324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x556622222dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 -- 172.21.15.60:0/1963797324 >> 172.21.15.60:0/1963797324 conn(0x5566222154b0 msgr2=0x556622226970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 -- 172.21.15.60:0/1963797324 shutdown_connections 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.681+0000 7fe6f59b9740 1 -- 172.21.15.60:0/1963797324 wait complete. 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 -- start start 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 Processor -- start 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x55662223fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622230220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1aa5c0 con 0x556622217fa0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1aba30 con 0x556622217680 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.682+0000 7fe6f59b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1aac90 con 0x556622228340 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f59b9740 10 client.0 _ll_register_callbacks cb 0x7fe6dc009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53794/0 (socket says 172.21.15.60:53794) 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 -- 172.21.15.60:0/939581662 learned_addr learned my addr 172.21.15.60:0/939581662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x55662223fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622230220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 -- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 msgr2=0x556622230220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622230220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 -- 172.21.15.60:0/939581662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 msgr2=0x55662223fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x55662223fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 -- 172.21.15.60:0/939581662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556622230990 con 0x556622217fa0 2024-07-24T21:49:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x55662223fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f13a4640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc01d040 tx=0x7fe6dc011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12543a4de62f1900 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc01a600 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe6dc017040 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.683+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e0031890 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc01a7a0 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 10 client.16245 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556622219670 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e0031890 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6dc02c460 con 0x556622217fa0 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 10 client.16245 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 20 client.16245 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5566222a2f80 0x5566222a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 4 client.16245 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55662218c350 con 0x5566222a2f80 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.684+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5566222a2f80 0x5566222a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f0ba3640 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5566222a2f80 0x5566222a53a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce19244270d6285 server_cookie=1470a0288a835bd3 in_seq=0 out_seq=0 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6d0ff9640 10 client.16245 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6d0ff9640 1 -- 172.21.15.60:0/939581662 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55662218c350 con 0x5566222a2f80 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6d0ff9640 10 client.16245 handle_command_reply: tid=0 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 client.16245 shutdown 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 msgr2=0x556622240350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc01d040 tx=0x7fe6dc011870 comp rx=0 tx=0).stop 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 shutdown_connections 2024-07-24T21:49:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5566222a2f80 0x5566222a53a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556622228340 0x556622230220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556622217fa0 0x556622240350 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 --2- 172.21.15.60:0/939581662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556622217680 0x55662223fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 >> 172.21.15.60:0/939581662 conn(0x5566222154b0 msgr2=0x556622227190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6d0ff9640 0 client.16245 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 shutdown_connections 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.685+0000 7fe6f59b9740 1 -- 172.21.15.60:0/939581662 wait complete. 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f59b9740 20 client.16245 trim_cache size 0 max 16384 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:49:18.692 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 msgr2=0x7fe6d40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 0x7fe6d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4018000 tx=0x7fe6e40e41b0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 msgr2=0x7fe6ec19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec19e530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e970 tx=0x7fe6e400ee30 comp rx=0 tx=0).stop 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 shutdown_connections 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe6d40b3dc0 0x7fe6d40b6280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ec1079e0 0x7fe6ec19e530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec107060 0x7fe6ec199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 --2- 172.21.15.60:0/572542139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ec101650 0x7fe6ec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 >> 172.21.15.60:0/572542139 conn(0x7fe6ec0fd0c0 msgr2=0x7fe6ec0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 shutdown_connections 2024-07-24T21:49:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:18.686+0000 7fe6f3608640 1 -- 172.21.15.60:0/572542139 wait complete. 2024-07-24T21:49:19.634 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:49:19.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:49:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:19 smithi123 ceph-mon[37143]: pgmap v456: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 386 op/s 2024-07-24T21:49:19.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:19 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:49:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:19 smithi086 ceph-mon[37128]: pgmap v456: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 386 op/s 2024-07-24T21:49:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:19 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:49:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:19 smithi060 ceph-mon[33000]: pgmap v456: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 386 op/s 2024-07-24T21:49:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:19 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:49:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:49:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:49:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:49:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:49:20.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:49:20.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:49:20.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:49:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:49:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:49:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:21 smithi123 ceph-mon[37143]: pgmap v457: 97 pgs: 97 active+clean; 216 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-07-24T21:49:21.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:21 smithi060 ceph-mon[33000]: pgmap v457: 97 pgs: 97 active+clean; 216 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-07-24T21:49:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:21 smithi086 ceph-mon[37128]: pgmap v457: 97 pgs: 97 active+clean; 216 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-07-24T21:49:21.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 -- 172.21.15.60:0/1659441575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d2410cac0 msgr2=0x7f6d2410ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/1659441575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d2410cac0 0x7f6d2410ef10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009860 tx=0x7f6d0c02a340 comp rx=0 tx=0).stop 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 -- 172.21.15.60:0/1659441575 shutdown_connections 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/1659441575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d2410cac0 0x7f6d2410ef10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/1659441575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d24103f10 0x7f6d2410c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/1659441575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 0x7f6d241038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 -- 172.21.15.60:0/1659441575 >> 172.21.15.60:0/1659441575 conn(0x7f6d240fd0c0 msgr2=0x7f6d240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 -- 172.21.15.60:0/1659441575 shutdown_connections 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.886+0000 7f6d2adcc640 1 -- 172.21.15.60:0/1659441575 wait complete. 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 Processor -- start 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 -- start start 2024-07-24T21:49:21.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 0x7f6d2419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d2410cac0 0x7f6d241a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24112700 con 0x7f6d2410cac0 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24112580 con 0x7f6d24103510 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d2adcc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24112880 con 0x7f6d24103f10 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d23fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d28b68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 0x7f6d2419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d23fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49248/0 (socket says 172.21.15.60:49248) 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d23fff640 1 -- 172.21.15.60:0/3071970535 learned_addr learned my addr 172.21.15.60:0/3071970535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.887+0000 7f6d29369640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d2410cac0 0x7f6d241a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 -- 172.21.15.60:0/3071970535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 msgr2=0x7f6d2419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 0x7f6d2419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 -- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d2410cac0 msgr2=0x7f6d241a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d2410cac0 0x7f6d241a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 -- 172.21.15.60:0/3071970535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d241a0eb0 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d23fff640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400c420 tx=0x7f6d14009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14003970 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d14004020 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400b980 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d241a11a0 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.888+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d241a7a30 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.889+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2406aaf0 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.891+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6d1401d020 con 0x7f6d24103f10 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.891+0000 7f6d21ffb640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 0x7f6d100b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.892+0000 7f6d28b68640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 0x7f6d100b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.892+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6d140e3e00 con 0x7f6d24103f10 2024-07-24T21:49:21.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.892+0000 7f6d28b68640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 0x7f6d100b6280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009bd0 tx=0x7f6d1c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:21.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:21.896+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f6d140ab110 con 0x7f6d24103f10 2024-07-24T21:49:22.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.241+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d241a7c00 con 0x7f6d24103f10 2024-07-24T21:49:22.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.242+0000 7f6d21ffb640 1 -- 172.21.15.60:0/3071970535 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f6d140b33e0 con 0x7f6d24103f10 2024-07-24T21:49:22.243 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:49:22.244 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"btime":"2024-07-24T21:48:44:369573+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:48:44.225462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:49:22.244 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 msgr2=0x7f6d100b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 0x7f6d100b6280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009bd0 tx=0x7f6d1c005f90 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 msgr2=0x7f6d2419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400c420 tx=0x7f6d14009f90 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 shutdown_connections 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d100b3dc0 0x7f6d100b6280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d2410cac0 0x7f6d241a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d24103f10 0x7f6d2419b340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 --2- 172.21.15.60:0/3071970535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d24103510 0x7f6d2419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.244+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 >> 172.21.15.60:0/3071970535 conn(0x7f6d240fd0c0 msgr2=0x7f6d240ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:22.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.245+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 shutdown_connections 2024-07-24T21:49:22.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:22.245+0000 7f6d2adcc640 1 -- 172.21.15.60:0/3071970535 wait complete. 2024-07-24T21:49:22.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3071970535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:22.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3071970535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:22.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3071970535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:23.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:49:23.527 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:23.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:23 smithi123 ceph-mon[37143]: pgmap v458: 97 pgs: 97 active+clean; 221 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2024-07-24T21:49:23.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:23 smithi060 ceph-mon[33000]: pgmap v458: 97 pgs: 97 active+clean; 221 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2024-07-24T21:49:23.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:23 smithi086 ceph-mon[37128]: pgmap v458: 97 pgs: 97 active+clean; 221 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2024-07-24T21:49:24.708 INFO:tasks.workunit.client.0.smithi060.stdout: 111 345559 1686 235129 1370 86578 732 2024-07-24T21:49:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 -- 172.21.15.60:0/1803305681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 msgr2=0x7f88d8106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 --2- 172.21.15.60:0/1803305681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d8106c50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00aa10 tx=0x7f88cc02b720 comp rx=0 tx=0).stop 2024-07-24T21:49:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 -- 172.21.15.60:0/1803305681 shutdown_connections 2024-07-24T21:49:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 --2- 172.21.15.60:0/1803305681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 0x7f88d810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 --2- 172.21.15.60:0/1803305681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d8106c50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 --2- 172.21.15.60:0/1803305681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 -- 172.21.15.60:0/1803305681 >> 172.21.15.60:0/1803305681 conn(0x7f88d8076940 msgr2=0x7f88d8076d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 -- 172.21.15.60:0/1803305681 shutdown_connections 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.516+0000 7f88dd08f640 1 -- 172.21.15.60:0/1803305681 wait complete. 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.517+0000 7f88dd08f640 1 Processor -- start 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.517+0000 7f88dd08f640 1 -- start start 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.517+0000 7f88dd08f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d80fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.517+0000 7f88dd08f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.517+0000 7f88dd08f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 0x7f88d80feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88dd08f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8110560 con 0x7f88d8106810 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88dd08f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81103e0 con 0x7f88d81011e0 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88dd08f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81106e0 con 0x7f88d8107190 2024-07-24T21:49:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53832/0 (socket says 172.21.15.60:53832) 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 -- 172.21.15.60:0/33502654 learned_addr learned my addr 172.21.15.60:0/33502654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d80fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d759e640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 0x7f88d80feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 -- 172.21.15.60:0/33502654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 msgr2=0x7f88d80feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 0x7f88d80feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 -- 172.21.15.60:0/33502654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 msgr2=0x7f88d80fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d80fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 -- 172.21.15.60:0/33502654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d81a70d0 con 0x7f88d8106810 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88d659c640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f88cc005940 tx=0x7f88cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88bffff640 1 -- 172.21.15.60:0/33502654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc03f070 con 0x7f88d8106810 2024-07-24T21:49:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88bffff640 1 -- 172.21.15.60:0/33502654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc003ca0 con 0x7f88d8106810 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d81a7360 con 0x7f88d8106810 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88bffff640 1 -- 172.21.15.60:0/33502654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc036040 con 0x7f88d8106810 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.518+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d81a7840 con 0x7f88d8106810 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88bffff640 1 -- 172.21.15.60:0/33502654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f88cc042070 con 0x7f88d8106810 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 Processor -- start 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 -- start start 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5561ac45c9b0 con 0x5561ac4f1f80 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561ac44beb0 con 0x5561ac4e56d0 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88df440740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561ac44fef0 con 0x5561ac4e5ff0 2024-07-24T21:49:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88d759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.521+0000 7f88d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36622/0 (socket says 172.21.15.60:36622) 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 -- 172.21.15.60:0/2073433408 learned_addr learned my addr 172.21.15.60:0/2073433408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 -- 172.21.15.60:0/2073433408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 msgr2=0x5561ac4f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 -- 172.21.15.60:0/2073433408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 msgr2=0x5561ac4f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 -- 172.21.15.60:0/2073433408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ac3d5320 con 0x5561ac4e56d0 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88bffff640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 0x7f88c00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d759e640 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009f80 tx=0x7f88d0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a14f56243a0b261 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 0x7f88c00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88bffff640 1 -- 172.21.15.60:0/33502654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f88cc1048e0 con 0x7f88d8106810 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88bd7fa640 1 -- 172.21.15.60:0/2073433408 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0003b70 con 0x5561ac4e56d0 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88bd7fa640 1 -- 172.21.15.60:0/2073433408 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d0003d10 con 0x5561ac4e56d0 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.522+0000 7f88bd7fa640 1 -- 172.21.15.60:0/2073433408 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00119c0 con 0x5561ac4e56d0 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 -- 172.21.15.60:0/2073433408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 msgr2=0x5561ac4e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009f80 tx=0x7f88d0010860 comp rx=0 tx=0).stop 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 0x7f88c00b6200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5561ac4e43e0 tx=0x7f88c8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 -- 172.21.15.60:0/2073433408 shutdown_connections 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 --2- 172.21.15.60:0/2073433408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4e5ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 -- 172.21.15.60:0/2073433408 >> 172.21.15.60:0/2073433408 conn(0x5561ac4e3500 msgr2=0x5561ac4e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 -- 172.21.15.60:0/2073433408 shutdown_connections 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.523+0000 7f88df440740 1 -- 172.21.15.60:0/2073433408 wait complete. 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 -- start start 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 Processor -- start 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac4ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5561ac2af700 con 0x5561ac4e56d0 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561ac3ee5f0 con 0x5561ac4e5ff0 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561ac4f4c00 con 0x5561ac4f1f80 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88df440740 10 client.0 _ll_register_callbacks cb 0x5561ac40e960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53854/0 (socket says 172.21.15.60:53854) 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d6d9d640 1 -- 172.21.15.60:0/1649597838 learned_addr learned my addr 172.21.15.60:0/1649597838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:25.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d659c640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac4ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 -- 172.21.15.60:0/1649597838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 msgr2=0x5561ac4ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac4ef4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 -- 172.21.15.60:0/1649597838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 msgr2=0x5561ac4eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d759e640 1 -- 172.21.15.60:0/1649597838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ac3e1680 con 0x5561ac4f1f80 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.524+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88d759e640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4ef9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88d0011010 tx=0x7f88d0004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135dfe9532722b96 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88baff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d002a060 con 0x5561ac4f1f80 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d0003d10 con 0x5561ac4f1f80 2024-07-24T21:49:25.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f88d0011b50 con 0x5561ac4f1f80 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88d0011d50 con 0x5561ac4f1f80 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88df440740 10 client.25759 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561ac1ac650 con 0x5561ac4f1f80 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.525+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d0003d10 con 0x5561ac4f1f80 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88d00050d0 con 0x5561ac4f1f80 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 10 client.25759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 20 client.25759 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac5019c0 0x5561ac503db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 4 client.25759 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ac45e900 con 0x5561ac5019c0 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac5019c0 0x5561ac503db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88baff5640 10 client.25759 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.526+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac5019c0 0x5561ac503db0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f4b8b3b1416c7fd server_cookie=66855623dda1c8d in_seq=0 out_seq=0 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88baff5640 1 -- 172.21.15.60:0/1649597838 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561ac45e900 con 0x5561ac5019c0 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88baff5640 10 client.25759 handle_command_reply: tid=0 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 client.25759 shutdown 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 msgr2=0x5561ac4ef9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4ef9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88d0011010 tx=0x7f88d0004cf0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 shutdown_connections 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac5019c0 0x5561ac503db0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4ef9e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88baff5640 0 client.25759 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac4ef4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 --2- 172.21.15.60:0/1649597838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac4eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 >> 172.21.15.60:0/1649597838 conn(0x5561ac4e3500 msgr2=0x5561ac4e90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 shutdown_connections 2024-07-24T21:49:25.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 1 -- 172.21.15.60:0/1649597838 wait complete. 2024-07-24T21:49:25.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.527+0000 7f88df440740 20 client.25759 trim_cache size 0 max 16384 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.543+0000 7f88df440740 1 Processor -- start 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.543+0000 7f88df440740 1 -- start start 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac5098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81b2c80 con 0x5561ac4e56d0 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81b04f0 con 0x5561ac4e5ff0 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88df440740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81ae340 con 0x5561ac4f1f80 2024-07-24T21:49:25.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53860/0 (socket says 172.21.15.60:53860) 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 -- 172.21.15.60:0/3451874377 learned_addr learned my addr 172.21.15.60:0/3451874377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d659c640 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac5098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 -- 172.21.15.60:0/3451874377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 msgr2=0x5561ac4eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 -- 172.21.15.60:0/3451874377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 msgr2=0x5561ac5098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac5098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 -- 172.21.15.60:0/3451874377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ac4e32f0 con 0x5561ac4e56d0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d659c640 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac5098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f88c8012e50 tx=0x7f88c801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8617f3050449e1f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.544+0000 7f88bcff9640 1 -- 172.21.15.60:0/3451874377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c802e070 con 0x5561ac4e56d0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88bcff9640 1 -- 172.21.15.60:0/3451874377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c801dd90 con 0x5561ac4e56d0 2024-07-24T21:49:25.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88bcff9640 1 -- 172.21.15.60:0/3451874377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8004a00 con 0x5561ac4e56d0 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 -- 172.21.15.60:0/3451874377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 msgr2=0x5561ac509380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f88c8012e50 tx=0x7f88c801b000 comp rx=0 tx=0).stop 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 -- 172.21.15.60:0/3451874377 shutdown_connections 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4f1f80 0x5561ac4eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e5ff0 0x5561ac5098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 --2- 172.21.15.60:0/3451874377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4e56d0 0x5561ac509380 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 -- 172.21.15.60:0/3451874377 >> 172.21.15.60:0/3451874377 conn(0x5561ac4e3500 msgr2=0x5561ac4f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 -- 172.21.15.60:0/3451874377 shutdown_connections 2024-07-24T21:49:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.545+0000 7f88df440740 1 -- 172.21.15.60:0/3451874377 wait complete. 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 -- start start 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 Processor -- start 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81ae9b0 con 0x5561ac4f1f80 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81afe20 con 0x5561ac4e56d0 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81af080 con 0x5561ac4e5ff0 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88df440740 10 client.0 _ll_register_callbacks cb 0x7f88cc10a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88d759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88d759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53866/0 (socket says 172.21.15.60:53866) 2024-07-24T21:49:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88d759e640 1 -- 172.21.15.60:0/1546573581 learned_addr learned my addr 172.21.15.60:0/1546573581 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.546+0000 7f88d659c640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 -- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 msgr2=0x5561ac4fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 -- 172.21.15.60:0/1546573581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 msgr2=0x5561ac4fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 -- 172.21.15.60:0/1546573581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ac4ebfb0 con 0x5561ac4f1f80 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88d759e640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f88d0005520 tx=0x7f88d0011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8ccfee2d1c0bd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88baff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d002a070 con 0x5561ac4f1f80 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f88d0011950 con 0x5561ac4f1f80 2024-07-24T21:49:25.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c801dd90 con 0x5561ac4f1f80 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88d0043050 con 0x5561ac4f1f80 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88df440740 10 client.16269 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.547+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561ac56be10 con 0x5561ac4f1f80 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c801dd90 con 0x5561ac4f1f80 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88d0025460 con 0x5561ac4f1f80 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 10 client.16269 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 10 client.16269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 20 client.16269 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac56c550 0x5561ac56e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 4 client.16269 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561ac45a3a0 con 0x5561ac56c550 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.548+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac56c550 0x5561ac56e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:25.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88baff5640 10 client.16269 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88d6d9d640 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac56c550 0x5561ac56e970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd737194968187b7 server_cookie=9b492ffcd6b8c0f6 in_seq=0 out_seq=0 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88baff5640 1 -- 172.21.15.60:0/1546573581 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561ac45a3a0 con 0x5561ac56c550 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88baff5640 10 client.16269 handle_command_reply: tid=0 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 client.16269 shutdown 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 msgr2=0x5561ac4eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f88d0005520 tx=0x7f88d0011ef0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 shutdown_connections 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5561ac56c550 0x5561ac56e970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88baff5640 0 client.16269 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561ac4f1f80 0x5561ac4eb780 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561ac4e5ff0 0x5561ac4fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 --2- 172.21.15.60:0/1546573581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561ac4e56d0 0x5561ac4fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 >> 172.21.15.60:0/1546573581 conn(0x5561ac4e3500 msgr2=0x5561ac4eeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 shutdown_connections 2024-07-24T21:49:25.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 1 -- 172.21.15.60:0/1546573581 wait complete. 2024-07-24T21:49:25.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.549+0000 7f88df440740 20 client.16269 trim_cache size 0 max 16384 2024-07-24T21:49:25.564 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 msgr2=0x7f88c00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 0x7f88c00b6200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5561ac4e43e0 tx=0x7f88c8005f90 comp rx=0 tx=0).stop 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 msgr2=0x7f88d80fe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f88cc005940 tx=0x7f88cc004270 comp rx=0 tx=0).stop 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 shutdown_connections 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88c00b3d40 0x7f88c00b6200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88d8107190 0x7f88d80feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88d8106810 0x7f88d80fe5c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 --2- 172.21.15.60:0/33502654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88d81011e0 0x7f88d80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:25.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 >> 172.21.15.60:0/33502654 conn(0x7f88d8076940 msgr2=0x7f88d8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:25.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 shutdown_connections 2024-07-24T21:49:25.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:25.550+0000 7f88dd08f640 1 -- 172.21.15.60:0/33502654 wait complete. 2024-07-24T21:49:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:25 smithi123 ceph-mon[37143]: pgmap v459: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-07-24T21:49:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:25 smithi086 ceph-mon[37128]: pgmap v459: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-07-24T21:49:25.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:25 smithi060 ceph-mon[33000]: pgmap v459: 97 pgs: 97 active+clean; 222 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.315+0000 7f4746d32640 1 -- 172.21.15.60:0/1264155129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740106f90 msgr2=0x7f47401073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.315+0000 7f4746d32640 1 --2- 172.21.15.60:0/1264155129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740106f90 0x7f47401073d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f472c009900 tx=0x7f472c02a410 comp rx=0 tx=0).stop 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 -- 172.21.15.60:0/1264155129 shutdown_connections 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 --2- 172.21.15.60:0/1264155129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4740107910 0x7f474010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 --2- 172.21.15.60:0/1264155129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740106f90 0x7f47401073d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 --2- 172.21.15.60:0/1264155129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47401015e0 0x7f47401019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 -- 172.21.15.60:0/1264155129 >> 172.21.15.60:0/1264155129 conn(0x7f47400782e0 msgr2=0x7f47400ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 -- 172.21.15.60:0/1264155129 shutdown_connections 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 -- 172.21.15.60:0/1264155129 wait complete. 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 Processor -- start 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.316+0000 7f4746d32640 1 -- start start 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 0x7f474019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 0x7f474019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4740111580 con 0x7f4740107910 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4740111400 con 0x7f47401015e0 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4746d32640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4740111700 con 0x7f4740106f90 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f47452cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f47452cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48848/0 (socket says 172.21.15.60:48848) 2024-07-24T21:49:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f47452cf640 1 -- 172.21.15.60:0/2139957956 learned_addr learned my addr 172.21.15.60:0/2139957956 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 0x7f474019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4744ace640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 0x7f474019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 -- 172.21.15.60:0/2139957956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 msgr2=0x7f474019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 0x7f474019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 -- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 msgr2=0x7f47401a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 -- 172.21.15.60:0/2139957956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47401a0e70 con 0x7f4740106f90 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f47452cf640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f4744ace640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 0x7f474019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.317+0000 7f473ffff640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 0x7f474019b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f472c02aa00 tx=0x7f472c003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.318+0000 7f473dffb640 1 -- 172.21.15.60:0/2139957956 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c03f070 con 0x7f4740106f90 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.318+0000 7f473dffb640 1 -- 172.21.15.60:0/2139957956 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f472c031d60 con 0x7f4740106f90 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.318+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47401a1100 con 0x7f4740106f90 2024-07-24T21:49:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.318+0000 7f473dffb640 1 -- 172.21.15.60:0/2139957956 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c035030 con 0x7f4740106f90 2024-07-24T21:49:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.318+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47401a7820 con 0x7f4740106f90 2024-07-24T21:49:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.321+0000 7f473dffb640 1 -- 172.21.15.60:0/2139957956 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f472c042070 con 0x7f4740106f90 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f473dffb640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 0x7f47300b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4744ace640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 0x7f47300b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f473dffb640 1 -- 172.21.15.60:0/2139957956 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f472c1046b0 con 0x7f4740106f90 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 Processor -- start 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 -- start start 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4744ace640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 0x7f47300b6200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f473400bfd0 tx=0x7f473400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 0x563d52c11560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563d52b78280 con 0x563d52c031e0 2024-07-24T21:49:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d52b67780 con 0x563d52c0f170 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.322+0000 7f4749b65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d52b6b7c0 con 0x563d52c028c0 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48862/0 (socket says 172.21.15.60:48862) 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 -- 172.21.15.60:0/2234778671 learned_addr learned my addr 172.21.15.60:0/2234778671 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f4744ace640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f47452cf640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 0x563d52c11560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 -- 172.21.15.60:0/2234778671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 msgr2=0x563d52c02ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c02ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 -- 172.21.15.60:0/2234778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 msgr2=0x563d52c11560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 0x563d52c11560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 -- 172.21.15.60:0/2234778671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d528cb790 con 0x563d52c031e0 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f4744ace640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f47452cf640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 0x563d52c11560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f473ffff640 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f472c0cb300 tx=0x7f472c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e950dd2fd3721a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f472affd640 1 -- 172.21.15.60:0/2234778671 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c049070 con 0x563d52c031e0 2024-07-24T21:49:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f472affd640 1 -- 172.21.15.60:0/2234778671 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f472c0cf8c0 con 0x563d52c031e0 2024-07-24T21:49:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.323+0000 7f472affd640 1 -- 172.21.15.60:0/2234778671 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0cb830 con 0x563d52c031e0 2024-07-24T21:49:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.324+0000 7f4749b65740 1 -- 172.21.15.60:0/2234778671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 msgr2=0x563d52c0ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.324+0000 7f4749b65740 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f472c0cb300 tx=0x7f472c031f10 comp rx=0 tx=0).stop 2024-07-24T21:49:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 -- 172.21.15.60:0/2234778671 shutdown_connections 2024-07-24T21:49:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c0f170 0x563d52c11560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c031e0 0x563d52c0ec30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 --2- 172.21.15.60:0/2234778671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 -- 172.21.15.60:0/2234778671 >> 172.21.15.60:0/2234778671 conn(0x563d52c006f0 msgr2=0x563d52c01890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 -- 172.21.15.60:0/2234778671 shutdown_connections 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 -- 172.21.15.60:0/2234778671 wait complete. 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 -- start start 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.325+0000 7f4749b65740 1 Processor -- start 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563d52aa1ab0 con 0x563d52c028c0 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d52aef9e0 con 0x563d52c031e0 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d52c11c10 con 0x563d52c0f170 2024-07-24T21:49:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4749b65740 10 client.0 _ll_register_callbacks cb 0x563d52877a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4744ace640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4744ace640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48864/0 (socket says 172.21.15.60:48864) 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f4744ace640 1 -- 172.21.15.60:0/3105918286 learned_addr learned my addr 172.21.15.60:0/3105918286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f473ffff640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.326+0000 7f47452cf640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 -- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 msgr2=0x563d52c0c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c0c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 -- 172.21.15.60:0/3105918286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 msgr2=0x563d52c0be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 -- 172.21.15.60:0/3105918286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d52944620 con 0x563d52c028c0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f473ffff640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4744ace640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4734011ab0 tx=0x7f4734004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a7208a51f702e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f471ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473401ec00 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f473401eda0 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d52b16ce0 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4734005020 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4749b65740 10 client.16281 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.327+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d52c1d350 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d52b16ce0 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f473401f430 con 0x563d52c028c0 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 20 client.16281 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c1e810 0x563d52c20c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 4 client.16281 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d52b7a1d0 con 0x563d52c1e810 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.328+0000 7f473ffff640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c1e810 0x563d52c20c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.329+0000 7f471ffff640 10 client.16281 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.329+0000 7f473ffff640 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c1e810 0x563d52c20c70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad49ff1f74f39c8c server_cookie=fa6b694b59f1e554 in_seq=0 out_seq=0 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.329+0000 7f471ffff640 1 -- 172.21.15.60:0/3105918286 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d52b7a1d0 con 0x563d52c1e810 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.329+0000 7f471ffff640 10 client.16281 handle_command_reply: tid=0 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.329+0000 7f4749b65740 1 client.16281 shutdown 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 msgr2=0x563d52c0e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4734011ab0 tx=0x7f4734004730 comp rx=0 tx=0).stop 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 shutdown_connections 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c1e810 0x563d52c20c70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f471ffff640 0 client.16281 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 --2- 172.21.15.60:0/3105918286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c0e920 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 >> 172.21.15.60:0/3105918286 conn(0x563d52c006f0 msgr2=0x563d52c06290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 shutdown_connections 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 1 -- 172.21.15.60:0/3105918286 wait complete. 2024-07-24T21:49:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.330+0000 7f4749b65740 20 client.16281 trim_cache size 0 max 16384 2024-07-24T21:49:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 Processor -- start 2024-07-24T21:49:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 -- start start 2024-07-24T21:49:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47401b2bd0 con 0x563d52c028c0 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47401b0440 con 0x563d52c031e0 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4749b65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47401ae290 con 0x563d52c0f170 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48874/0 (socket says 172.21.15.60:48874) 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 -- 172.21.15.60:0/4064793159 learned_addr learned my addr 172.21.15.60:0/4064793159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f47452cf640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f473ffff640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 -- 172.21.15.60:0/4064793159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 msgr2=0x563d52c26580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c26580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 -- 172.21.15.60:0/4064793159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 msgr2=0x563d52c0e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f4744ace640 1 -- 172.21.15.60:0/4064793159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d52bfac00 con 0x563d52c028c0 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f473ffff640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.346+0000 7f47452cf640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.347+0000 7f4744ace640 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f473400bcb0 tx=0x7f473401fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584b455a0a64620 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.347+0000 7f472a7fc640 1 -- 172.21.15.60:0/4064793159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734035040 con 0x563d52c028c0 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.347+0000 7f472a7fc640 1 -- 172.21.15.60:0/4064793159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473401f430 con 0x563d52c028c0 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.347+0000 7f472a7fc640 1 -- 172.21.15.60:0/4064793159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473401ede0 con 0x563d52c028c0 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- 172.21.15.60:0/4064793159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 msgr2=0x563d52c1ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f473400bcb0 tx=0x7f473401fe90 comp rx=0 tx=0).stop 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- 172.21.15.60:0/4064793159 shutdown_connections 2024-07-24T21:49:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c0f170 0x563d52c26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c0e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 --2- 172.21.15.60:0/4064793159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c028c0 0x563d52c1ac60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- 172.21.15.60:0/4064793159 >> 172.21.15.60:0/4064793159 conn(0x563d52c006f0 msgr2=0x563d52c0a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- 172.21.15.60:0/4064793159 shutdown_connections 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- 172.21.15.60:0/4064793159 wait complete. 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 -- start start 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 Processor -- start 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.348+0000 7f4749b65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c08960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47401ae900 con 0x563d52c0f170 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47401afd70 con 0x563d52c031e0 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47401aefd0 con 0x563d52c028c0 2024-07-24T21:49:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4749b65740 10 client.0 _ll_register_callbacks cb 0x7f473400bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f47452cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f473ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c08960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f47452cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48890/0 (socket says 172.21.15.60:48890) 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f473ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c08960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52958/0 (socket says 172.21.15.60:52958) 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f47452cf640 1 -- 172.21.15.60:0/803185954 learned_addr learned my addr 172.21.15.60:0/803185954 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 -- 172.21.15.60:0/803185954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 msgr2=0x563d52c08960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c08960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 -- 172.21.15.60:0/803185954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 msgr2=0x563d52c08ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 -- 172.21.15.60:0/803185954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d52c876f0 con 0x563d52c028c0 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f47452cf640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f4744ace640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c08420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f473401f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31fc58a694872ff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.349+0000 7f471ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734046020 con 0x563d52c028c0 2024-07-24T21:49:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473401f430 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4734004230 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4734004450 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 10 client.25780 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d52c88a00 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473401f430 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4734004730 con 0x563d52c028c0 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 10 client.25780 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 20 client.25780 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c89140 0x563d52c8b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 4 client.25780 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.350+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d52c0ecb0 con 0x563d52c89140 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.351+0000 7f473ffff640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c89140 0x563d52c8b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.351+0000 7f471ffff640 10 client.25780 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:26.351+0000 7f473ffff640 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c89140 0x563d52c8b560 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c8b009e81ab929 server_cookie=55ed985deba783e9 in_seq=0 out_seq=0 2024-07-24T21:49:26.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:49:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:27.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:27 smithi060 ceph-mon[33000]: pgmap v460: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-07-24T21:49:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:27 smithi123 ceph-mon[37143]: pgmap v460: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-07-24T21:49:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:27 smithi086 ceph-mon[37128]: pgmap v460: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 327 op/s 2024-07-24T21:49:29.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 -- 172.21.15.60:0/1129882155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 msgr2=0x7efdd4102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 --2- 172.21.15.60:0/1129882155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4102b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009900 tx=0x7efdd002a410 comp rx=0 tx=0).stop 2024-07-24T21:49:29.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 -- 172.21.15.60:0/1129882155 shutdown_connections 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 --2- 172.21.15.60:0/1129882155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 0x7efdd410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 --2- 172.21.15.60:0/1129882155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 0x7efdd41085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 --2- 172.21.15.60:0/1129882155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4102b30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 -- 172.21.15.60:0/1129882155 >> 172.21.15.60:0/1129882155 conn(0x7efdd40fe200 msgr2=0x7efdd4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 -- 172.21.15.60:0/1129882155 shutdown_connections 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.246+0000 7efddbda7640 1 -- 172.21.15.60:0/1129882155 wait complete. 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 Processor -- start 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 -- start start 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 0x7efdd40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 0x7efdd4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4112760 con 0x7efdd4108160 2024-07-24T21:49:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41125e0 con 0x7efdd4102750 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efddbda7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41128e0 con 0x7efdd4108ae0 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efdd9342640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 0x7efdd40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52966/0 (socket says 172.21.15.60:52966) 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efdd9b43640 1 -- 172.21.15.60:0/266405998 learned_addr learned my addr 172.21.15.60:0/266405998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.247+0000 7efdda344640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 0x7efdd4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 -- 172.21.15.60:0/266405998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 msgr2=0x7efdd4075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 0x7efdd4075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 -- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 msgr2=0x7efdd40753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 0x7efdd40753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 -- 172.21.15.60:0/266405998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd41a7150 con 0x7efdd4102750 2024-07-24T21:49:29.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdd9b43640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005ab0 tx=0x7efdd0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdcaffd640 1 -- 172.21.15.60:0/266405998 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd003f070 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdcaffd640 1 -- 172.21.15.60:0/266405998 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd0031b60 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efdcaffd640 1 -- 172.21.15.60:0/266405998 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0035030 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd41a73e0 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.248+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41a7870 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 Processor -- start 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 -- start start 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdcaffd640 1 -- 172.21.15.60:0/266405998 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7efdd0042070 con 0x7efdd4102750 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd162a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fd15929b0 con 0x557fd1627f80 2024-07-24T21:49:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557fd1581eb0 con 0x557fd161bff0 2024-07-24T21:49:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdde158740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557fd1585ef0 con 0x557fd161b6d0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52972/0 (socket says 172.21.15.60:52972) 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdd9b43640 1 -- 172.21.15.60:0/2664702332 learned_addr learned my addr 172.21.15.60:0/2664702332 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdd9342640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd162a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.252+0000 7efdda344640 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 -- 172.21.15.60:0/2664702332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 msgr2=0x557fd161bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 -- 172.21.15.60:0/2664702332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 msgr2=0x557fd162a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd162a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 -- 172.21.15.60:0/2664702332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd150b320 con 0x557fd161bff0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdd0101c90 tx=0x7efdd010acd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ab6cb548305dec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdcaffd640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 0x7efdc00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdda344640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 0x7efdc00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.253+0000 7efdcaffd640 1 -- 172.21.15.60:0/266405998 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efdd01045d0 con 0x7efdd4102750 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdbffff640 1 -- 172.21.15.60:0/2664702332 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0104c90 con 0x557fd161bff0 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdbffff640 1 -- 172.21.15.60:0/2664702332 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd0114020 con 0x557fd161bff0 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdbffff640 1 -- 172.21.15.60:0/2664702332 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd010e9a0 con 0x557fd161bff0 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 -- 172.21.15.60:0/2664702332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 msgr2=0x557fd1627a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdd0101c90 tx=0x7efdd010acd0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdda344640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 0x7efdc00b6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdd4076f50 tx=0x7efdcc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 -- 172.21.15.60:0/2664702332 shutdown_connections 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd162a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd1627a40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 --2- 172.21.15.60:0/2664702332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 -- 172.21.15.60:0/2664702332 >> 172.21.15.60:0/2664702332 conn(0x557fd1619500 msgr2=0x557fd161a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 -- 172.21.15.60:0/2664702332 shutdown_connections 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.254+0000 7efdde158740 1 -- 172.21.15.60:0/2664702332 wait complete. 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 -- start start 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 Processor -- start 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1625450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1625990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fd13e5700 con 0x557fd1627f80 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557fd15245f0 con 0x557fd161b6d0 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557fd16331d0 con 0x557fd161bff0 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdde158740 10 client.0 _ll_register_callbacks cb 0x557fd1544960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52978/0 (socket says 172.21.15.60:52978) 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdd9b43640 1 -- 172.21.15.60:0/2329945366 learned_addr learned my addr 172.21.15.60:0/2329945366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.255+0000 7efdda344640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1625990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9342640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1625450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 -- 172.21.15.60:0/2329945366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 msgr2=0x557fd1625450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1625450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 -- 172.21.15.60:0/2329945366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 msgr2=0x557fd1625990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1625990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 -- 172.21.15.60:0/2329945366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd1517680 con 0x557fd161b6d0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9342640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1625450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdd9b43640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd00c7260 tx=0x7efdd003b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c055a38d6b75c388 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0113030 con 0x557fd161b6d0 2024-07-24T21:49:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efdd0101fa0 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd0114020 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdd00c70a0 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdde158740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fd12e2650 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.256+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd0114020 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efdd0104c90 con 0x557fd161b6d0 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 10 client.25781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 20 client.25781 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd163fe20 0x557fd1642210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 4 client.25781 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:29.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fd1594900 con 0x557fd163fe20 2024-07-24T21:49:29.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdd9342640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd163fe20 0x557fd1642210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdd9342640 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd163fe20 0x557fd1642210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f6176bb11efbd8c server_cookie=13fcb666186341a3 in_seq=0 out_seq=0 2024-07-24T21:49:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.257+0000 7efdbd7fa640 10 client.25781 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdbd7fa640 1 -- 172.21.15.60:0/2329945366 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fd1594900 con 0x557fd163fe20 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdbd7fa640 10 client.25781 handle_command_reply: tid=0 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 client.25781 shutdown 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 msgr2=0x557fd1624f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd00c7260 tx=0x7efdd003b800 comp rx=0 tx=0).stop 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 shutdown_connections 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd163fe20 0x557fd1642210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1625990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1625450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 --2- 172.21.15.60:0/2329945366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161b6d0 0x557fd1624f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdbd7fa640 0 client.25781 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:49:29.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 >> 172.21.15.60:0/2329945366 conn(0x557fd1619500 msgr2=0x557fd161f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 shutdown_connections 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 1 -- 172.21.15.60:0/2329945366 wait complete. 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.260+0000 7efdde158740 20 client.25781 trim_cache size 0 max 16384 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 Processor -- start 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 -- start start 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd16255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41ae8e0 con 0x557fd1627f80 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41ac150 con 0x557fd161bff0 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdde158740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41a9fa0 con 0x557fd161b6d0 2024-07-24T21:49:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdda344640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdda344640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48920/0 (socket says 172.21.15.60:48920) 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdd9342640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd16255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.274+0000 7efdda344640 1 -- 172.21.15.60:0/1468916105 learned_addr learned my addr 172.21.15.60:0/1468916105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 -- 172.21.15.60:0/1468916105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 msgr2=0x557fd161aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 -- 172.21.15.60:0/1468916105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 msgr2=0x557fd16255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd16255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 -- 172.21.15.60:0/1468916105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd161be20 con 0x557fd1627f80 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdd9b43640 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdda344640 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003eb0 tx=0x7efdcc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af9f9fe374c536 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdbf7fe640 1 -- 172.21.15.60:0/1468916105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00f600 con 0x557fd1627f80 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdbf7fe640 1 -- 172.21.15.60:0/1468916105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc00fc50 con 0x557fd1627f80 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.275+0000 7efdbf7fe640 1 -- 172.21.15.60:0/1468916105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc01ec50 con 0x557fd1627f80 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- 172.21.15.60:0/1468916105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 msgr2=0x557fd1645230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003eb0 tx=0x7efdcc01a000 comp rx=0 tx=0).stop 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- 172.21.15.60:0/1468916105 shutdown_connections 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd1627f80 0x557fd1645230 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd161bff0 0x557fd16255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- 172.21.15.60:0/1468916105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161b6d0 0x557fd161aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- 172.21.15.60:0/1468916105 >> 172.21.15.60:0/1468916105 conn(0x557fd1619500 msgr2=0x557fd1624ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- 172.21.15.60:0/1468916105 shutdown_connections 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- 172.21.15.60:0/1468916105 wait complete. 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 -- start start 2024-07-24T21:49:29.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 Processor -- start 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.276+0000 7efdde158740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1642fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd1627f80 0x557fd1643750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41aa610 con 0x557fd161b6d0 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41aba80 con 0x557fd1627f80 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41aace0 con 0x557fd161bff0 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 10 client.0 _ll_register_callbacks cb 0x7efdc4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48924/0 (socket says 172.21.15.60:48924) 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 -- 172.21.15.60:0/4285411766 learned_addr learned my addr 172.21.15.60:0/4285411766 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9342640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1642fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 -- 172.21.15.60:0/4285411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 msgr2=0x557fd1642fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1642fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 -- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd1627f80 msgr2=0x557fd1643750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd1627f80 0x557fd1643750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 -- 172.21.15.60:0/4285411766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd16a9760 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdd9b43640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efdd00cb170 tx=0x7efdd00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16468b5009ae8da0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0113060 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7efdd01040d0 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.277+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc00fc50 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdd00ca950 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 10 client.16305 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fd16a9760 con 0x557fd161b6d0 2024-07-24T21:49:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc00fc50 con 0x557fd161b6d0 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efdd003b460 con 0x557fd161b6d0 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 10 client.16305 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 20 client.16305 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd16aad50 0x557fd16ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 4 client.16305 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fd15903a0 con 0x557fd16aad50 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.278+0000 7efdd9342640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd16aad50 0x557fd16ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.279+0000 7efdd9342640 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd16aad50 0x557fd16ad170 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5d8760254f82bf1 server_cookie=7afc69d537209e9d in_seq=0 out_seq=0 2024-07-24T21:49:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.279+0000 7efdbd7fa640 10 client.16305 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:49:29.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdbd7fa640 1 -- 172.21.15.60:0/4285411766 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fd15903a0 con 0x557fd16aad50 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdbd7fa640 10 client.16305 handle_command_reply: tid=0 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 client.16305 shutdown 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 msgr2=0x557fd1645ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efdd00cb170 tx=0x7efdd00045e0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 shutdown_connections 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557fd16aad50 0x557fd16ad170 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557fd1627f80 0x557fd1643750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fd161bff0 0x557fd1642fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 --2- 172.21.15.60:0/4285411766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fd161b6d0 0x557fd1645ad0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.280+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 >> 172.21.15.60:0/4285411766 conn(0x557fd1619500 msgr2=0x557fd163bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efdbd7fa640 0 client.16305 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 shutdown_connections 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efdde158740 1 -- 172.21.15.60:0/4285411766 wait complete. 2024-07-24T21:49:29.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efdde158740 20 client.16305 trim_cache size 0 max 16384 2024-07-24T21:49:29.302 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:49:29.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 msgr2=0x7efdc00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 0x7efdc00b6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdd4076f50 tx=0x7efdcc009500 comp rx=0 tx=0).stop 2024-07-24T21:49:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 msgr2=0x7efdd4074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdd0005ab0 tx=0x7efdd0003c10 comp rx=0 tx=0).stop 2024-07-24T21:49:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 shutdown_connections 2024-07-24T21:49:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efdc00b3dc0 0x7efdc00b6280 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdd4108ae0 0x7efdd4075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdd4108160 0x7efdd40753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 --2- 172.21.15.60:0/266405998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdd4102750 0x7efdd4074e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 >> 172.21.15.60:0/266405998 conn(0x7efdd40fe200 msgr2=0x7efdd40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 shutdown_connections 2024-07-24T21:49:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:29.281+0000 7efddbda7640 1 -- 172.21.15.60:0/266405998 wait complete. 2024-07-24T21:49:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:29 smithi123 ceph-mon[37143]: pgmap v461: 97 pgs: 97 active+clean; 256 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-07-24T21:49:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:29 smithi086 ceph-mon[37128]: pgmap v461: 97 pgs: 97 active+clean; 256 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-07-24T21:49:29.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:29 smithi060 ceph-mon[33000]: pgmap v461: 97 pgs: 97 active+clean; 256 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-07-24T21:49:30.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:49:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:31 smithi123 ceph-mon[37143]: pgmap v462: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-07-24T21:49:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:31 smithi086 ceph-mon[37128]: pgmap v462: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-07-24T21:49:31.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:31 smithi060 ceph-mon[33000]: pgmap v462: 97 pgs: 97 active+clean; 347 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-07-24T21:49:32.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 -- 172.21.15.60:0/58793959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20103530 msgr2=0x7fee20103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 --2- 172.21.15.60:0/58793959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20103530 0x7fee20103910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fee10009860 tx=0x7fee1002a340 comp rx=0 tx=0).stop 2024-07-24T21:49:32.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 -- 172.21.15.60:0/58793959 shutdown_connections 2024-07-24T21:49:32.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 --2- 172.21.15.60:0/58793959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee2010cae0 0x7fee2010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 --2- 172.21.15.60:0/58793959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103f30 0x7fee2010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 --2- 172.21.15.60:0/58793959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee20103530 0x7fee20103910 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 -- 172.21.15.60:0/58793959 >> 172.21.15.60:0/58793959 conn(0x7fee200fd0c0 msgr2=0x7fee200ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 -- 172.21.15.60:0/58793959 shutdown_connections 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 -- 172.21.15.60:0/58793959 wait complete. 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.602+0000 7fee26640640 1 Processor -- start 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 -- start start 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103530 0x7fee2019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 0x7fee201a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112700 con 0x7fee2010cae0 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112580 con 0x7fee20103f30 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee26640640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee20112880 con 0x7fee20103530 2024-07-24T21:49:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 0x7fee201a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103530 0x7fee2019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 0x7fee201a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48956/0 (socket says 172.21.15.60:48956) 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53006/0 (socket says 172.21.15.60:53006) 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.603+0000 7fee24bdd640 1 -- 172.21.15.60:0/1096177422 learned_addr learned my addr 172.21.15.60:0/1096177422 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 -- 172.21.15.60:0/1096177422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103530 msgr2=0x7fee2019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103530 0x7fee2019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 -- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 msgr2=0x7fee201a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 0x7fee201a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 -- 172.21.15.60:0/1096177422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee201a0f80 con 0x7fee20103f30 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee1400e9f0 tx=0x7fee1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1d7fa640 1 -- 172.21.15.60:0/1096177422 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400ccc0 con 0x7fee20103f30 2024-07-24T21:49:32.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee201a1270 con 0x7fee20103f30 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1d7fa640 1 -- 172.21.15.60:0/1096177422 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140108e0 con 0x7fee20103f30 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee1d7fa640 1 -- 172.21.15.60:0/1096177422 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14018880 con 0x7fee20103f30 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.604+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee201a7930 con 0x7fee20103f30 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 Processor -- start 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 -- start start 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c099570a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c099573370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c0994db9b0 con 0x55c0995646d0 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0994caeb0 con 0x55c099570f80 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee289f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0994ceef0 con 0x55c099564ff0 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee1d7fa640 1 -- 172.21.15.60:0/1096177422 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fee14010430 con 0x7fee20103f30 2024-07-24T21:49:32.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48960/0 (socket says 172.21.15.60:48960) 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee24bdd640 1 -- 172.21.15.60:0/3496088558 learned_addr learned my addr 172.21.15.60:0/3496088558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c099573370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.607+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c099570a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 -- 172.21.15.60:0/3496088558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 msgr2=0x55c099570a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c099570a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 -- 172.21.15.60:0/3496088558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 msgr2=0x55c099573370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c099573370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 -- 172.21.15.60:0/3496088558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c099454320 con 0x55c0995646d0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c099570a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee24bdd640 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e550 tx=0x7fee180095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc9fe034f9ebca server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee067fc640 1 -- 172.21.15.60:0/3496088558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1800d990 con 0x55c0995646d0 2024-07-24T21:49:32.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee067fc640 1 -- 172.21.15.60:0/3496088558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1801c070 con 0x55c0995646d0 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee067fc640 1 -- 172.21.15.60:0/3496088558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18016380 con 0x55c0995646d0 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee1d7fa640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 0x7fee080b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee1ffff640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 0x7fee080b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.608+0000 7fee1d7fa640 1 -- 172.21.15.60:0/1096177422 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fee140e43e0 con 0x7fee20103f30 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- 172.21.15.60:0/3496088558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 msgr2=0x55c099564ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e550 tx=0x7fee180095c0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee1ffff640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 0x7fee080b6200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fee10009830 tx=0x7fee1002c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- 172.21.15.60:0/3496088558 shutdown_connections 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c099573370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c099570a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 --2- 172.21.15.60:0/3496088558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099564ab0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- 172.21.15.60:0/3496088558 >> 172.21.15.60:0/3496088558 conn(0x55c099562500 msgr2=0x55c0995636a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- 172.21.15.60:0/3496088558 shutdown_connections 2024-07-24T21:49:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- 172.21.15.60:0/3496088558 wait complete. 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.609+0000 7fee289f1740 1 -- start start 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 Processor -- start 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099570a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c09956ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c09956e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c09932e700 con 0x55c0995646d0 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c09946d5f0 con 0x55c099570f80 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c099573c00 con 0x55c099564ff0 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee289f1740 10 client.0 _ll_register_callbacks cb 0x55c09948d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099570a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c09956e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099570a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48962/0 (socket says 172.21.15.60:48962) 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee1ffff640 1 -- 172.21.15.60:0/2100884470 learned_addr learned my addr 172.21.15.60:0/2100884470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:32.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.610+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c09956ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 -- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 msgr2=0x55c09956ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c09956ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 -- 172.21.15.60:0/2100884470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 msgr2=0x55c099570a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099570a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 -- 172.21.15.60:0/2100884470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c099460680 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee24bdd640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c09956e660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e550 tx=0x7fee18009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e54f8ea1ff20f2e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fedff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18004e50 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fee18017420 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee1801c070 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee18017660 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee289f1740 10 client.25796 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.611+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c09922b650 con 0x55c099570f80 2024-07-24T21:49:32.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.612+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1801c070 con 0x55c099570f80 2024-07-24T21:49:32.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.612+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee1800b040 con 0x55c099570f80 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 10 client.25796 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 20 client.25796 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c099580a00 0x55c099582df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 4 client.25796 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0994dd900 con 0x55c099580a00 2024-07-24T21:49:32.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.613+0000 7fee1ffff640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c099580a00 0x55c099582df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.614+0000 7fee1ffff640 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c099580a00 0x55c099582df0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e8e3e22fef2a62 server_cookie=eda0605f41b03e57 in_seq=0 out_seq=0 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.614+0000 7fedff7fe640 10 client.25796 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.614+0000 7fedff7fe640 1 -- 172.21.15.60:0/2100884470 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0994dd900 con 0x55c099580a00 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.614+0000 7fedff7fe640 10 client.25796 handle_command_reply: tid=0 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 client.25796 shutdown 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 msgr2=0x55c09956e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c09956e660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e550 tx=0x7fee18009ea0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 shutdown_connections 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c099580a00 0x55c099582df0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099570f80 0x55c09956e660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fedff7fe640 0 client.25796 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099564ff0 0x55c09956ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 --2- 172.21.15.60:0/2100884470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0995646d0 0x55c099570a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 >> 172.21.15.60:0/2100884470 conn(0x55c099562500 msgr2=0x55c0995680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 shutdown_connections 2024-07-24T21:49:32.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 1 -- 172.21.15.60:0/2100884470 wait complete. 2024-07-24T21:49:32.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.615+0000 7fee289f1740 20 client.25796 trim_cache size 0 max 16384 2024-07-24T21:49:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.632+0000 7fee289f1740 1 Processor -- start 2024-07-24T21:49:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 -- start start 2024-07-24T21:49:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0995646d0 0x55c099563520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 0x55c0995884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee201b2db0 con 0x55c099570f80 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee201b0620 con 0x55c099564ff0 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee289f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee201ae470 con 0x55c0995646d0 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee24bdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48968/0 (socket says 172.21.15.60:48968) 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee24bdd640 1 -- 172.21.15.60:0/3137452670 learned_addr learned my addr 172.21.15.60:0/3137452670 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 0x55c0995884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0995646d0 0x55c099563520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3137452670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0995646d0 msgr2=0x55c099563520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0995646d0 0x55c099563520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3137452670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 msgr2=0x55c0995889e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3137452670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c099564e20 con 0x55c099564ff0 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.633+0000 7fee24bdd640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.634+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 0x55c0995884a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee140e1ad0 tx=0x7fee14009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2584ffd2e75b5421 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.634+0000 7fee05ffb640 1 -- 172.21.15.60:0/3137452670 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14010720 con 0x55c099564ff0 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.634+0000 7fee05ffb640 1 -- 172.21.15.60:0/3137452670 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140aab50 con 0x55c099564ff0 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.634+0000 7fee05ffb640 1 -- 172.21.15.60:0/3137452670 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140e1d50 con 0x55c099564ff0 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- 172.21.15.60:0/3137452670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 msgr2=0x55c0995884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 0x55c0995884a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee140e1ad0 tx=0x7fee14009700 comp rx=0 tx=0).stop 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- 172.21.15.60:0/3137452670 shutdown_connections 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099570f80 0x55c0995889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c099564ff0 0x55c0995884a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 --2- 172.21.15.60:0/3137452670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0995646d0 0x55c099563520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- 172.21.15.60:0/3137452670 >> 172.21.15.60:0/3137452670 conn(0x55c099562500 msgr2=0x55c09956fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- 172.21.15.60:0/3137452670 shutdown_connections 2024-07-24T21:49:32.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- 172.21.15.60:0/3137452670 wait complete. 2024-07-24T21:49:32.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 -- start start 2024-07-24T21:49:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.635+0000 7fee289f1740 1 Processor -- start 2024-07-24T21:49:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099570f80 0x55c099575e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee201aeae0 con 0x55c099564ff0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee201aff50 con 0x55c0995646d0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee201af1b0 con 0x55c099570f80 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee289f1740 10 client.0 _ll_register_callbacks cb 0x7fee140e7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48976/0 (socket says 172.21.15.60:48976) 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53044/0 (socket says 172.21.15.60:53044) 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3215496216 learned_addr learned my addr 172.21.15.60:0/3215496216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee24bdd640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099570f80 0x55c099575e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099570f80 msgr2=0x55c099575e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099570f80 0x55c099575e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3215496216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 msgr2=0x55c099578e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 -- 172.21.15.60:0/3215496216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c099576420 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fee1f7fe640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fee14000c00 tx=0x7fee140e7c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baee1289dc50c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.636+0000 7fedff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140e1f60 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fee140aa2a0 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee140aab50 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee14010720 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 10 client.16323 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0995ebcb0 con 0x55c099564ff0 2024-07-24T21:49:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140aab50 con 0x55c099564ff0 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee140c5ad0 con 0x55c099564ff0 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 10 client.16323 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 20 client.16323 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c0995ec410 0x55c0995ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 4 client.16323 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.637+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0994d93a0 con 0x55c0995ec410 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c0995ec410 0x55c0995ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fedff7fe640 10 client.16323 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee1ffff640 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c0995ec410 0x55c0995ee830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e50ccb71c4a14 server_cookie=3919ffa7cf579c6f in_seq=0 out_seq=0 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fedff7fe640 1 -- 172.21.15.60:0/3215496216 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0994d93a0 con 0x55c0995ec410 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fedff7fe640 10 client.16323 handle_command_reply: tid=0 2024-07-24T21:49:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 client.16323 shutdown 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 msgr2=0x55c0995793d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fee14000c00 tx=0x7fee140e7c60 comp rx=0 tx=0).stop 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 shutdown_connections 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c0995ec410 0x55c0995ee830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c099570f80 0x55c099575e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c099564ff0 0x55c0995793d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fedff7fe640 0 client.16323 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 --2- 172.21.15.60:0/3215496216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0995646d0 0x55c099578e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.638+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 >> 172.21.15.60:0/3215496216 conn(0x55c099562500 msgr2=0x55c099567810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 shutdown_connections 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee289f1740 1 -- 172.21.15.60:0/3215496216 wait complete. 2024-07-24T21:49:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee289f1740 20 client.16323 trim_cache size 0 max 16384 2024-07-24T21:49:32.645 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:49:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 msgr2=0x7fee080b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 0x7fee080b6200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fee10009830 tx=0x7fee1002c040 comp rx=0 tx=0).stop 2024-07-24T21:49:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 msgr2=0x7fee2019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fee1400e9f0 tx=0x7fee1400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 shutdown_connections 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fee080b3d40 0x7fee080b6200 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2010cae0 0x7fee201a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee20103f30 0x7fee2019b410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 --2- 172.21.15.60:0/1096177422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee20103530 0x7fee2019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 >> 172.21.15.60:0/1096177422 conn(0x7fee200fd0c0 msgr2=0x7fee200ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.639+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 shutdown_connections 2024-07-24T21:49:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:32.640+0000 7fee26640640 1 -- 172.21.15.60:0/1096177422 wait complete. 2024-07-24T21:49:33.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:33 smithi123 ceph-mon[37143]: pgmap v463: 97 pgs: 97 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-07-24T21:49:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:33 smithi086 ceph-mon[37128]: pgmap v463: 97 pgs: 97 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-07-24T21:49:33.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:33 smithi060 ceph-mon[33000]: pgmap v463: 97 pgs: 97 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-07-24T21:49:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f471ffff640 1 -- 172.21.15.60:0/803185954 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d52c0ecb0 con 0x563d52c89140 2024-07-24T21:49:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f471ffff640 10 client.25780 handle_command_reply: tid=0 2024-07-24T21:49:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 client.25780 shutdown 2024-07-24T21:49:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 msgr2=0x563d52c08420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c08420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4734000c00 tx=0x7f473401f940 comp rx=0 tx=0).stop 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 shutdown_connections 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563d52c89140 0x563d52c8b560 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563d52c0f170 0x563d52c08ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d52c031e0 0x563d52c08960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f471ffff640 0 client.25780 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 --2- 172.21.15.60:0/803185954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d52c028c0 0x563d52c08420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 >> 172.21.15.60:0/803185954 conn(0x563d52c006f0 msgr2=0x563d52c0a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 shutdown_connections 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.342+0000 7f4749b65740 1 -- 172.21.15.60:0/803185954 wait complete. 2024-07-24T21:49:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4749b65740 20 client.25780 trim_cache size 0 max 16384 2024-07-24T21:49:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:49:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:49:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:49:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 msgr2=0x7f47300b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 0x7f47300b6200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f473400bfd0 tx=0x7f473400b390 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 msgr2=0x7f474019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 0x7f474019b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f472c02aa00 tx=0x7f472c003cd0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 shutdown_connections 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47300b3d40 0x7f47300b6200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4740107910 0x7f47401a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4740106f90 0x7f474019b310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 --2- 172.21.15.60:0/2139957956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47401015e0 0x7f474019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 >> 172.21.15.60:0/2139957956 conn(0x7f47400782e0 msgr2=0x7f474006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.343+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 shutdown_connections 2024-07-24T21:49:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:34.344+0000 7f4746d32640 1 -- 172.21.15.60:0/2139957956 wait complete. 2024-07-24T21:49:34.373 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:49:34.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:49:34.708 INFO:tasks.workunit.client.0.smithi060.stdout: 122 621539 472 413761 659 167763 307 2024-07-24T21:49:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:35.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:35 smithi123 ceph-mon[37143]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-07-24T21:49:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:35 smithi086 ceph-mon[37128]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-07-24T21:49:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:35 smithi060 ceph-mon[33000]: pgmap v464: 97 pgs: 97 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-07-24T21:49:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.472+0000 7fda3f2dd640 1 -- 172.21.15.60:0/2241731292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109b40 msgr2=0x7fda38110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.472+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/2241731292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109b40 0x7fda38110ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda34009900 tx=0x7fda3402a4b0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.474+0000 7fda3f2dd640 1 -- 172.21.15.60:0/2241731292 shutdown_connections 2024-07-24T21:49:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.474+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/2241731292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda3811ba90 0x7fda3811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.474+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/2241731292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda38109f20 0x7fda3810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.474+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/2241731292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109b40 0x7fda38110ba0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.474+0000 7fda3f2dd640 1 -- 172.21.15.60:0/2241731292 >> 172.21.15.60:0/2241731292 conn(0x7fda3806d980 msgr2=0x7fda3806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.479+0000 7fda3f2dd640 1 -- 172.21.15.60:0/2241731292 shutdown_connections 2024-07-24T21:49:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.482+0000 7fda3f2dd640 1 -- 172.21.15.60:0/2241731292 wait complete. 2024-07-24T21:49:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.483+0000 7fda3f2dd640 1 Processor -- start 2024-07-24T21:49:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.484+0000 7fda3f2dd640 1 -- start start 2024-07-24T21:49:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.484+0000 7fda3f2dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3f2dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109f20 0x7fda381193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3f2dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3811ba90 0x7fda38112210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3f2dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fda381215e0 con 0x7fda38109b40 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3f2dd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda38121460 con 0x7fda3811ba90 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3f2dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda38121760 con 0x7fda38109f20 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3d079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3d079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58074/0 (socket says 172.21.15.60:58074) 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.485+0000 7fda3d079640 1 -- 172.21.15.60:0/1835431878 learned_addr learned my addr 172.21.15.60:0/1835431878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.486+0000 7fda3d079640 1 -- 172.21.15.60:0/1835431878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109f20 msgr2=0x7fda381193d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.486+0000 7fda3d079640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109f20 0x7fda381193d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.486+0000 7fda3d079640 1 -- 172.21.15.60:0/1835431878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3811ba90 msgr2=0x7fda38112210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:49:37.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.486+0000 7fda3d079640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3811ba90 0x7fda38112210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.486+0000 7fda3d079640 1 -- 172.21.15.60:0/1835431878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda38112ad0 con 0x7fda38109b40 2024-07-24T21:49:37.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.487+0000 7fda3d079640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda34005ab0 tx=0x7fda34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.489+0000 7fda2e7fc640 1 -- 172.21.15.60:0/1835431878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3403f070 con 0x7fda38109b40 2024-07-24T21:49:37.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.489+0000 7fda2e7fc640 1 -- 172.21.15.60:0/1835431878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda34031b50 con 0x7fda38109b40 2024-07-24T21:49:37.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.489+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda38112d60 con 0x7fda38109b40 2024-07-24T21:49:37.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.489+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda38117d50 con 0x7fda38109b40 2024-07-24T21:49:37.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.489+0000 7fda2e7fc640 1 -- 172.21.15.60:0/1835431878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035030 con 0x7fda38109b40 2024-07-24T21:49:37.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.494+0000 7fda4168e740 1 Processor -- start 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.496+0000 7fda4168e740 1 -- start start 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.496+0000 7fda4168e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b3394310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.496+0000 7fda4168e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda4168e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda4168e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634b330b9f0 con 0x5634b3393f30 2024-07-24T21:49:37.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda4168e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634b32faef0 con 0x5634b33a07e0 2024-07-24T21:49:37.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda4168e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634b32fef30 con 0x5634b3394850 2024-07-24T21:49:37.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda3d079640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda3d079640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46226/0 (socket says 172.21.15.60:46226) 2024-07-24T21:49:37.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda3d079640 1 -- 172.21.15.60:0/561970031 learned_addr learned my addr 172.21.15.60:0/561970031 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:37.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.497+0000 7fda3d079640 1 -- 172.21.15.60:0/561970031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 msgr2=0x5634b33a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3c878640 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3d079640 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3d079640 1 -- 172.21.15.60:0/561970031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 msgr2=0x5634b3394310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3d079640 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b3394310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3d079640 1 -- 172.21.15.60:0/561970031 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b305b930 con 0x5634b3394850 2024-07-24T21:49:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.498+0000 7fda3d079640 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda34031fc0 tx=0x7fda3402a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed4f2a621bb21f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.499+0000 7fda3c878640 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:49:37.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.499+0000 7fda237fe640 1 -- 172.21.15.60:0/561970031 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34044860 con 0x5634b3394850 2024-07-24T21:49:37.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.499+0000 7fda237fe640 1 -- 172.21.15.60:0/561970031 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda340319a0 con 0x5634b3394850 2024-07-24T21:49:37.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.500+0000 7fda4168e740 1 -- 172.21.15.60:0/561970031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 msgr2=0x5634b33a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.500+0000 7fda4168e740 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda34031fc0 tx=0x7fda3402a4b0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.510+0000 7fda2e7fc640 1 -- 172.21.15.60:0/1835431878 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fda34042070 con 0x7fda38109b40 2024-07-24T21:49:37.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.511+0000 7fda2e7fc640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 0x7fda240b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.512+0000 7fda2e7fc640 1 -- 172.21.15.60:0/1835431878 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fda34108a00 con 0x7fda38109b40 2024-07-24T21:49:37.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.513+0000 7fda3d87a640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 0x7fda240b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.514+0000 7fda3d87a640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 0x7fda240b6410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda30005c40 tx=0x7fda30005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.516+0000 7fda4168e740 1 -- 172.21.15.60:0/561970031 shutdown_connections 2024-07-24T21:49:37.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.516+0000 7fda4168e740 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.516+0000 7fda4168e740 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a02a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.516+0000 7fda4168e740 1 --2- 172.21.15.60:0/561970031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b3394310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.516+0000 7fda4168e740 1 -- 172.21.15.60:0/561970031 >> 172.21.15.60:0/561970031 conn(0x5634b3391d60 msgr2=0x5634b3392f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.518+0000 7fda4168e740 1 -- 172.21.15.60:0/561970031 shutdown_connections 2024-07-24T21:49:37.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.522+0000 7fda4168e740 1 -- 172.21.15.60:0/561970031 wait complete. 2024-07-24T21:49:37.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.522+0000 7fda4168e740 1 -- start start 2024-07-24T21:49:37.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.522+0000 7fda4168e740 1 Processor -- start 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3394850 0x5634b339d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b339dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634b315e890 con 0x5634b3393f30 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634b329d390 con 0x5634b3394850 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634b32fef30 con 0x5634b33a07e0 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.523+0000 7fda4168e740 10 client.0 _ll_register_callbacks cb 0x5634b32a7500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.524+0000 7fda3d079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.525+0000 7fda3d87a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b339dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.525+0000 7fda3d079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58104/0 (socket says 172.21.15.60:58104) 2024-07-24T21:49:37.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.525+0000 7fda3d079640 1 -- 172.21.15.60:0/1726739809 learned_addr learned my addr 172.21.15.60:0/1726739809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:37.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 -- 172.21.15.60:0/1726739809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3394850 msgr2=0x5634b339d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3394850 0x5634b339d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 -- 172.21.15.60:0/1726739809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 msgr2=0x5634b33a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 -- 172.21.15.60:0/1726739809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b3297d60 con 0x5634b33a07e0 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda3d87a640 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b339dbe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda30004ce0 tx=0x7fda30004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce67e040d1cb7a00 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.526+0000 7fda20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.530+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30021070 con 0x5634b33a07e0 2024-07-24T21:49:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.530+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3404de50 con 0x5634b33a07e0 2024-07-24T21:49:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.530+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fda3001d660 con 0x5634b33a07e0 2024-07-24T21:49:37.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.533+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda3002b050 con 0x5634b33a07e0 2024-07-24T21:49:37.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.533+0000 7fda4168e740 10 client.25825 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:37.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.533+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634b33d0ef0 con 0x5634b33a07e0 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.534+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3404de50 con 0x5634b33a07e0 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.535+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fda30026850 con 0x5634b33a07e0 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.535+0000 7fda4168e740 10 client.25825 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.535+0000 7fda4168e740 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.535+0000 7fda4168e740 20 client.25825 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.536+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b33d2880 0x5634b33d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.536+0000 7fda4168e740 4 client.25825 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.536+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634b330d940 con 0x5634b33d2880 2024-07-24T21:49:37.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.538+0000 7fda3d079640 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b33d2880 0x5634b33d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.544 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:37 smithi060 ceph-mon[33000]: pgmap v465: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 973 op/s 2024-07-24T21:49:37.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.542+0000 7fda3d079640 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b33d2880 0x5634b33d4d00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8117ba94f5583c62 server_cookie=a654295c2580c065 in_seq=0 out_seq=0 2024-07-24T21:49:37.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.542+0000 7fda20ff9640 10 client.25825 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:37.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.544+0000 7fda20ff9640 1 -- 172.21.15.60:0/1726739809 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634b330d940 con 0x5634b33d2880 2024-07-24T21:49:37.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.544+0000 7fda20ff9640 10 client.25825 handle_command_reply: tid=0 2024-07-24T21:49:37.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.544+0000 7fda4168e740 1 client.25825 shutdown 2024-07-24T21:49:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.550+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 msgr2=0x5634b339dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.550+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b339dbe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda30004ce0 tx=0x7fda30004a40 comp rx=0 tx=0).stop 2024-07-24T21:49:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 shutdown_connections 2024-07-24T21:49:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b33d2880 0x5634b33d4d00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b339dbe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3394850 0x5634b339d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 --2- 172.21.15.60:0/1726739809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 >> 172.21.15.60:0/1726739809 conn(0x5634b3391d60 msgr2=0x5634b3397900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.551+0000 7fda20ff9640 0 client.25825 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.557+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 shutdown_connections 2024-07-24T21:49:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.557+0000 7fda4168e740 1 -- 172.21.15.60:0/1726739809 wait complete. 2024-07-24T21:49:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.557+0000 7fda4168e740 20 client.25825 trim_cache size 0 max 16384 2024-07-24T21:49:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.606+0000 7fda4168e740 1 Processor -- start 2024-07-24T21:49:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.606+0000 7fda4168e740 1 -- start start 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 0x5634b33da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b3397070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c011240 con 0x5634b3394850 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c00eab0 con 0x5634b3393f30 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda4168e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c00c900 con 0x5634b33a07e0 2024-07-24T21:49:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda3c878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda3c878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58112/0 (socket says 172.21.15.60:58112) 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.607+0000 7fda3c878640 1 -- 172.21.15.60:0/890725960 learned_addr learned my addr 172.21.15.60:0/890725960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3d079640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 0x5634b33da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3d87a640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b3397070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3c878640 1 -- 172.21.15.60:0/890725960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 msgr2=0x5634b3397070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3c878640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b3397070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3c878640 1 -- 172.21.15.60:0/890725960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 msgr2=0x5634b33da5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3c878640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 0x5634b33da5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3c878640 1 -- 172.21.15.60:0/890725960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b33a0480 con 0x5634b3394850 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3d079640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 0x5634b33da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.608+0000 7fda3d87a640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b3397070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:49:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.610+0000 7fda3c878640 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda2800a6f0 tx=0x7fda2800abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6925f358d080552e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.610+0000 7fda22ffd640 1 -- 172.21.15.60:0/890725960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28015070 con 0x5634b3394850 2024-07-24T21:49:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.610+0000 7fda22ffd640 1 -- 172.21.15.60:0/890725960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2800cdf0 con 0x5634b3394850 2024-07-24T21:49:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.610+0000 7fda22ffd640 1 -- 172.21.15.60:0/890725960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28018360 con 0x5634b3394850 2024-07-24T21:49:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.612+0000 7fda4168e740 1 -- 172.21.15.60:0/890725960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 msgr2=0x5634b33dab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.612+0000 7fda4168e740 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda2800a6f0 tx=0x7fda2800abb0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.615+0000 7fda4168e740 1 -- 172.21.15.60:0/890725960 shutdown_connections 2024-07-24T21:49:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.615+0000 7fda4168e740 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b33a07e0 0x5634b3397070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.615+0000 7fda4168e740 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3394850 0x5634b33dab10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.615+0000 7fda4168e740 1 --2- 172.21.15.60:0/890725960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b3393f30 0x5634b33da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.615+0000 7fda4168e740 1 -- 172.21.15.60:0/890725960 >> 172.21.15.60:0/890725960 conn(0x5634b3391d60 msgr2=0x5634b33987b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.620+0000 7fda4168e740 1 -- 172.21.15.60:0/890725960 shutdown_connections 2024-07-24T21:49:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.620+0000 7fda4168e740 1 -- 172.21.15.60:0/890725960 wait complete. 2024-07-24T21:49:37.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.621+0000 7fda4168e740 1 -- start start 2024-07-24T21:49:37.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.621+0000 7fda4168e740 1 Processor -- start 2024-07-24T21:49:37.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c00cf70 con 0x5634b3393f30 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c00e3e0 con 0x5634b33a07e0 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c00d640 con 0x5634b3394850 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda4168e740 10 client.0 _ll_register_callbacks cb 0x7fda3000ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda3c878640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda3c878640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46248/0 (socket says 172.21.15.60:46248) 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda3c878640 1 -- 172.21.15.60:0/365955035 learned_addr learned my addr 172.21.15.60:0/365955035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda3d87a640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.622+0000 7fda3d079640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.623+0000 7fda3d079640 1 -- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 msgr2=0x5634b33a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.623+0000 7fda3d079640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.623+0000 7fda3d079640 1 -- 172.21.15.60:0/365955035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 msgr2=0x5634b33a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.623+0000 7fda3d079640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.623+0000 7fda3d079640 1 -- 172.21.15.60:0/365955035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b33a1fb0 con 0x5634b3393f30 2024-07-24T21:49:37.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.624+0000 7fda3d079640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a8890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda340d8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb56ab2e20c38e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:37.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.624+0000 7fda20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:49:37.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.631+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340cf150 con 0x5634b3393f30 2024-07-24T21:49:37.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.631+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2800cdf0 con 0x5634b3393f30 2024-07-24T21:49:37.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.631+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fda340d60b0 con 0x5634b3393f30 2024-07-24T21:49:37.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.633+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda340cfd40 con 0x5634b3393f30 2024-07-24T21:49:37.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.636+0000 7fda4168e740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:37.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.636+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634b33a7250 con 0x5634b3393f30 2024-07-24T21:49:37.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.642+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2800cdf0 con 0x5634b3393f30 2024-07-24T21:49:37.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.642+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fda34106370 con 0x5634b3393f30 2024-07-24T21:49:37.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:37.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:37.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 20 client.16338 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:37.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b343da70 0x5634b343fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:37.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 4 client.16338 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:49:37.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.643+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5634b33093e0 con 0x5634b343da70 2024-07-24T21:49:37.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.645+0000 7fda3c878640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b343da70 0x5634b343fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:37.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.651+0000 7fda3c878640 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b343da70 0x5634b343fe90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fe4cdededc20d9 server_cookie=5a715e54280be54c in_seq=0 out_seq=0 2024-07-24T21:49:37.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.651+0000 7fda20ff9640 10 client.16338 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:37.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.656+0000 7fda20ff9640 1 -- 172.21.15.60:0/365955035 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5634b33093e0 con 0x5634b343da70 2024-07-24T21:49:37.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.656+0000 7fda20ff9640 10 client.16338 handle_command_reply: tid=0 2024-07-24T21:49:37.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.656+0000 7fda4168e740 1 client.16338 shutdown 2024-07-24T21:49:37.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 msgr2=0x5634b33a8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a8890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda340d8040 comp rx=0 tx=0).stop 2024-07-24T21:49:37.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 shutdown_connections 2024-07-24T21:49:37.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634b343da70 0x5634b343fe90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634b33a07e0 0x5634b33a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634b3394850 0x5634b33a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 --2- 172.21.15.60:0/365955035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634b3393f30 0x5634b33a8890 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.658+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 >> 172.21.15.60:0/365955035 conn(0x5634b3391d60 msgr2=0x5634b3397070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.660+0000 7fda20ff9640 0 client.16338 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:37.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.662+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 shutdown_connections 2024-07-24T21:49:37.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.664+0000 7fda4168e740 1 -- 172.21.15.60:0/365955035 wait complete. 2024-07-24T21:49:37.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.664+0000 7fda4168e740 20 client.16338 trim_cache size 0 max 16384 2024-07-24T21:49:37.665 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:49:37.666 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:49:37.666 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "e25d7ad3-9469-4619-9972-71a284a6d5b8", 2024-07-24T21:49:37.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:49:37.666 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 msgr2=0x7fda240b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 0x7fda240b6410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda30005c40 tx=0x7fda30005bd0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 msgr2=0x7fda38118e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda34005ab0 tx=0x7fda34003c20 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 shutdown_connections 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fda240b3f50 0x7fda240b6410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3811ba90 0x7fda38112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda38109f20 0x7fda381193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 --2- 172.21.15.60:0/1835431878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda38109b40 0x7fda38118e90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.665+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 >> 172.21.15.60:0/1835431878 conn(0x7fda3806d980 msgr2=0x7fda3811c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:37.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.670+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 shutdown_connections 2024-07-24T21:49:37.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:37.672+0000 7fda3f2dd640 1 -- 172.21.15.60:0/1835431878 wait complete. 2024-07-24T21:49:37.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:37 smithi123 ceph-mon[37143]: pgmap v465: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 973 op/s 2024-07-24T21:49:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:37 smithi086 ceph-mon[37128]: pgmap v465: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 973 op/s 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:38 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:38 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:38 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:49:39.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:49:39.509 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:39 smithi060 ceph-mon[33000]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-07-24T21:49:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:39 smithi086 ceph-mon[37128]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-07-24T21:49:39.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:39 smithi123 ceph-mon[37143]: pgmap v466: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-07-24T21:49:40.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:40 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:40.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:40 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:40.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:40 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:41.144 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-24T21:49:41.144 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-24T21:49:41.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:49:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:41 smithi086 ceph-mon[37128]: pgmap v467: 97 pgs: 97 active+clean; 426 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 939 op/s 2024-07-24T21:49:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:41 smithi060 ceph-mon[33000]: pgmap v467: 97 pgs: 97 active+clean; 426 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 939 op/s 2024-07-24T21:49:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:41.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:41 smithi123 ceph-mon[37143]: pgmap v467: 97 pgs: 97 active+clean; 426 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 939 op/s 2024-07-24T21:49:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:42.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:42.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:49:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:43 smithi086 ceph-mon[37128]: pgmap v468: 97 pgs: 97 active+clean; 433 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-07-24T21:49:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:43 smithi060 ceph-mon[33000]: pgmap v468: 97 pgs: 97 active+clean; 433 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-07-24T21:49:43.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:43 smithi123 ceph-mon[37143]: pgmap v468: 97 pgs: 97 active+clean; 433 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-07-24T21:49:44.708 INFO:tasks.workunit.client.0.smithi060.stdout: 162 2013116 2022 1374475 2283 501105 1155 2024-07-24T21:49:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:45.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:45 smithi060 ceph-mon[33000]: pgmap v469: 97 pgs: 97 active+clean; 437 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-07-24T21:49:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:45 smithi086 ceph-mon[37128]: pgmap v469: 97 pgs: 97 active+clean; 437 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-07-24T21:49:45.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 -- 172.21.15.60:0/1112517789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 msgr2=0x7f7a081074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/1112517789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a081074a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009900 tx=0x7f79fc02a410 comp rx=0 tx=0).stop 2024-07-24T21:49:45.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 -- 172.21.15.60:0/1112517789 shutdown_connections 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/1112517789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a0810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/1112517789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a081074a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/1112517789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a08101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 -- 172.21.15.60:0/1112517789 >> 172.21.15.60:0/1112517789 conn(0x7f7a080fd0c0 msgr2=0x7f7a080ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 -- 172.21.15.60:0/1112517789 shutdown_connections 2024-07-24T21:49:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 -- 172.21.15.60:0/1112517789 wait complete. 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.927+0000 7f7a0d55a640 1 Processor -- start 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 -- start start 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a080fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a080fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111610 con 0x7f7a08107060 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111490 con 0x7f7a08101650 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a0d55a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111790 con 0x7f7a081079e0 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a080fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58606/0 (socket says 172.21.15.60:58606) 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 -- 172.21.15.60:0/900059521 learned_addr learned my addr 172.21.15.60:0/900059521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a077fe640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a080fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 -- 172.21.15.60:0/900059521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 msgr2=0x7f7a080fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a080fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 -- 172.21.15.60:0/900059521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 msgr2=0x7f7a080fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a080fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a067fc640 1 -- 172.21.15.60:0/900059521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a081a70a0 con 0x7f7a08107060 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a080fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.928+0000 7f7a077fe640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a080fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f7a067fc640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005940 tx=0x7f79fc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f79effff640 1 -- 172.21.15.60:0/900059521 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc03f030 con 0x7f7a08107060 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f79effff640 1 -- 172.21.15.60:0/900059521 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79fc0044c0 con 0x7f7a08107060 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a081a7330 con 0x7f7a08107060 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f79effff640 1 -- 172.21.15.60:0/900059521 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc035070 con 0x7f7a08107060 2024-07-24T21:49:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.929+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a081a7860 con 0x7f7a08107060 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f79effff640 1 -- 172.21.15.60:0/900059521 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f79fc042030 con 0x7f7a08107060 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 Processor -- start 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 -- start start 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 0x55dfd974ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd975d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dfd96c5960 con 0x55dfd975af30 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dfd96b4e60 con 0x55dfd974e680 2024-07-24T21:49:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dfd96b8ea0 con 0x55dfd974efa0 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 0x55dfd974ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58980/0 (socket says 172.21.15.60:58980) 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 0x55dfd974ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39422/0 (socket says 172.21.15.60:39422) 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a06ffd640 1 -- 172.21.15.60:0/1916123705 learned_addr learned my addr 172.21.15.60:0/1916123705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.932+0000 7f7a067fc640 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd975d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 -- 172.21.15.60:0/1916123705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 msgr2=0x55dfd974ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 0x55dfd974ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 -- 172.21.15.60:0/1916123705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 msgr2=0x55dfd975d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd975d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 -- 172.21.15.60:0/1916123705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd963e2d0 con 0x55dfd974efa0 2024-07-24T21:49:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79f80115f0 tx=0x7f79f8011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f7b85974f2710c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f79effff640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 0x7f79f00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f79ed7fa640 1 -- 172.21.15.60:0/1916123705 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f801e070 con 0x55dfd974efa0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f79ed7fa640 1 -- 172.21.15.60:0/1916123705 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f800faf0 con 0x55dfd974efa0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f79ed7fa640 1 -- 172.21.15.60:0/1916123705 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8019ac0 con 0x55dfd974efa0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f7a077fe640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 0x7f79f00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.933+0000 7f79effff640 1 -- 172.21.15.60:0/900059521 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79fc1042b0 con 0x7f7a08107060 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- 172.21.15.60:0/1916123705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 msgr2=0x55dfd975a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79f80115f0 tx=0x7f79f8011ab0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a077fe640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 0x7f79f00b6280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009fd0 tx=0x7f7a00009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- 172.21.15.60:0/1916123705 shutdown_connections 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd975d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd975a9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/1916123705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974e680 0x55dfd974ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- 172.21.15.60:0/1916123705 >> 172.21.15.60:0/1916123705 conn(0x55dfd974c4b0 msgr2=0x55dfd974d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- 172.21.15.60:0/1916123705 shutdown_connections 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- 172.21.15.60:0/1916123705 wait complete. 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.934+0000 7f7a0f90b740 1 -- start start 2024-07-24T21:49:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 Processor -- start 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd9757db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dfd9518700 con 0x55dfd974e680 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dfd96575a0 con 0x55dfd974efa0 2024-07-24T21:49:45.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dfd975dbb0 con 0x55dfd975af30 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a0f90b740 10 client.0 _ll_register_callbacks cb 0x55dfd9677910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58630/0 (socket says 172.21.15.60:58630) 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58994/0 (socket says 172.21.15.60:58994) 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a06ffd640 1 -- 172.21.15.60:0/4136080530 learned_addr learned my addr 172.21.15.60:0/4136080530 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a067fc640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd9757db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.935+0000 7f7a077fe640 1 -- 172.21.15.60:0/4136080530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 msgr2=0x55dfd9757db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a077fe640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd9757db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a077fe640 1 -- 172.21.15.60:0/4136080530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 msgr2=0x55dfd975a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a077fe640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a077fe640 1 -- 172.21.15.60:0/4136080530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd964a630 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a067fc640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd9757db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a077fe640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a080ff680 tx=0x7f7a00004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f675e56c927ea5ce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f79eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a000044a0 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f800faf0 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7a00018030 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7a00004640 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a0f90b740 10 client.25840 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.936+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfd9415650 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f800faf0 con 0x55dfd975af30 2024-07-24T21:49:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7a00028420 con 0x55dfd975af30 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 10 client.25840 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 20 client.25840 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd976aa60 0x55dfd976ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 4 client.25840 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd96c78b0 con 0x55dfd976aa60 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.937+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd976aa60 0x55dfd976ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.938+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd976aa60 0x55dfd976ce50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c2bdb9a56276cc server_cookie=b9a76444d2b51692 in_seq=0 out_seq=0 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.938+0000 7f79eaff5640 10 client.25840 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f79eaff5640 1 -- 172.21.15.60:0/4136080530 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfd96c78b0 con 0x55dfd976aa60 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f79eaff5640 10 client.25840 handle_command_reply: tid=0 2024-07-24T21:49:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 client.25840 shutdown 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 msgr2=0x55dfd97584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a080ff680 tx=0x7f7a00004ab0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 shutdown_connections 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd976aa60 0x55dfd976ce50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f79eaff5640 0 client.25840 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd975af30 0x55dfd97584f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd9757db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/4136080530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd975a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.940+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 >> 172.21.15.60:0/4136080530 conn(0x55dfd974c4b0 msgr2=0x55dfd9752050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.941+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 shutdown_connections 2024-07-24T21:49:45.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.941+0000 7f7a0f90b740 1 -- 172.21.15.60:0/4136080530 wait complete. 2024-07-24T21:49:45.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.941+0000 7f7a0f90b740 20 client.25840 trim_cache size 0 max 16384 2024-07-24T21:49:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.982+0000 7f7a0f90b740 1 Processor -- start 2024-07-24T21:49:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 -- start start 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd9759af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd975af30 0x55dfd9772760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081b2c90 con 0x55dfd974e680 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081b0500 con 0x55dfd975af30 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081ae350 con 0x55dfd974efa0 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58642/0 (socket says 172.21.15.60:58642) 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a067fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd9759af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2493602108 learned_addr learned my addr 172.21.15.60:0/2493602108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a077fe640 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd975af30 0x55dfd9772760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2493602108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 msgr2=0x55dfd9759af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd9759af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2493602108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd975af30 msgr2=0x55dfd9772760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd975af30 0x55dfd9772760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.983+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2493602108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd975acd0 con 0x55dfd974e680 2024-07-24T21:49:45.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.984+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79f80195d0 tx=0x7f79f80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3805e7c9378d165 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.984+0000 7f79ecff9640 1 -- 172.21.15.60:0/2493602108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f801e040 con 0x55dfd974e680 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.984+0000 7f79ecff9640 1 -- 172.21.15.60:0/2493602108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f8029030 con 0x55dfd974e680 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.984+0000 7f79ecff9640 1 -- 172.21.15.60:0/2493602108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8019a30 con 0x55dfd974e680 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2493602108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 msgr2=0x55dfd9766d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f79f80195d0 tx=0x7f79f80196a0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2493602108 shutdown_connections 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd975af30 0x55dfd9772760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974efa0 0x55dfd9759af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2493602108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd974e680 0x55dfd9766d60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.985+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2493602108 >> 172.21.15.60:0/2493602108 conn(0x55dfd974c4b0 msgr2=0x55dfd97593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.986+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2493602108 shutdown_connections 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.986+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2493602108 wait complete. 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.986+0000 7f7a0f90b740 1 -- start start 2024-07-24T21:49:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.986+0000 7f7a0f90b740 1 Processor -- start 2024-07-24T21:49:45.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd97549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd9754f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081ae9c0 con 0x55dfd975af30 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081afe30 con 0x55dfd974efa0 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081af090 con 0x55dfd974e680 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a0f90b740 10 client.0 _ll_register_callbacks cb 0x7f7a000126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59024/0 (socket says 172.21.15.60:59024) 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2473619602 learned_addr learned my addr 172.21.15.60:0/2473619602 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a067fc640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd97549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a077fe640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd9754f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2473619602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 msgr2=0x55dfd97549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd97549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2473619602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 msgr2=0x55dfd9754f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd9754f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 -- 172.21.15.60:0/2473619602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd97555b0 con 0x55dfd974e680 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f7a06ffd640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79f8011510 tx=0x7f79f8012fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3121a872ecee3ff0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.987+0000 7f79eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f801e030 con 0x55dfd974e680 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f79f800fa70 con 0x55dfd974e680 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f8029030 con 0x55dfd974e680 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79f8005070 con 0x55dfd974e680 2024-07-24T21:49:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 10 client.25846 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfd97d5610 con 0x55dfd974e680 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79f80197d0 con 0x55dfd974e680 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79f80035f0 con 0x55dfd974e680 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 10 client.25846 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 20 client.25846 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd97d5b10 0x55dfd97d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 4 client.25846 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.988+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f79f8029030 con 0x55dfd97d5b10 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.989+0000 7f7a067fc640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd97d5b10 0x55dfd97d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.989+0000 7f79eaff5640 10 client.25846 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:45.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.989+0000 7f7a067fc640 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd97d5b10 0x55dfd97d7f30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339bf92bbb9842f5 server_cookie=f2bc01165eea9f20 in_seq=0 out_seq=0 2024-07-24T21:49:45.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:45 smithi123 ceph-mon[37143]: pgmap v469: 97 pgs: 97 active+clean; 437 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f79eaff5640 1 -- 172.21.15.60:0/2473619602 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f79f8029030 con 0x55dfd97d5b10 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f79eaff5640 10 client.25846 handle_command_reply: tid=0 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 client.25846 shutdown 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 msgr2=0x55dfd9763400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79f8011510 tx=0x7f79f8012fc0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 shutdown_connections 2024-07-24T21:49:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dfd97d5b10 0x55dfd97d7f30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfd975af30 0x55dfd9754f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dfd974efa0 0x55dfd97549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f79eaff5640 0 client.25846 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 --2- 172.21.15.60:0/2473619602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfd974e680 0x55dfd9763400 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 >> 172.21.15.60:0/2473619602 conn(0x55dfd974c4b0 msgr2=0x55dfd9759c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 shutdown_connections 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.990+0000 7f7a0f90b740 1 -- 172.21.15.60:0/2473619602 wait complete. 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0f90b740 20 client.25846 trim_cache size 0 max 16384 2024-07-24T21:49:45.996 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "e25d7ad3-9469-4619-9972-71a284a6d5b8": { 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "e25d7ad3-9469-4619-9972-71a284a6d5b8", 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:49:45.997 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 msgr2=0x7f79f00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 0x7f79f00b6280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009fd0 tx=0x7f7a00009500 comp rx=0 tx=0).stop 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 msgr2=0x7f7a080fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005940 tx=0x7f79fc0040c0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 shutdown_connections 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f79f00b3dc0 0x7f79f00b6280 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a081079e0 0x7f7a080fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a08107060 0x7f7a080fdb00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 --2- 172.21.15.60:0/900059521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a08101650 0x7f7a080fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.991+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 >> 172.21.15.60:0/900059521 conn(0x7f7a080fd0c0 msgr2=0x7f7a080fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.992+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 shutdown_connections 2024-07-24T21:49:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:45.992+0000 7f7a0d55a640 1 -- 172.21.15.60:0/900059521 wait complete. 2024-07-24T21:49:47.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:47 smithi060 ceph-mon[33000]: pgmap v470: 97 pgs: 97 active+clean; 444 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 569 op/s 2024-07-24T21:49:47.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:47 smithi086 ceph-mon[37128]: pgmap v470: 97 pgs: 97 active+clean; 444 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 569 op/s 2024-07-24T21:49:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.830+0000 7fc265fd0640 1 -- 172.21.15.60:0/2791038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 msgr2=0x7fc260110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.830+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2791038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc260110ba0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc250009860 tx=0x7fc25002a360 comp rx=0 tx=0).stop 2024-07-24T21:49:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.834+0000 7fc265fd0640 1 -- 172.21.15.60:0/2791038788 shutdown_connections 2024-07-24T21:49:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.834+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2791038788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc26011ba90 0x7fc26011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.834+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2791038788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc26010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.834+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2791038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc260110ba0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.834+0000 7fc265fd0640 1 -- 172.21.15.60:0/2791038788 >> 172.21.15.60:0/2791038788 conn(0x7fc26006d5a0 msgr2=0x7fc26006d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.841+0000 7fc265fd0640 1 -- 172.21.15.60:0/2791038788 shutdown_connections 2024-07-24T21:49:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.863+0000 7fc265fd0640 1 -- 172.21.15.60:0/2791038788 wait complete. 2024-07-24T21:49:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.864+0000 7fc265fd0640 1 Processor -- start 2024-07-24T21:49:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 -- start start 2024-07-24T21:49:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc2601172d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc260117810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc26011ba90 0x7fc2601a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113b00 con 0x7fc26011ba90 2024-07-24T21:49:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113980 con 0x7fc260109f20 2024-07-24T21:49:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.873+0000 7fc265fd0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113c80 con 0x7fc260109b40 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc2601172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc2601172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59038/0 (socket says 172.21.15.60:59038) 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25f7fe640 1 -- 172.21.15.60:0/2478978666 learned_addr learned my addr 172.21.15.60:0/2478978666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc260117810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 -- 172.21.15.60:0/2478978666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 msgr2=0x7fc2601172d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc2601172d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 -- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc26011ba90 msgr2=0x7fc2601a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc26011ba90 0x7fc2601a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.874+0000 7fc25effd640 1 -- 172.21.15.60:0/2478978666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2601a50c0 con 0x7fc260109f20 2024-07-24T21:49:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.875+0000 7fc25effd640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc260117810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d650 tx=0x7fc25400db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.876+0000 7fc25cff9640 1 -- 172.21.15.60:0/2478978666 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017070 con 0x7fc260109f20 2024-07-24T21:49:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.876+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2601a53b0 con 0x7fc260109f20 2024-07-24T21:49:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.877+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2601aba70 con 0x7fc260109f20 2024-07-24T21:49:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.877+0000 7fc25cff9640 1 -- 172.21.15.60:0/2478978666 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc254009c70 con 0x7fc260109f20 2024-07-24T21:49:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.877+0000 7fc25cff9640 1 -- 172.21.15.60:0/2478978666 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254003b00 con 0x7fc260109f20 2024-07-24T21:49:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.884+0000 7fc25cff9640 1 -- 172.21.15.60:0/2478978666 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc254003ca0 con 0x7fc260109f20 2024-07-24T21:49:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.889+0000 7fc268e06740 1 Processor -- start 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 -- start start 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000faa3c0 0x556000fb5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556000f1f280 con 0x556000fb6350 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556000f0e780 con 0x556000faa3c0 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.894+0000 7fc268e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556000f127c0 con 0x556000fa9aa0 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.895+0000 7fc25ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.895+0000 7fc25ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59046/0 (socket says 172.21.15.60:59046) 2024-07-24T21:49:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.895+0000 7fc25ffff640 1 -- 172.21.15.60:0/1183423677 learned_addr learned my addr 172.21.15.60:0/1183423677 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.895+0000 7fc25cff9640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 0x7fc2480b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.901+0000 7fc25cff9640 1 -- 172.21.15.60:0/2478978666 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc2540e3650 con 0x7fc260109f20 2024-07-24T21:49:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 0x7fc2480b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 -- 172.21.15.60:0/1183423677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000faa3c0 msgr2=0x556000fb5e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:49:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000faa3c0 0x556000fb5e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 -- 172.21.15.60:0/1183423677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 msgr2=0x556000fb8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 -- 172.21.15.60:0/1183423677 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556000c72790 con 0x556000fa9aa0 2024-07-24T21:49:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.902+0000 7fc25ffff640 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc258021740 tx=0x7fc258021c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df37854fef47ee94 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.903+0000 7fc245ffb640 1 -- 172.21.15.60:0/1183423677 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258004b20 con 0x556000fa9aa0 2024-07-24T21:49:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.910+0000 7fc245ffb640 1 -- 172.21.15.60:0/1183423677 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25801dd90 con 0x556000fa9aa0 2024-07-24T21:49:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.910+0000 7fc245ffb640 1 -- 172.21.15.60:0/1183423677 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258005610 con 0x556000fa9aa0 2024-07-24T21:49:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.912+0000 7fc268e06740 1 -- 172.21.15.60:0/1183423677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 msgr2=0x556000fa9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.912+0000 7fc268e06740 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc258021740 tx=0x7fc258021c00 comp rx=0 tx=0).stop 2024-07-24T21:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc268e06740 1 -- 172.21.15.60:0/1183423677 shutdown_connections 2024-07-24T21:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc268e06740 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc268e06740 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000faa3c0 0x556000fb5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc268e06740 1 --2- 172.21.15.60:0/1183423677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fa9e80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc268e06740 1 -- 172.21.15.60:0/1183423677 >> 172.21.15.60:0/1183423677 conn(0x556000fa78d0 msgr2=0x556000fa8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.913+0000 7fc25ffff640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 0x7fc2480b6410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc258021740 tx=0x7fc258021c00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.914+0000 7fc268e06740 1 -- 172.21.15.60:0/1183423677 shutdown_connections 2024-07-24T21:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.918+0000 7fc268e06740 1 -- 172.21.15.60:0/1183423677 wait complete. 2024-07-24T21:49:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.918+0000 7fc268e06740 1 -- start start 2024-07-24T21:49:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.918+0000 7fc268e06740 1 Processor -- start 2024-07-24T21:49:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556000e48ab0 con 0x556000fb6350 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556000e969e0 con 0x556000fa9aa0 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556000fb9030 con 0x556000faa3c0 2024-07-24T21:49:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.924+0000 7fc268e06740 10 client.0 _ll_register_callbacks cb 0x556000c1ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.927+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.927+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39476/0 (socket says 172.21.15.60:39476) 2024-07-24T21:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.927+0000 7fc25f7fe640 1 -- 172.21.15.60:0/2521879002 learned_addr learned my addr 172.21.15.60:0/2521879002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 -- 172.21.15.60:0/2521879002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 msgr2=0x556000fb5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 -- 172.21.15.60:0/2521879002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 msgr2=0x556000fb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.928+0000 7fc25effd640 1 -- 172.21.15.60:0/2521879002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556000ceb620 con 0x556000faa3c0 2024-07-24T21:49:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.929+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.962+0000 7fc25effd640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb2fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2540e0d40 tx=0x7fc2540a9c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d0bd60214028e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.966+0000 7fc2437f6640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.966+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254020040 con 0x556000faa3c0 2024-07-24T21:49:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.966+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556000ebdce0 con 0x556000faa3c0 2024-07-24T21:49:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.966+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc2540b1b80 con 0x556000faa3c0 2024-07-24T21:49:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.972+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2540aa900 con 0x556000faa3c0 2024-07-24T21:49:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.972+0000 7fc268e06740 10 client.25858 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.972+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556000fc48a0 con 0x556000faa3c0 2024-07-24T21:49:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556000ebdce0 con 0x556000faa3c0 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc2540a9460 con 0x556000faa3c0 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 10 client.25858 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 10 client.25858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 20 client.25858 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556000fc6300 0x556000fc8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 4 client.25858 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.975+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556000f211d0 con 0x556000fc6300 2024-07-24T21:49:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.980+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556000fc6300 0x556000fc8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:47.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:47 smithi123 ceph-mon[37143]: pgmap v470: 97 pgs: 97 active+clean; 444 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 569 op/s 2024-07-24T21:49:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.991+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556000fc6300 0x556000fc8720 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e260d28afbfc09 server_cookie=d489121d93e7bf86 in_seq=0 out_seq=0 2024-07-24T21:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.996+0000 7fc2437f6640 10 client.25858 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.996+0000 7fc2437f6640 1 -- 172.21.15.60:0/2521879002 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556000f211d0 con 0x556000fc6300 2024-07-24T21:49:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:47.996+0000 7fc2437f6640 10 client.25858 handle_command_reply: tid=0 2024-07-24T21:49:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.003+0000 7fc268e06740 1 client.25858 shutdown 2024-07-24T21:49:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.008+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 msgr2=0x556000fb2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.008+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb2fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2540e0d40 tx=0x7fc2540a9c10 comp rx=0 tx=0).stop 2024-07-24T21:49:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 shutdown_connections 2024-07-24T21:49:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556000fc6300 0x556000fc8720 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb2fd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 --2- 172.21.15.60:0/2521879002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fb5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.014+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 >> 172.21.15.60:0/2521879002 conn(0x556000fa78d0 msgr2=0x556000fad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.015+0000 7fc2437f6640 0 client.25858 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.015+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 shutdown_connections 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.015+0000 7fc268e06740 1 -- 172.21.15.60:0/2521879002 wait complete. 2024-07-24T21:49:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.015+0000 7fc268e06740 20 client.25858 trim_cache size 0 max 16384 2024-07-24T21:49:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.175+0000 7fc268e06740 1 Processor -- start 2024-07-24T21:49:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.175+0000 7fc268e06740 1 -- start start 2024-07-24T21:49:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.179+0000 7fc268e06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.179+0000 7fc268e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.180+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.180+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39480/0 (socket says 172.21.15.60:39480) 2024-07-24T21:49:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc25f7fe640 1 -- 172.21.15.60:0/4118366626 learned_addr learned my addr 172.21.15.60:0/4118366626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc268e06740 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc25effd640 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c011260 con 0x556000fb6350 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00ead0 con 0x556000fa9aa0 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.181+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00c920 con 0x556000faa3c0 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 -- 172.21.15.60:0/4118366626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 msgr2=0x556000fb34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 -- 172.21.15.60:0/4118366626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 msgr2=0x556000fcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 -- 172.21.15.60:0/4118366626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556000fa9370 con 0x556000fa9aa0 2024-07-24T21:49:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.182+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc250009830 tx=0x7fc25002a900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec11cda159842fd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.183+0000 7fc2457fa640 1 -- 172.21.15.60:0/4118366626 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250037040 con 0x556000fa9aa0 2024-07-24T21:49:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.183+0000 7fc2457fa640 1 -- 172.21.15.60:0/4118366626 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc250030df0 con 0x556000fa9aa0 2024-07-24T21:49:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.183+0000 7fc2457fa640 1 -- 172.21.15.60:0/4118366626 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2500046c0 con 0x556000fa9aa0 2024-07-24T21:49:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.185+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 msgr2=0x556000fc21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.185+0000 7fc268e06740 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc250009830 tx=0x7fc25002a900 comp rx=0 tx=0).stop 2024-07-24T21:49:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.186+0000 7fc25effd640 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:49:48.191 INFO:tasks.cephfs.filesystem:scrub status for tag:e25d7ad3-9469-4619-9972-71a284a6d5b8 - {'path': '/', 'tag': 'e25d7ad3-9469-4619-9972-71a284a6d5b8', 'options': 'recursive,force'} 2024-07-24T21:49:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 shutdown_connections 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000fb6350 0x556000fcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000faa3c0 0x556000fb34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 --2- 172.21.15.60:0/4118366626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fa9aa0 0x556000fc21e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 >> 172.21.15.60:0/4118366626 conn(0x556000fa78d0 msgr2=0x556000fa85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 shutdown_connections 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.194+0000 7fc268e06740 1 -- 172.21.15.60:0/4118366626 wait complete. 2024-07-24T21:49:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.195+0000 7fc268e06740 1 -- start start 2024-07-24T21:49:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.195+0000 7fc268e06740 1 Processor -- start 2024-07-24T21:49:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.196+0000 7fc268e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.196+0000 7fc268e06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000faa3c0 0x556000fbe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.196+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc268e06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fb6350 0x556000fb9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59060/0 (socket says 172.21.15.60:59060) 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc268e06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00cf90 con 0x556000faa3c0 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc268e06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00e400 con 0x556000fb6350 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc268e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00d660 con 0x556000fa9aa0 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc268e06740 10 client.0 _ll_register_callbacks cb 0x7fc2540e11f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 -- 172.21.15.60:0/668209572 learned_addr learned my addr 172.21.15.60:0/668209572 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25ffff640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fb6350 0x556000fb9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 -- 172.21.15.60:0/668209572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fb6350 msgr2=0x556000fb9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fb6350 0x556000fb9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 -- 172.21.15.60:0/668209572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000faa3c0 msgr2=0x556000fbe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000faa3c0 0x556000fbe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.197+0000 7fc25f7fe640 1 -- 172.21.15.60:0/668209572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556000fb98a0 con 0x556000fa9aa0 2024-07-24T21:49:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.198+0000 7fc25f7fe640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc250031000 tx=0x7fc250030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe60f623263b299 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.216+0000 7fc2437f6640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.218+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250054060 con 0x556000fa9aa0 2024-07-24T21:49:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.218+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc250030df0 con 0x556000fa9aa0 2024-07-24T21:49:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.219+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fc250005040 con 0x556000fa9aa0 2024-07-24T21:49:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.222+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc250037040 con 0x556000fa9aa0 2024-07-24T21:49:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.223+0000 7fc268e06740 10 client.25861 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.223+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55600102f770 con 0x556000fa9aa0 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc250030df0 con 0x556000fa9aa0 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc2500035f0 con 0x556000fa9aa0 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc268e06740 10 client.25861 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc268e06740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc268e06740 20 client.25861 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.225+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55600102fe80 0x5560010322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.226+0000 7fc268e06740 4 client.25861 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:49:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.226+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556000f1cc70 con 0x55600102fe80 2024-07-24T21:49:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.229+0000 7fc25effd640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55600102fe80 0x5560010322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.230+0000 7fc25effd640 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55600102fe80 0x5560010322a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b8cebd193a47d1 server_cookie=47a0d5cf81e84c8d in_seq=0 out_seq=0 2024-07-24T21:49:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:48.232+0000 7fc2437f6640 10 client.25861 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.684+0000 7f6538327640 1 -- 172.21.15.60:0/3847780727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530107060 msgr2=0x7f65301074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.684+0000 7f6538327640 1 --2- 172.21.15.60:0/3847780727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530107060 0x7f65301074a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f652c009860 tx=0x7f652c02a380 comp rx=0 tx=0).stop 2024-07-24T21:49:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.687+0000 7f6538327640 1 -- 172.21.15.60:0/3847780727 shutdown_connections 2024-07-24T21:49:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.687+0000 7f6538327640 1 --2- 172.21.15.60:0/3847780727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 0x7f653006a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.687+0000 7f6538327640 1 --2- 172.21.15.60:0/3847780727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530107060 0x7f65301074a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.687+0000 7f6538327640 1 --2- 172.21.15.60:0/3847780727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530101650 0x7f6530101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.687+0000 7f6538327640 1 -- 172.21.15.60:0/3847780727 >> 172.21.15.60:0/3847780727 conn(0x7f65300fd0c0 msgr2=0x7f65300ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.688+0000 7f6538327640 1 -- 172.21.15.60:0/3847780727 shutdown_connections 2024-07-24T21:49:48.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.688+0000 7f6538327640 1 -- 172.21.15.60:0/3847780727 wait complete. 2024-07-24T21:49:48.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.689+0000 7f6538327640 1 Processor -- start 2024-07-24T21:49:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.693+0000 7f6538327640 1 -- start start 2024-07-24T21:49:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.693+0000 7f6538327640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.693+0000 7f6538327640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 0x7f6530109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.693+0000 7f6538327640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 0x7f65301aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.693+0000 7f65360c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.694+0000 7f6538327640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6530110660 con 0x7f6530107060 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.694+0000 7f6538327640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65301104e0 con 0x7f6530101650 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.694+0000 7f6538327640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65301107e0 con 0x7f65301079e0 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.694+0000 7f65360c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39514/0 (socket says 172.21.15.60:39514) 2024-07-24T21:49:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.694+0000 7f65360c3640 1 -- 172.21.15.60:0/55928235 learned_addr learned my addr 172.21.15.60:0/55928235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.695+0000 7f65368c4640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 0x7f65301aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.696+0000 7f65358c2640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 0x7f6530109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.697+0000 7f65358c2640 1 -- 172.21.15.60:0/55928235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 msgr2=0x7f65301aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.697+0000 7f65358c2640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 0x7f65301aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.697+0000 7f65358c2640 1 -- 172.21.15.60:0/55928235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 msgr2=0x7f6530109650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.697+0000 7f65358c2640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.697+0000 7f65358c2640 1 -- 172.21.15.60:0/55928235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301ab3c0 con 0x7f6530107060 2024-07-24T21:49:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.700+0000 7f65360c3640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:49:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.700+0000 7f65358c2640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 0x7f6530109b90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f652c009780 tx=0x7f652c030720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.701+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c037050 con 0x7f6530107060 2024-07-24T21:49:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.701+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f652c038040 con 0x7f6530107060 2024-07-24T21:49:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.701+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c003b20 con 0x7f6530107060 2024-07-24T21:49:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.701+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65301ab650 con 0x7f6530107060 2024-07-24T21:49:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.701+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301aba20 con 0x7f6530107060 2024-07-24T21:49:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.704+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653010a780 con 0x7f6530107060 2024-07-24T21:49:48.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.706+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f652c045020 con 0x7f6530107060 2024-07-24T21:49:48.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.708+0000 7f65277fe640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 0x7f651c0b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.709+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f652c0c7240 con 0x7f6530107060 2024-07-24T21:49:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.710+0000 7f65360c3640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 0x7f651c0b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.711+0000 7f65360c3640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 0x7f651c0b63b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6528010410 tx=0x7f65280095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:48.719+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f652c04c020 con 0x7f6530107060 2024-07-24T21:49:49.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.275+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65301ac050 con 0x7f6530107060 2024-07-24T21:49:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.277+0000 7f65277fe640 1 -- 172.21.15.60:0/55928235 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f652c0cb350 con 0x7f6530107060 2024-07-24T21:49:49.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:49:49.281 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"btime":"2024-07-24T21:48:44:369573+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:48:44.225462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:49:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-07-24T21:49:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.282+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 msgr2=0x7f651c0b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.283+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 0x7f651c0b63b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6528010410 tx=0x7f65280095c0 comp rx=0 tx=0).stop 2024-07-24T21:49:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.283+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 msgr2=0x7f6530109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.283+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 0x7f6530109b90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f652c009780 tx=0x7f652c030720 comp rx=0 tx=0).stop 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 shutdown_connections 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f651c0b3ef0 0x7f651c0b63b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65301079e0 0x7f65301aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6530107060 0x7f6530109b90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 --2- 172.21.15.60:0/55928235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6530101650 0x7f6530109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.284+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 >> 172.21.15.60:0/55928235 conn(0x7f65300fd0c0 msgr2=0x7f6530104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.288+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 shutdown_connections 2024-07-24T21:49:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:49.291+0000 7f6538327640 1 -- 172.21.15.60:0/55928235 wait complete. 2024-07-24T21:49:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:49 smithi086 ceph-mon[37128]: pgmap v471: 97 pgs: 97 active+clean; 452 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 598 op/s 2024-07-24T21:49:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:49 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/55928235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:49 smithi060 ceph-mon[33000]: pgmap v471: 97 pgs: 97 active+clean; 452 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 598 op/s 2024-07-24T21:49:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:49 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/55928235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:49.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:49 smithi123 ceph-mon[37143]: pgmap v471: 97 pgs: 97 active+clean; 452 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 598 op/s 2024-07-24T21:49:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:49 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/55928235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:49:50.007 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-24T21:49:50.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-24T21:49:50.696 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:49:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.451+0000 7fc2437f6640 1 -- 172.21.15.60:0/668209572 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556000f1cc70 con 0x55600102fe80 2024-07-24T21:49:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.451+0000 7fc2437f6640 10 client.25861 handle_command_reply: tid=0 2024-07-24T21:49:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.451+0000 7fc268e06740 1 client.25861 shutdown 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 msgr2=0x556000fbe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc250031000 tx=0x7fc250030d50 comp rx=0 tx=0).stop 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 shutdown_connections 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55600102fe80 0x5560010322a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556000fb6350 0x556000fb9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556000faa3c0 0x556000fbe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc2437f6640 0 client.25861 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 --2- 172.21.15.60:0/668209572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556000fa9aa0 0x556000fbe360 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 >> 172.21.15.60:0/668209572 conn(0x556000fa78d0 msgr2=0x556000fadcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 shutdown_connections 2024-07-24T21:49:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 1 -- 172.21.15.60:0/668209572 wait complete. 2024-07-24T21:49:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.452+0000 7fc268e06740 20 client.25861 trim_cache size 0 max 16384 2024-07-24T21:49:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:49:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:49:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:49:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.453+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 msgr2=0x7fc2480b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.453+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 0x7fc2480b6410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc258021740 tx=0x7fc258021c00 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.453+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 msgr2=0x7fc260117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.453+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc260117810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d650 tx=0x7fc25400db10 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 shutdown_connections 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc2480b3f50 0x7fc2480b6410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc26011ba90 0x7fc2601a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc260109f20 0x7fc260117810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 --2- 172.21.15.60:0/2478978666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc260109b40 0x7fc2601172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 >> 172.21.15.60:0/2478978666 conn(0x7fc26006d5a0 msgr2=0x7fc260070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 shutdown_connections 2024-07-24T21:49:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:49:51.454+0000 7fc265fd0640 1 -- 172.21.15.60:0/2478978666 wait complete. 2024-07-24T21:49:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:51 smithi060 ceph-mon[33000]: pgmap v472: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 10 MiB/s wr, 895 op/s 2024-07-24T21:49:51.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:51 smithi123 ceph-mon[37143]: pgmap v472: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 10 MiB/s wr, 895 op/s 2024-07-24T21:49:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:51 smithi086 ceph-mon[37128]: pgmap v472: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 10 MiB/s wr, 895 op/s 2024-07-24T21:49:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1503816340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 msgr2=0x7f8a101074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1503816340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a101074a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009900 tx=0x7f8a0402a410 comp rx=0 tx=0).stop 2024-07-24T21:49:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1503816340 shutdown_connections 2024-07-24T21:49:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1503816340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a1010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1503816340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a101074a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1503816340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 0x7f8a10101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1503816340 >> 172.21.15.60:0/1503816340 conn(0x7f8a100fd0c0 msgr2=0x7f8a100ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1503816340 shutdown_connections 2024-07-24T21:49:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.950+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1503816340 wait complete. 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.951+0000 7f8a15ea1640 1 Processor -- start 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.951+0000 7f8a15ea1640 1 -- start start 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 0x7f8a100fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a100fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10111600 con 0x7f8a10107060 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10111480 con 0x7f8a101079e0 2024-07-24T21:49:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a15ea1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10111780 con 0x7f8a10101650 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58724/0 (socket says 172.21.15.60:58724) 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a100fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 0x7f8a100fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 -- 172.21.15.60:0/1806118938 learned_addr learned my addr 172.21.15.60:0/1806118938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 -- 172.21.15.60:0/1806118938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 msgr2=0x7f8a100fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 0x7f8a100fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 -- 172.21.15.60:0/1806118938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 msgr2=0x7f8a100fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a100fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0effd640 1 -- 172.21.15.60:0/1806118938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a101a7090 con 0x7f8a10107060 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.952+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a100fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a0effd640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005940 tx=0x7f8a040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a0cff9640 1 -- 172.21.15.60:0/1806118938 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0403f030 con 0x7f8a10107060 2024-07-24T21:49:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a101a7320 con 0x7f8a10107060 2024-07-24T21:49:52.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a0cff9640 1 -- 172.21.15.60:0/1806118938 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a040044c0 con 0x7f8a10107060 2024-07-24T21:49:52.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a0cff9640 1 -- 172.21.15.60:0/1806118938 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04035070 con 0x7f8a10107060 2024-07-24T21:49:52.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.953+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101a7850 con 0x7f8a10107060 2024-07-24T21:49:52.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0cff9640 1 -- 172.21.15.60:0/1806118938 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8a04042030 con 0x7f8a10107060 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 Processor -- start 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 -- start start 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5a97a0 0x5574ff5a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5b55d0 0x5574ff5b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5574ff51f6a0 con 0x5574ff5b55d0 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574ff50eba0 con 0x5574ff5aad00 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a18252740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5574ff512be0 con 0x5574ff5a97a0 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5b55d0 0x5574ff5b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39552/0 (socket says 172.21.15.60:39552) 2024-07-24T21:49:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0ffff640 1 -- 172.21.15.60:0/2158652539 learned_addr learned my addr 172.21.15.60:0/2158652539 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.956+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5a97a0 0x5574ff5a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 -- 172.21.15.60:0/2158652539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5a97a0 msgr2=0x5574ff5a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5a97a0 0x5574ff5a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 -- 172.21.15.60:0/2158652539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5b55d0 msgr2=0x5574ff5b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5b55d0 0x5574ff5b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 -- 172.21.15.60:0/2158652539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ff4a6d60 con 0x5574ff5aad00 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0cff9640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 0x7f89f80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 0x7f89f80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a08011470 tx=0x7f8a08011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dcdcb6f4c983323 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f8a0cff9640 1 -- 172.21.15.60:0/1806118938 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8a04104320 con 0x7f8a10107060 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f89f5ffb640 1 -- 172.21.15.60:0/2158652539 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004020 con 0x5574ff5aad00 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f89f5ffb640 1 -- 172.21.15.60:0/2158652539 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080041c0 con 0x5574ff5aad00 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.957+0000 7f89f5ffb640 1 -- 172.21.15.60:0/2158652539 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800f7c0 con 0x5574ff5aad00 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 -- 172.21.15.60:0/2158652539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 msgr2=0x5574ff5a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 0x7f89f80b6200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5574ff5ac3e0 tx=0x7f8a00009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a08011470 tx=0x7f8a08011930 comp rx=0 tx=0).stop 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 -- 172.21.15.60:0/2158652539 shutdown_connections 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5b55d0 0x5574ff5b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5a97a0 0x5574ff5a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 --2- 172.21.15.60:0/2158652539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5aad00 0x5574ff5a9260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 -- 172.21.15.60:0/2158652539 >> 172.21.15.60:0/2158652539 conn(0x5574ff573750 msgr2=0x5574ff5a7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 -- 172.21.15.60:0/2158652539 shutdown_connections 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.958+0000 7f8a18252740 1 -- 172.21.15.60:0/2158652539 wait complete. 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 -- start start 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 Processor -- start 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 0x5574ff5b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 0x5574ff5b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5574ff3727a0 con 0x5574ff5aad00 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574ff4b1020 con 0x5574ff5a97a0 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5574ff5b8490 con 0x5574ff5b55d0 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 0x5574ff5b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 0x5574ff5b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58732/0 (socket says 172.21.15.60:58732) 2024-07-24T21:49:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a0effd640 1 -- 172.21.15.60:0/1941629509 learned_addr learned my addr 172.21.15.60:0/1941629509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.959+0000 7f8a18252740 10 client.0 _ll_register_callbacks cb 0x5574ff21c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 -- 172.21.15.60:0/1941629509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 msgr2=0x5574ff5b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 0x5574ff5b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 -- 172.21.15.60:0/1941629509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 msgr2=0x5574ff5b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 -- 172.21.15.60:0/1941629509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ff4b50d0 con 0x5574ff5b55d0 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0effd640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 0x5574ff5b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 0x5574ff5b2fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a0801b040 tx=0x7f8a08005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b7c6ca45db2b25 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f89eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08028020 con 0x5574ff5b55d0 2024-07-24T21:49:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a080041c0 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8a08016040 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a08033050 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a18252740 10 client.25876 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574ff27eb90 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.960+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080041c0 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a08004a90 con 0x5574ff5b55d0 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 10 client.25876 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 10 client.25876 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 20 client.25876 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff5c5640 0x5574ff5c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 4 client.25876 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:49:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ff5215f0 con 0x5574ff5c5640 2024-07-24T21:49:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff5c5640 0x5574ff5c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff5c5640 0x5574ff5c7a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5dbb2853e1365a server_cookie=51c81aba898683e5 in_seq=0 out_seq=0 2024-07-24T21:49:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.961+0000 7f89eeffd640 10 client.25876 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.962+0000 7f89eeffd640 1 -- 172.21.15.60:0/1941629509 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574ff5215f0 con 0x5574ff5c5640 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f89eeffd640 10 client.25876 handle_command_reply: tid=0 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 client.25876 shutdown 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 msgr2=0x5574ff5b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 0x5574ff5b2fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a0801b040 tx=0x7f8a08005720 comp rx=0 tx=0).stop 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 shutdown_connections 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff5c5640 0x5574ff5c7a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5b55d0 0x5574ff5b2fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5aad00 0x5574ff5b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 --2- 172.21.15.60:0/1941629509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5a97a0 0x5574ff5b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f89eeffd640 0 client.25876 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 >> 172.21.15.60:0/1941629509 conn(0x5574ff573750 msgr2=0x5574ff5acc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 shutdown_connections 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 1 -- 172.21.15.60:0/1941629509 wait complete. 2024-07-24T21:49:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.963+0000 7f8a18252740 20 client.25876 trim_cache size 0 max 16384 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.983+0000 7f8a18252740 1 Processor -- start 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.983+0000 7f8a18252740 1 -- start start 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b2c90 con 0x5574ff5a97a0 2024-07-24T21:49:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b0500 con 0x5574ff5b55d0 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a18252740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ae350 con 0x5574ff5aad00 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58748/0 (socket says 172.21.15.60:58748) 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0f7fe640 1 -- 172.21.15.60:0/472700938 learned_addr learned my addr 172.21.15.60:0/472700938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 -- 172.21.15.60:0/472700938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 msgr2=0x5574ff5b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 -- 172.21.15.60:0/472700938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 msgr2=0x5574ff5a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 -- 172.21.15.60:0/472700938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ff5b7f20 con 0x5574ff5aad00 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f8a0effd640 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5b0dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a040c8250 tx=0x7f8a04107e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9cf6b4ed9fa5f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.984+0000 7f89f57fa640 1 -- 172.21.15.60:0/472700938 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04049040 con 0x5574ff5aad00 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f89f57fa640 1 -- 172.21.15.60:0/472700938 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a040caaf0 con 0x5574ff5aad00 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f89f57fa640 1 -- 172.21.15.60:0/472700938 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040cf530 con 0x5574ff5aad00 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 -- 172.21.15.60:0/472700938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 msgr2=0x5574ff5b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5b0dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a040c8250 tx=0x7f8a04107e70 comp rx=0 tx=0).stop 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 -- 172.21.15.60:0/472700938 shutdown_connections 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5b0dd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 --2- 172.21.15.60:0/472700938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 -- 172.21.15.60:0/472700938 >> 172.21.15.60:0/472700938 conn(0x5574ff573750 msgr2=0x5574ff5b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 -- 172.21.15.60:0/472700938 shutdown_connections 2024-07-24T21:49:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.985+0000 7f8a18252740 1 -- 172.21.15.60:0/472700938 wait complete. 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 -- start start 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 Processor -- start 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ae9c0 con 0x5574ff5a97a0 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101afe30 con 0x5574ff5b55d0 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101af090 con 0x5574ff5aad00 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a18252740 10 client.0 _ll_register_callbacks cb 0x7f8a04107d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a0effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59134/0 (socket says 172.21.15.60:59134) 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a0effd640 1 -- 172.21.15.60:0/7153711 learned_addr learned my addr 172.21.15.60:0/7153711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:49:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.986+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58760/0 (socket says 172.21.15.60:58760) 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 -- 172.21.15.60:0/7153711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 msgr2=0x5574ff5af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 -- 172.21.15.60:0/7153711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 msgr2=0x5574ff5bda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 -- 172.21.15.60:0/7153711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ff5b0200 con 0x5574ff5aad00 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0ffff640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a0effd640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a04102070 tx=0x7f8a041078b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e961b9dc78688c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f89eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040e4070 con 0x5574ff5aad00 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8a040cb240 con 0x5574ff5aad00 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a040caaf0 con 0x5574ff5aad00 2024-07-24T21:49:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a04049050 con 0x5574ff5aad00 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a18252740 10 client.25882 fetch_fsmap learned FSMap version 29 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.987+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574ff5cd6f0 con 0x5574ff5aad00 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a040caaf0 con 0x5574ff5aad00 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a040d3b50 con 0x5574ff5aad00 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 10 client.25882 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 10 client.25882 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:49:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 20 client.25882 populate_metadata read hostname 'smithi060' 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff6307b0 0x5574ff632bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 4 client.25882 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5574ff51d090 con 0x5574ff6307b0 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff6307b0 0x5574ff632bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f89eeffd640 10 client.25882 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:49:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:49:52.988+0000 7f8a0f7fe640 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff6307b0 0x5574ff632bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f5499a2b91ca08 server_cookie=6e60e5437d55897a in_seq=0 out_seq=0 2024-07-24T21:49:53.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:53 smithi060 ceph-mon[33000]: pgmap v473: 97 pgs: 97 active+clean; 604 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-07-24T21:49:53.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:53 smithi123 ceph-mon[37143]: pgmap v473: 97 pgs: 97 active+clean; 604 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-07-24T21:49:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:53 smithi086 ceph-mon[37128]: pgmap v473: 97 pgs: 97 active+clean; 604 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-07-24T21:49:54.708 INFO:tasks.workunit.client.0.smithi060.stdout: 174 750255 633 513714 388 182242 405 2024-07-24T21:49:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:55 smithi060 ceph-mon[33000]: pgmap v474: 97 pgs: 97 active+clean; 626 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2024-07-24T21:49:55.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:55 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:55 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:55 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:49:55.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:55 smithi123 ceph-mon[37143]: pgmap v474: 97 pgs: 97 active+clean; 626 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2024-07-24T21:49:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:55 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:55 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:55 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:49:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:55 smithi086 ceph-mon[37128]: pgmap v474: 97 pgs: 97 active+clean; 626 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2024-07-24T21:49:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:55 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:55 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:55 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:49:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:57 smithi060 ceph-mon[33000]: pgmap v475: 97 pgs: 97 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2024-07-24T21:49:57.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:57 smithi123 ceph-mon[37143]: pgmap v475: 97 pgs: 97 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2024-07-24T21:49:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:57 smithi086 ceph-mon[37128]: pgmap v475: 97 pgs: 97 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.84k op/s 2024-07-24T21:49:59.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:49:59 smithi060 ceph-mon[33000]: pgmap v476: 97 pgs: 97 active+clean; 630 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 18 MiB/s wr, 1.82k op/s 2024-07-24T21:49:59.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:49:59 smithi123 ceph-mon[37143]: pgmap v476: 97 pgs: 97 active+clean; 630 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 18 MiB/s wr, 1.82k op/s 2024-07-24T21:50:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:49:59 smithi086 ceph-mon[37128]: pgmap v476: 97 pgs: 97 active+clean; 630 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 18 MiB/s wr, 1.82k op/s 2024-07-24T21:50:00.560 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[69167]: *** Caught signal (Segmentation fault) ** 2024-07-24T21:50:00.561 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[69167]: in thread 7f1f81b7f640 thread_name:msgr-worker-0 2024-07-24T21:50:00.561 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:00 smithi123 podman[72268]: 2024-07-24 21:50:00.327141566 +0000 UTC m=+0.023283845 container died 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=main-86385b1, GIT_BRANCH=HEAD) 2024-07-24T21:50:00.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:00 smithi060 ceph-mon[33000]: overall HEALTH_OK 2024-07-24T21:50:00.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:00 smithi123 ceph-mon[37143]: overall HEALTH_OK 2024-07-24T21:50:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:00 smithi086 ceph-mon[37128]: overall HEALTH_OK 2024-07-24T21:50:01.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:01 smithi060 ceph-mon[33000]: pgmap v477: 97 pgs: 97 active+clean; 645 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-07-24T21:50:01.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:01 smithi123 ceph-mon[37143]: pgmap v477: 97 pgs: 97 active+clean; 645 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-07-24T21:50:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:01 smithi086 ceph-mon[37128]: pgmap v477: 97 pgs: 97 active+clean; 645 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-07-24T21:50:03.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:03 smithi060 ceph-mon[33000]: pgmap v478: 97 pgs: 97 active+clean; 659 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 1.58k op/s 2024-07-24T21:50:03.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:03 smithi123 ceph-mon[37143]: pgmap v478: 97 pgs: 97 active+clean; 659 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 1.58k op/s 2024-07-24T21:50:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:03 smithi086 ceph-mon[37128]: pgmap v478: 97 pgs: 97 active+clean; 659 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 1.58k op/s 2024-07-24T21:50:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- 172.21.15.60:0/2424257325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f875810ca80 msgr2=0x7f875810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- 172.21.15.60:0/2424257325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f875810ca80 0x7f875810ee70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8750009ca0 tx=0x7f875002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- 172.21.15.60:0/2424257325 shutdown_connections 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- 172.21.15.60:0/2424257325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f875810ca80 0x7f875810ee70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- 172.21.15.60:0/2424257325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8758103e40 0x7f875810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- 172.21.15.60:0/2424257325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- 172.21.15.60:0/2424257325 >> 172.21.15.60:0/2424257325 conn(0x7f87580782e0 msgr2=0x7f87580ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- 172.21.15.60:0/2424257325 shutdown_connections 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- 172.21.15.60:0/2424257325 wait complete. 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 Processor -- start 2024-07-24T21:50:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 -- start start 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.563+0000 7f875ea7f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875810ca80 0x7f8758101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875ea7f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112650 con 0x7f8758103e40 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875ea7f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87581124d0 con 0x7f8758103440 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875ea7f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87581127d0 con 0x7f875810ca80 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49824/0 (socket says 172.21.15.60:49824) 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 -- 172.21.15.60:0/1350947048 learned_addr learned my addr 172.21.15.60:0/1350947048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875c81b640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875d01c640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875810ca80 0x7f8758101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 -- 172.21.15.60:0/1350947048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875810ca80 msgr2=0x7f8758101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875810ca80 0x7f8758101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 -- 172.21.15.60:0/1350947048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 msgr2=0x7f8758101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 -- 172.21.15.60:0/1350947048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87581a6fc0 con 0x7f8758103e40 2024-07-24T21:50:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f875c81b640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.564+0000 7f8757fff640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.565+0000 7f8755ffb640 1 -- 172.21.15.60:0/1350947048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400cd80 con 0x7f8758103e40 2024-07-24T21:50:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.565+0000 7f8755ffb640 1 -- 172.21.15.60:0/1350947048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440054d0 con 0x7f8758103e40 2024-07-24T21:50:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.565+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87581a72b0 con 0x7f8758103e40 2024-07-24T21:50:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.565+0000 7f8755ffb640 1 -- 172.21.15.60:0/1350947048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744010690 con 0x7f8758103e40 2024-07-24T21:50:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.566+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a77b0 con 0x7f8758103e40 2024-07-24T21:50:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.568+0000 7f8755ffb640 1 -- 172.21.15.60:0/1350947048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8744004e90 con 0x7f8758103e40 2024-07-24T21:50:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.568+0000 7f87617d5740 1 Processor -- start 2024-07-24T21:50:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.568+0000 7f87617d5740 1 -- start start 2024-07-24T21:50:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e866c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e867170 0x56221e869560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56221e7d0280 con 0x56221e85a8c0 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56221e7bf780 con 0x56221e85b1e0 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f87617d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56221e7c37c0 con 0x56221e867170 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49836/0 (socket says 172.21.15.60:49836) 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 -- 172.21.15.60:0/3609029093 learned_addr learned my addr 172.21.15.60:0/3609029093 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f8757fff640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e867170 0x56221e869560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875c81b640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e866c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f8755ffb640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 0x7f87480b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 -- 172.21.15.60:0/3609029093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e867170 msgr2=0x56221e869560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e867170 0x56221e869560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 -- 172.21.15.60:0/3609029093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 msgr2=0x56221e866c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e866c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 -- 172.21.15.60:0/3609029093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221e523790 con 0x56221e85a8c0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875c81b640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e866c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f8755ffb640 1 -- 172.21.15.60:0/1350947048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f87440e4440 con 0x7f8758103e40 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 0x7f87480b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.569+0000 7f875d01c640 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8750035000 tx=0x7f8750005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0607287d6807c04 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f8742ffd640 1 -- 172.21.15.60:0/3609029093 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750042040 con 0x56221e85a8c0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f8742ffd640 1 -- 172.21.15.60:0/3609029093 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8750036070 con 0x56221e85a8c0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f8742ffd640 1 -- 172.21.15.60:0/3609029093 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750004650 con 0x56221e85a8c0 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 -- 172.21.15.60:0/3609029093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 msgr2=0x56221e85aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f875d01c640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 0x7f87480b6200 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f875004d000 tx=0x7f875004e000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8750035000 tx=0x7f8750005fa0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 -- 172.21.15.60:0/3609029093 shutdown_connections 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e867170 0x56221e869560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e866c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.570+0000 7f87617d5740 1 --2- 172.21.15.60:0/3609029093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e85a8c0 0x56221e85aca0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- 172.21.15.60:0/3609029093 >> 172.21.15.60:0/3609029093 conn(0x56221e8586f0 msgr2=0x56221e859890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- 172.21.15.60:0/3609029093 shutdown_connections 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- 172.21.15.60:0/3609029093 wait complete. 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- start start 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 Processor -- start 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e8668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e863dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56221e6f9ab0 con 0x56221e867170 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56221e7479e0 con 0x56221e85b1e0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56221e869e50 con 0x56221e85a8c0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.571+0000 7f87617d5740 10 client.0 _ll_register_callbacks cb 0x56221e4cfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49846/0 (socket says 172.21.15.60:49846) 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 -- 172.21.15.60:0/1154977662 learned_addr learned my addr 172.21.15.60:0/1154977662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875c81b640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e8668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f8757fff640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e863dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 -- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 msgr2=0x56221e8668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e8668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 -- 172.21.15.60:0/1154977662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 msgr2=0x56221e863dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e863dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 -- 172.21.15.60:0/1154977662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221e59c620 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f875d01c640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87500047c0 tx=0x7f87500093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8f5fc28975b9f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f8737fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750042070 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8750003b70 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56221e76ece0 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8750062050 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f87617d5740 10 client.16395 fetch_fsmap learned FSMap version 29 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.572+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56221e875630 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56221e76ece0 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87500305c0 con 0x56221e867170 2024-07-24T21:50:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 10 client.16395 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 20 client.16395 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e876980 0x56221e878d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 4 client.16395 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56221e7d21d0 con 0x56221e876980 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.573+0000 7f875c81b640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e876980 0x56221e878d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.574+0000 7f8737fff640 10 client.16395 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.574+0000 7f875c81b640 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e876980 0x56221e878d70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d832b715f25c9c9d server_cookie=10bc27509e81e196 in_seq=0 out_seq=0 2024-07-24T21:50:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.575+0000 7f8737fff640 1 -- 172.21.15.60:0/1154977662 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56221e7d21d0 con 0x56221e876980 2024-07-24T21:50:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.575+0000 7f8737fff640 10 client.16395 handle_command_reply: tid=0 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.575+0000 7f87617d5740 1 client.16395 shutdown 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 msgr2=0x56221e864300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87500047c0 tx=0x7f87500093a0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 shutdown_connections 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e876980 0x56221e878d70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e864300 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e863dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f8737fff640 0 client.16395 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 --2- 172.21.15.60:0/1154977662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e8668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 >> 172.21.15.60:0/1154977662 conn(0x56221e8586f0 msgr2=0x56221e85e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 shutdown_connections 2024-07-24T21:50:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 1 -- 172.21.15.60:0/1154977662 wait complete. 2024-07-24T21:50:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.576+0000 7f87617d5740 20 client.16395 trim_cache size 0 max 16384 2024-07-24T21:50:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 Processor -- start 2024-07-24T21:50:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 -- start start 2024-07-24T21:50:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 0x56221e872fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85b1e0 0x56221e8734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87581b2b30 con 0x56221e867170 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87581b03a0 con 0x56221e85a8c0 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f87617d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87581ae1f0 con 0x56221e85b1e0 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49854/0 (socket says 172.21.15.60:49854) 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 -- 172.21.15.60:0/315392696 learned_addr learned my addr 172.21.15.60:0/315392696 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f8757fff640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85b1e0 0x56221e8734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875c81b640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 0x56221e872fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 -- 172.21.15.60:0/315392696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85b1e0 msgr2=0x56221e8734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85b1e0 0x56221e8734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 -- 172.21.15.60:0/315392696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 msgr2=0x56221e872fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 0x56221e872fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875d01c640 1 -- 172.21.15.60:0/315392696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221e863ba0 con 0x56221e867170 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.593+0000 7f875c81b640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 0x56221e872fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f875d01c640 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f875002ade0 tx=0x7f8750009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f443cc874d48d2e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87427fc640 1 -- 172.21.15.60:0/315392696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750042040 con 0x56221e867170 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87427fc640 1 -- 172.21.15.60:0/315392696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875004b020 con 0x56221e867170 2024-07-24T21:50:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87427fc640 1 -- 172.21.15.60:0/315392696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875002f660 con 0x56221e867170 2024-07-24T21:50:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 -- 172.21.15.60:0/315392696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 msgr2=0x56221e862620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f875002ade0 tx=0x7f8750009c20 comp rx=0 tx=0).stop 2024-07-24T21:50:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 -- 172.21.15.60:0/315392696 shutdown_connections 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e862620 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85b1e0 0x56221e8734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 --2- 172.21.15.60:0/315392696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85a8c0 0x56221e872fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 -- 172.21.15.60:0/315392696 >> 172.21.15.60:0/315392696 conn(0x56221e8586f0 msgr2=0x56221e863d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 -- 172.21.15.60:0/315392696 shutdown_connections 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.594+0000 7f87617d5740 1 -- 172.21.15.60:0/315392696 wait complete. 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 -- start start 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 Processor -- start 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e86f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e86ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87581ae860 con 0x56221e867170 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87581afcd0 con 0x56221e85b1e0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87581aef30 con 0x56221e85a8c0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f87617d5740 10 client.0 _ll_register_callbacks cb 0x7f874c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f875c81b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e86f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f875c81b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e86f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:57008/0 (socket says 172.21.15.60:57008) 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f875d01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49862/0 (socket says 172.21.15.60:49862) 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.595+0000 7f875c81b640 1 -- 172.21.15.60:0/2472382237 learned_addr learned my addr 172.21.15.60:0/2472382237 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8757fff640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e86ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 -- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 msgr2=0x56221e86f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e86f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 -- 172.21.15.60:0/2472382237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 msgr2=0x56221e86ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e86ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 -- 172.21.15.60:0/2472382237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221e869980 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8757fff640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e86ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f875d01c640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8750005e30 tx=0x7f8750031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6de2bc56ca16ff2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8737fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875006a020 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f87500305c0 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875004b020 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8750042040 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f87617d5740 10 client.16401 fetch_fsmap learned FSMap version 29 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56221e8e0010 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.596+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875004b020 con 0x56221e867170 2024-07-24T21:50:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8750065070 con 0x56221e867170 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 10 client.16401 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 20 client.16401 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e8e0570 0x56221e8e2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 4 client.16401 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56221e7cdc70 con 0x56221e8e0570 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.597+0000 7f875c81b640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e8e0570 0x56221e8e2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.598+0000 7f875c81b640 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e8e0570 0x56221e8e2990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aad8e6bc60d2eb5 server_cookie=6173e0bc7780d68c in_seq=0 out_seq=0 2024-07-24T21:50:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:04.598+0000 7f8737fff640 10 client.16401 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:04.708 INFO:tasks.workunit.client.0.smithi060.stdout: 174 3636 0 3400 0 872 2 2024-07-24T21:50:05.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:05 smithi060 ceph-mon[33000]: pgmap v479: 97 pgs: 97 active+clean; 680 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 940 op/s 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f89eeffd640 1 -- 172.21.15.60:0/7153711 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5574ff51d090 con 0x5574ff6307b0 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f89eeffd640 10 client.25882 handle_command_reply: tid=0 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 client.25882 shutdown 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 msgr2=0x5574ff5be010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a04102070 tx=0x7f8a041078b0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 shutdown_connections 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5574ff6307b0 0x5574ff632bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574ff5b55d0 0x5574ff5af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ff5aad00 0x5574ff5be010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 --2- 172.21.15.60:0/7153711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574ff5a97a0 0x5574ff5bda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f89eeffd640 0 client.25882 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 >> 172.21.15.60:0/7153711 conn(0x5574ff573750 msgr2=0x5574ff5ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 shutdown_connections 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.855+0000 7f8a18252740 1 -- 172.21.15.60:0/7153711 wait complete. 2024-07-24T21:50:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a18252740 20 client.25882 trim_cache size 0 max 16384 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 29, 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 10, 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "15b6c0a6": { 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 10, 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:05.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:05.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:05.861 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 msgr2=0x7f89f80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 0x7f89f80b6200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5574ff5ac3e0 tx=0x7f8a00009500 comp rx=0 tx=0).stop 2024-07-24T21:50:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 msgr2=0x7f8a100fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005940 tx=0x7f8a040040c0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 shutdown_connections 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f89f80b3d40 0x7f89f80b6200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101079e0 0x7f8a100fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a10107060 0x7f8a100fdb00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 --2- 172.21.15.60:0/1806118938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a10101650 0x7f8a100fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.856+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 >> 172.21.15.60:0/1806118938 conn(0x7f8a100fd0c0 msgr2=0x7f8a1006a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.857+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 shutdown_connections 2024-07-24T21:50:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:05.857+0000 7f8a15ea1640 1 -- 172.21.15.60:0/1806118938 wait complete. 2024-07-24T21:50:05.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:05 smithi123 ceph-mon[37143]: pgmap v479: 97 pgs: 97 active+clean; 680 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 940 op/s 2024-07-24T21:50:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:05 smithi086 ceph-mon[37128]: pgmap v479: 97 pgs: 97 active+clean; 680 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 940 op/s 2024-07-24T21:50:06.633 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '15b6c0a6', quiesce took 25.5 seconds. Will release after: 4.5 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f8737fff640 1 -- 172.21.15.60:0/2472382237 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56221e7cdc70 con 0x56221e8e0570 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f8737fff640 10 client.16401 handle_command_reply: tid=0 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 client.16401 shutdown 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 msgr2=0x56221e86bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8750005e30 tx=0x7f8750031a30 comp rx=0 tx=0).stop 2024-07-24T21:50:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 shutdown_connections 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56221e8e0570 0x56221e8e2990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221e867170 0x56221e86bf80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f8737fff640 0 client.16401 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221e85b1e0 0x56221e86ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 --2- 172.21.15.60:0/2472382237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56221e85a8c0 0x56221e86f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 >> 172.21.15.60:0/2472382237 conn(0x56221e8586f0 msgr2=0x56221e863d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.731+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 shutdown_connections 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.732+0000 7f87617d5740 1 -- 172.21.15.60:0/2472382237 wait complete. 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.732+0000 7f87617d5740 20 client.16401 trim_cache size 0 max 16384 2024-07-24T21:50:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:50:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:50:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:50:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:50:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 msgr2=0x7f87480b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 0x7f87480b6200 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f875004d000 tx=0x7f875004e000 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 msgr2=0x7f8758101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400edf0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 shutdown_connections 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f87480b3d40 0x7f87480b6200 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875810ca80 0x7f8758101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8758103e40 0x7f8758101890 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 --2- 172.21.15.60:0/1350947048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8758103440 0x7f8758101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 >> 172.21.15.60:0/1350947048 conn(0x7f87580782e0 msgr2=0x7f87580feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 shutdown_connections 2024-07-24T21:50:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:07.733+0000 7f875ea7f640 1 -- 172.21.15.60:0/1350947048 wait complete. 2024-07-24T21:50:07.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:07 smithi060 ceph-mon[33000]: pgmap v480: 97 pgs: 97 active+clean; 696 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-07-24T21:50:07.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:07 smithi086 ceph-mon[37128]: pgmap v480: 97 pgs: 97 active+clean; 696 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-07-24T21:50:07.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:07 smithi123 ceph-mon[37143]: pgmap v480: 97 pgs: 97 active+clean; 696 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-07-24T21:50:09.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:08 smithi123 podman[72268]: 2024-07-24 21:50:08.967027125 +0000 UTC m=+8.663169412 container remove 4541a7c5f08f90a8cdb31782b7603314622be24c8f68062c2e3d332f49629acd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-24T21:50:09.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:08 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-07-24T21:50:09.570 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:09 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-07-24T21:50:09.570 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:09 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Consumed 5.302s CPU time. 2024-07-24T21:50:09.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:09 smithi060 ceph-mon[33000]: pgmap v481: 97 pgs: 97 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 278 op/s 2024-07-24T21:50:09.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:09 smithi123 ceph-mon[37143]: pgmap v481: 97 pgs: 97 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 278 op/s 2024-07-24T21:50:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:09 smithi086 ceph-mon[37128]: pgmap v481: 97 pgs: 97 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 278 op/s 2024-07-24T21:50:11.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:50:11.421 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:11.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:11 smithi060 ceph-mon[33000]: pgmap v482: 97 pgs: 97 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 308 op/s 2024-07-24T21:50:11.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:11 smithi123 ceph-mon[37143]: pgmap v482: 97 pgs: 97 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 308 op/s 2024-07-24T21:50:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:11 smithi086 ceph-mon[37128]: pgmap v482: 97 pgs: 97 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 308 op/s 2024-07-24T21:50:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 -- 172.21.15.60:0/1462564117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80101b10 msgr2=0x7f7a8010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 --2- 172.21.15.60:0/1462564117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80101b10 0x7f7a8010e860 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009900 tx=0x7f7a7402a480 comp rx=0 tx=0).stop 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 -- 172.21.15.60:0/1462564117 shutdown_connections 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 --2- 172.21.15.60:0/1462564117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8010eda0 0x7f7a80111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 --2- 172.21.15.60:0/1462564117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80101b10 0x7f7a8010e860 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 --2- 172.21.15.60:0/1462564117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 0x7f7a801015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 -- 172.21.15.60:0/1462564117 >> 172.21.15.60:0/1462564117 conn(0x7f7a800fd0c0 msgr2=0x7f7a800ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 -- 172.21.15.60:0/1462564117 shutdown_connections 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 -- 172.21.15.60:0/1462564117 wait complete. 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.242+0000 7f7a85933640 1 Processor -- start 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 -- start start 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 0x7f7a8019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a80101b10 0x7f7a8019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80114970 con 0x7f7a8010eda0 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a801147f0 con 0x7f7a801011f0 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a85933640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80114af0 con 0x7f7a80101b10 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60574/0 (socket says 172.21.15.60:60574) 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7e7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a80101b10 0x7f7a8019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 -- 172.21.15.60:0/217054660 learned_addr learned my addr 172.21.15.60:0/217054660 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7effd640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 0x7f7a8019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 -- 172.21.15.60:0/217054660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a80101b10 msgr2=0x7f7a8019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a80101b10 0x7f7a8019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 -- 172.21.15.60:0/217054660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 msgr2=0x7f7a8019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 0x7f7a8019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.243+0000 7f7a7f7fe640 1 -- 172.21.15.60:0/217054660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a801a0ed0 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a7f7fe640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800eb10 tx=0x7f7a7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004b20 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a801a11c0 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78004cc0 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78010670 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.244+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a801a7880 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.245+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a801015d0 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.246+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7a78005020 con 0x7f7a8010eda0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.247+0000 7f7a84931640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 0x7f7a680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.247+0000 7f7a7effd640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 0x7f7a680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.247+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7a780e45d0 con 0x7f7a8010eda0 2024-07-24T21:50:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.248+0000 7f7a7effd640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 0x7f7a680b6200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000bbe0 tx=0x7f7a7000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.251+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7a780ab8e0 con 0x7f7a8010eda0 2024-07-24T21:50:13.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.509+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a801a7b80 con 0x7f7a8010eda0 2024-07-24T21:50:13.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.510+0000 7f7a84931640 1 -- 172.21.15.60:0/217054660 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f7a780ab1e0 con 0x7f7a8010eda0 2024-07-24T21:50:13.511 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:50:13.512 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"btime":"2024-07-24T21:48:44:369573+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24556,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:48:44.225462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/3784109187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3784109187},{"type":"v1","addr":"172.21.15.123:6835","nonce":3784109187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:50:13.512 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-07-24T21:50:13.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 msgr2=0x7f7a680b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:13.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 0x7f7a680b6200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000bbe0 tx=0x7f7a7000b390 comp rx=0 tx=0).stop 2024-07-24T21:50:13.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 msgr2=0x7f7a801a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800eb10 tx=0x7f7a7800c2d0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 shutdown_connections 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7a680b3d40 0x7f7a680b6200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a8010eda0 0x7f7a801a0700 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a80101b10 0x7f7a8019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 --2- 172.21.15.60:0/217054660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a801011f0 0x7f7a8019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 >> 172.21.15.60:0/217054660 conn(0x7f7a800fd0c0 msgr2=0x7f7a800ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 shutdown_connections 2024-07-24T21:50:13.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:13.512+0000 7f7a85933640 1 -- 172.21.15.60:0/217054660 wait complete. 2024-07-24T21:50:13.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:13 smithi060 ceph-mon[33000]: pgmap v483: 97 pgs: 97 active+clean; 690 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-07-24T21:50:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:13 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/217054660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:13 smithi123 ceph-mon[37143]: pgmap v483: 97 pgs: 97 active+clean; 690 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-07-24T21:50:13.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:13 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/217054660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:13 smithi086 ceph-mon[37128]: pgmap v483: 97 pgs: 97 active+clean; 690 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-07-24T21:50:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:13 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/217054660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:14.126 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 15b6c0a6 --release --await' 2024-07-24T21:50:14.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 15b6c0a6 --release --await 2024-07-24T21:50:14.708 INFO:tasks.workunit.client.0.smithi060.stdout: 174 253125 0 170108 0 60491 0 2024-07-24T21:50:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:15.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:15 smithi060 ceph-mon[33000]: pgmap v484: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-07-24T21:50:15.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:15 smithi123 ceph-mon[37143]: pgmap v484: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-07-24T21:50:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:15 smithi086 ceph-mon[37128]: pgmap v484: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 288 op/s 2024-07-24T21:50:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.781+0000 7f1176a95640 1 -- 172.21.15.60:0/275531493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103f10 msgr2=0x7f117010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.781+0000 7f1176a95640 1 --2- 172.21.15.60:0/275531493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103f10 0x7f117010c4f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1158009900 tx=0x7f115802a410 comp rx=0 tx=0).stop 2024-07-24T21:50:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 -- 172.21.15.60:0/275531493 shutdown_connections 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 --2- 172.21.15.60:0/275531493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f117010cac0 0x7f117010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 --2- 172.21.15.60:0/275531493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103f10 0x7f117010c4f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 --2- 172.21.15.60:0/275531493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1170103510 0x7f11701038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 -- 172.21.15.60:0/275531493 >> 172.21.15.60:0/275531493 conn(0x7f11700fd0c0 msgr2=0x7f11700ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 -- 172.21.15.60:0/275531493 shutdown_connections 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 -- 172.21.15.60:0/275531493 wait complete. 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 Processor -- start 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.782+0000 7f1176a95640 1 -- start start 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1170103f10 0x7f11701018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117010cac0 0x7f1170101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112780 con 0x7f1170103510 2024-07-24T21:50:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112600 con 0x7f117010cac0 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1176a95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112900 con 0x7f1170103f10 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43266/0 (socket says 172.21.15.60:43266) 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 -- 172.21.15.60:0/20426039 learned_addr learned my addr 172.21.15.60:0/20426039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f116ffff640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1170103f10 0x7f11701018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1175032640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117010cac0 0x7f1170101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 -- 172.21.15.60:0/20426039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1170103f10 msgr2=0x7f11701018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1170103f10 0x7f11701018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 -- 172.21.15.60:0/20426039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117010cac0 msgr2=0x7f1170101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117010cac0 0x7f1170101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 -- 172.21.15.60:0/20426039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a7050 con 0x7f1170103510 2024-07-24T21:50:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f1174831640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f116000eab0 tx=0x7f116000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.783+0000 7f116dffb640 1 -- 172.21.15.60:0/20426039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116000ce40 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.784+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a7340 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.784+0000 7f116dffb640 1 -- 172.21.15.60:0/20426039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160011920 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.784+0000 7f116dffb640 1 -- 172.21.15.60:0/20426039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160010a30 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.784+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a78a0 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.786+0000 7f116dffb640 1 -- 172.21.15.60:0/20426039 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1160010bd0 con 0x7f1170103510 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 Processor -- start 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 -- start start 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec048e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 0x5620ec0576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5620ebfbfdd0 con 0x5620ec049350 2024-07-24T21:50:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620ebfaf2d0 con 0x5620ec048a30 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1178e46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5620ebfb3310 con 0x5620ec0552e0 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f116dffb640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 0x7f115c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1174831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1174831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43272/0 (socket says 172.21.15.60:43272) 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1174831640 1 -- 172.21.15.60:0/2925686371 learned_addr learned my addr 172.21.15.60:0/2925686371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f116ffff640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 0x5620ec0576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.787+0000 7f1175032640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec048e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f116dffb640 1 -- 172.21.15.60:0/20426039 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f11600e4670 con 0x7f1170103510 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 -- 172.21.15.60:0/2925686371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 msgr2=0x5620ec0576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 0x5620ec0576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 -- 172.21.15.60:0/2925686371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 msgr2=0x5620ec048e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec048e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1175032640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 0x7f115c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 -- 172.21.15.60:0/2925686371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ebd0fb00 con 0x5620ec049350 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f116ffff640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 0x5620ec0576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1174831640 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f11600e1d30 tx=0x7f11600104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9cd8a55e371428 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1166ffd640 1 -- 172.21.15.60:0/2925686371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600ab980 con 0x5620ec049350 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.788+0000 7f1166ffd640 1 -- 172.21.15.60:0/2925686371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11600ab280 con 0x5620ec049350 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1166ffd640 1 -- 172.21.15.60:0/2925686371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600b3ea0 con 0x5620ec049350 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 -- 172.21.15.60:0/2925686371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 msgr2=0x5620ec054da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f11600e1d30 tx=0x7f11600104d0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1175032640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 0x7f115c0b6200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1168005fd0 tx=0x7f1168005ce0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 -- 172.21.15.60:0/2925686371 shutdown_connections 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec0552e0 0x5620ec0576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec049350 0x5620ec054da0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 --2- 172.21.15.60:0/2925686371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec048e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 -- 172.21.15.60:0/2925686371 >> 172.21.15.60:0/2925686371 conn(0x5620ec046860 msgr2=0x5620ec047a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 -- 172.21.15.60:0/2925686371 shutdown_connections 2024-07-24T21:50:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.789+0000 7f1178e46740 1 -- 172.21.15.60:0/2925686371 wait complete. 2024-07-24T21:50:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 -- start start 2024-07-24T21:50:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 Processor -- start 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec052280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5620ebe12bb0 con 0x5620ec0552e0 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620ebf51900 con 0x5620ec048a30 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5620ec057f50 con 0x5620ec049350 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1178e46740 10 client.0 _ll_register_callbacks cb 0x5620ebf5bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1175032640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1175032640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43282/0 (socket says 172.21.15.60:43282) 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.790+0000 7f1175032640 1 -- 172.21.15.60:0/907522317 learned_addr learned my addr 172.21.15.60:0/907522317 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1174831640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f116ffff640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 -- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 msgr2=0x5620ec0527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 -- 172.21.15.60:0/907522317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 msgr2=0x5620ec052280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec052280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 -- 172.21.15.60:0/907522317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ebd86960 con 0x5620ec0552e0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1174831640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec052280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1175032640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1168010ee0 tx=0x7f11680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d4c36b8f8a029 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f114ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168013e60 con 0x5620ec0552e0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f116801e610 con 0x5620ec0552e0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11600ab280 con 0x5620ec0552e0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f116801e810 con 0x5620ec0552e0 2024-07-24T21:50:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1178e46740 10 client.16419 fetch_fsmap learned FSMap version 29 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.791+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620ec063b70 con 0x5620ec0552e0 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11600ab280 con 0x5620ec0552e0 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f116801a070 con 0x5620ec0552e0 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 10 client.16419 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 10 client.16419 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 20 client.16419 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec065020 0x5620ec067460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 4 client.16419 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620ebfc1d20 con 0x5620ec065020 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.792+0000 7f1174831640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec065020 0x5620ec067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f1174831640 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec065020 0x5620ec067460 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb47dc0288a965 server_cookie=464dd0ada448695d in_seq=0 out_seq=0 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f114ffff640 10 client.16419 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f114ffff640 1 -- 172.21.15.60:0/907522317 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620ebfc1d20 con 0x5620ec065020 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f114ffff640 10 client.16419 handle_command_reply: tid=0 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f1178e46740 1 client.16419 shutdown 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 msgr2=0x5620ec052d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.793+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1168010ee0 tx=0x7f11680046c0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 shutdown_connections 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec065020 0x5620ec067460 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec0552e0 0x5620ec052d00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f114ffff640 0 client.16419 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 --2- 172.21.15.60:0/907522317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec048a30 0x5620ec052280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 >> 172.21.15.60:0/907522317 conn(0x5620ec046860 msgr2=0x5620ec04c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 shutdown_connections 2024-07-24T21:50:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 1 -- 172.21.15.60:0/907522317 wait complete. 2024-07-24T21:50:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.794+0000 7f1178e46740 20 client.16419 trim_cache size 0 max 16384 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 Processor -- start 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 -- start start 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec04cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b2c90 con 0x5620ec048a30 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b0500 con 0x5620ec046860 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f1178e46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ae350 con 0x5620ec049350 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.812+0000 7f116ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43288/0 (socket says 172.21.15.60:43288) 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 -- 172.21.15.60:0/838181172 learned_addr learned my addr 172.21.15.60:0/838181172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f1174831640 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec04cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f1175032640 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 -- 172.21.15.60:0/838181172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 msgr2=0x5620ec06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 -- 172.21.15.60:0/838181172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 msgr2=0x5620ec04cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec04cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 -- 172.21.15.60:0/838181172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ec053120 con 0x5620ec048a30 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f116ffff640 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1158009230 tx=0x7f1158004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79994a5c975f1c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f11667fc640 1 -- 172.21.15.60:0/838181172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158035040 con 0x5620ec048a30 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f11667fc640 1 -- 172.21.15.60:0/838181172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1158004770 con 0x5620ec048a30 2024-07-24T21:50:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.813+0000 7f11667fc640 1 -- 172.21.15.60:0/838181172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158003900 con 0x5620ec048a30 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 -- 172.21.15.60:0/838181172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 msgr2=0x5620ec06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1158009230 tx=0x7f1158004440 comp rx=0 tx=0).stop 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 -- 172.21.15.60:0/838181172 shutdown_connections 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec06cd30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 --2- 172.21.15.60:0/838181172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec04cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 -- 172.21.15.60:0/838181172 >> 172.21.15.60:0/838181172 conn(0x5620ec0544e0 msgr2=0x5620ec0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 -- 172.21.15.60:0/838181172 shutdown_connections 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.814+0000 7f1178e46740 1 -- 172.21.15.60:0/838181172 wait complete. 2024-07-24T21:50:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 -- start start 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 Processor -- start 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec059a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ae9c0 con 0x5620ec048a30 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11701afe30 con 0x5620ec046860 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11701af090 con 0x5620ec049350 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1178e46740 10 client.0 _ll_register_callbacks cb 0x7f1168010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f116ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f116ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43300/0 (socket says 172.21.15.60:43300) 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f116ffff640 1 -- 172.21.15.60:0/4280447366 learned_addr learned my addr 172.21.15.60:0/4280447366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.815+0000 7f1174831640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec059a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f1175032640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 -- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 msgr2=0x5620ec0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 -- 172.21.15.60:0/4280447366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 msgr2=0x5620ec059a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec059a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 -- 172.21.15.60:0/4280447366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ec0c9480 con 0x5620ec048a30 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f1174831640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec059a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f116ffff640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f11580092a0 tx=0x7f1158004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2421be9db6793ae server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f114ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158035030 con 0x5620ec048a30 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1158005260 con 0x5620ec048a30 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1158004770 con 0x5620ec048a30 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1158005460 con 0x5620ec048a30 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f1178e46740 10 client.16425 fetch_fsmap learned FSMap version 29 2024-07-24T21:50:16.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.816+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620ec0d0870 con 0x5620ec048a30 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1158004770 con 0x5620ec048a30 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11580315a0 con 0x5620ec048a30 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 10 client.16425 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 20 client.16425 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec0552e0 0x5620ec0d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 4 client.16425 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "15b6c0a6", "release": true, "await": true}] 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "15b6c0a6", "release": true, "await": true}) -- 0x5620ebfbd7c0 con 0x5620ec0552e0 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.817+0000 7f1174831640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec0552e0 0x5620ec0d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.818+0000 7f1174831640 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec0552e0 0x5620ec0d2f30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1ad7ca28321e30 server_cookie=12bf8d76f0c31e48 in_seq=0 out_seq=0 2024-07-24T21:50:16.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:16.818+0000 7f114ffff640 10 client.16425 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.280+0000 7f114ffff640 1 -- 172.21.15.60:0/4280447366 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5620ebfbd7c0 con 0x5620ec0552e0 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.280+0000 7f114ffff640 10 client.16425 handle_command_reply: tid=0 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.280+0000 7f1178e46740 1 client.16425 shutdown 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 msgr2=0x5620ec05a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f11580092a0 tx=0x7f1158004b30 comp rx=0 tx=0).stop 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 shutdown_connections 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5620ec0552e0 0x5620ec0d2f30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5620ec049350 0x5620ec0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620ec048a30 0x5620ec05a000 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 --2- 172.21.15.60:0/4280447366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620ec046860 0x5620ec059a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f114ffff640 0 client.16425 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 >> 172.21.15.60:0/4280447366 conn(0x5620ec0544e0 msgr2=0x5620ec050b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 shutdown_connections 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 1 -- 172.21.15.60:0/4280447366 wait complete. 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.281+0000 7f1178e46740 20 client.16425 trim_cache size 0 max 16384 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 29, 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 12, 2024-07-24T21:50:17.284 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "15b6c0a6": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 12, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:17.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:17.286 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 msgr2=0x7f115c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 0x7f115c0b6200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1168005fd0 tx=0x7f1168005ce0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 msgr2=0x7f1170101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f116000eab0 tx=0x7f116000ef70 comp rx=0 tx=0).stop 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 shutdown_connections 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f115c0b3d40 0x7f115c0b6200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117010cac0 0x7f1170101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1170103f10 0x7f11701018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 --2- 172.21.15.60:0/20426039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1170103510 0x7f1170101390 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 >> 172.21.15.60:0/20426039 conn(0x7f11700fd0c0 msgr2=0x7f11700fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 shutdown_connections 2024-07-24T21:50:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:17.282+0000 7f1176a95640 1 -- 172.21.15.60:0/20426039 wait complete. 2024-07-24T21:50:17.858 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:17 smithi060 ceph-mon[33000]: pgmap v485: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-07-24T21:50:17.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:17 smithi123 ceph-mon[37143]: pgmap v485: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-07-24T21:50:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:17 smithi086 ceph-mon[37128]: pgmap v485: 97 pgs: 97 active+clean; 691 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-07-24T21:50:18.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:50:18.309 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '15b6c0a6', total seconds elapsed: 37.2 2024-07-24T21:50:18.309 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-24T21:50:18.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:50:18.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:18 smithi060 ceph-mon[33000]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-07-24T21:50:18.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:18 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:50:18.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:18 smithi123 ceph-mon[37143]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-07-24T21:50:18.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:18 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:50:19.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:18 smithi086 ceph-mon[37128]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-07-24T21:50:19.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:18 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:50:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:19.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:19 smithi123 ceph-mon[37143]: pgmap v486: 97 pgs: 97 active+clean; 693 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-07-24T21:50:19.746 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:19 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-07-24T21:50:19.746 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:19 smithi123 systemd[1]: Stopped Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:50:19.746 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:19 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Consumed 5.302s CPU time. 2024-07-24T21:50:19.746 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:19 smithi123 systemd[1]: Starting Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:50:19.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:19 smithi060 ceph-mon[33000]: pgmap v486: 97 pgs: 97 active+clean; 693 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-07-24T21:50:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:19 smithi086 ceph-mon[37128]: pgmap v486: 97 pgs: 97 active+clean; 693 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-07-24T21:50:20.242 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:19 smithi123 podman[72369]: 2024-07-24 21:50:19.647802833 +0000 UTC m=+0.024936542 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:50:20.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:20 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:50:20.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:20 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:50:21.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:20 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:50:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.096+0000 7f0b47203640 1 -- 172.21.15.60:0/3249191174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 msgr2=0x7f0b4011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.096+0000 7f0b47203640 1 --2- 172.21.15.60:0/3249191174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b4011de80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009ca0 tx=0x7f0b3802a8e0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 -- 172.21.15.60:0/3249191174 shutdown_connections 2024-07-24T21:50:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 --2- 172.21.15.60:0/3249191174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b4011de80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 --2- 172.21.15.60:0/3249191174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 --2- 172.21.15.60:0/3249191174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b40110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 -- 172.21.15.60:0/3249191174 >> 172.21.15.60:0/3249191174 conn(0x7f0b4006d5a0 msgr2=0x7f0b4006d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 -- 172.21.15.60:0/3249191174 shutdown_connections 2024-07-24T21:50:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.107+0000 7f0b47203640 1 -- 172.21.15.60:0/3249191174 wait complete. 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.108+0000 7f0b47203640 1 Processor -- start 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 -- start start 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b4011aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40121350 con 0x7f0b40109b40 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401211d0 con 0x7f0b4011ba90 2024-07-24T21:50:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b47203640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401214d0 con 0x7f0b40109f20 2024-07-24T21:50:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b457a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b44f9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b4011aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b457a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54208/0 (socket says 172.21.15.60:54208) 2024-07-24T21:50:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37252/0 (socket says 172.21.15.60:37252) 2024-07-24T21:50:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.109+0000 7f0b457a0640 1 -- 172.21.15.60:0/76475664 learned_addr learned my addr 172.21.15.60:0/76475664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b457a0640 1 -- 172.21.15.60:0/76475664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 msgr2=0x7f0b4011b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b457a0640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b457a0640 1 -- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 msgr2=0x7f0b4011aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b457a0640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b4011aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b457a0640 1 -- 172.21.15.60:0/76475664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b40114b40 con 0x7f0b4011ba90 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.110+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b4011aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.111+0000 7f0b457a0640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003ba0 tx=0x7f0b38003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.113+0000 7f0b3dffb640 1 -- 172.21.15.60:0/76475664 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38004040 con 0x7f0b4011ba90 2024-07-24T21:50:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.113+0000 7f0b3dffb640 1 -- 172.21.15.60:0/76475664 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b38004690 con 0x7f0b4011ba90 2024-07-24T21:50:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.113+0000 7f0b3dffb640 1 -- 172.21.15.60:0/76475664 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38030dd0 con 0x7f0b4011ba90 2024-07-24T21:50:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.113+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b40114dd0 con 0x7f0b4011ba90 2024-07-24T21:50:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.114+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b40119d80 con 0x7f0b4011ba90 2024-07-24T21:50:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.122+0000 7f0b4a037740 1 Processor -- start 2024-07-24T21:50:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.122+0000 7f0b4a037740 1 -- start start 2024-07-24T21:50:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.124+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.125+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.125+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43318/0 (socket says 172.21.15.60:43318) 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.125+0000 7f0b3ffff640 1 -- 172.21.15.60:0/2541954936 learned_addr learned my addr 172.21.15.60:0/2541954936 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.125+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2edabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2edae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed15280 con 0x55ed2ed9faa0 2024-07-24T21:50:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2edabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed04780 con 0x55ed2eda03c0 2024-07-24T21:50:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed087c0 con 0x55ed2edac350 2024-07-24T21:50:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b44f9f640 1 -- 172.21.15.60:0/2541954936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 msgr2=0x55ed2edae740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:50:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2edae740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.126+0000 7f0b44f9f640 1 -- 172.21.15.60:0/2541954936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 msgr2=0x55ed2ed9fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.127+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.127+0000 7f0b44f9f640 1 -- 172.21.15.60:0/2541954936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2ea68790 con 0x55ed2eda03c0 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.127+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.127+0000 7f0b3dffb640 1 -- 172.21.15.60:0/76475664 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0b380041e0 con 0x7f0b4011ba90 2024-07-24T21:50:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.127+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2edabe10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000afc0 tx=0x7f0b3000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdba5f79d70ce4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.129+0000 7f0b3dffb640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 0x7f0b280b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.129+0000 7f0b2effd640 1 -- 172.21.15.60:0/2541954936 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30005220 con 0x55ed2eda03c0 2024-07-24T21:50:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.129+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 0x7f0b280b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.130+0000 7f0b2effd640 1 -- 172.21.15.60:0/2541954936 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b300047a0 con 0x55ed2eda03c0 2024-07-24T21:50:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.131+0000 7f0b3dffb640 1 -- 172.21.15.60:0/76475664 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0b381051e0 con 0x7f0b4011ba90 2024-07-24T21:50:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.131+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 msgr2=0x55ed2edabe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.131+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 0x7f0b280b6410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b34006ca0 tx=0x7f0b34009060 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.131+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2edabe10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000afc0 tx=0x7f0b3000f850 comp rx=0 tx=0).stop 2024-07-24T21:50:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 shutdown_connections 2024-07-24T21:50:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2edae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2edabe10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 --2- 172.21.15.60:0/2541954936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2ed9fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 >> 172.21.15.60:0/2541954936 conn(0x55ed2ed9d8d0 msgr2=0x55ed2ed9ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 shutdown_connections 2024-07-24T21:50:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 -- 172.21.15.60:0/2541954936 wait complete. 2024-07-24T21:50:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.137+0000 7f0b4a037740 1 -- start start 2024-07-24T21:50:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.138+0000 7f0b4a037740 1 Processor -- start 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2eda9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ec3eab0 con 0x55ed2ed9faa0 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ec8c9e0 con 0x55ed2eda03c0 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed2edaf0d0 con 0x55ed2edac350 2024-07-24T21:50:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.140+0000 7f0b4a037740 10 client.0 _ll_register_callbacks cb 0x55ed2ea14a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54224/0 (socket says 172.21.15.60:54224) 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b3ffff640 1 -- 172.21.15.60:0/760791618 learned_addr learned my addr 172.21.15.60:0/760791618 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2eda9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 -- 172.21.15.60:0/760791618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 msgr2=0x55ed2eda8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 -- 172.21.15.60:0/760791618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 msgr2=0x55ed2edab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.141+0000 7f0b457a0640 1 -- 172.21.15.60:0/760791618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2eae1620 con 0x55ed2edac350 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.142+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.142+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.142+0000 7f0b457a0640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2eda9380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b381029e0 tx=0x7f0b38030810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f3f2b9a3bad654 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.155+0000 7f0b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:50:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.155+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38004840 con 0x55ed2edac350 2024-07-24T21:50:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.155+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed2ecb3ce0 con 0x55ed2edac350 2024-07-24T21:50:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.155+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0b380ce070 con 0x55ed2edac350 2024-07-24T21:50:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.161+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b380cb9b0 con 0x55ed2edac350 2024-07-24T21:50:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.162+0000 7f0b4a037740 10 client.25927 fetch_fsmap learned FSMap version 30 2024-07-24T21:50:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.162+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed2edba780 con 0x55ed2edac350 2024-07-24T21:50:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.166+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ed2ecb3ce0 con 0x55ed2edac350 2024-07-24T21:50:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.166+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 7757+0+0 (secure 0 0 0) 0x7f0b380d9080 con 0x55ed2edac350 2024-07-24T21:50:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.166+0000 7f0b4a037740 10 client.25927 fetch_fsmap finished waiting for FSMap version 30 2024-07-24T21:50:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.166+0000 7f0b4a037740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.167+0000 7f0b4a037740 20 client.25927 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.167+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2edbbea0 0x55ed2edbe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.167+0000 7f0b4a037740 4 client.25927 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.167+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed2ed171d0 con 0x55ed2edbbea0 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.167+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2edbbea0 0x55ed2edbe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.168+0000 7f0b23fff640 10 client.25927 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.168+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2edbbea0 0x55ed2edbe320 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703d6d8460ce312d server_cookie=1602459206422a6e in_seq=0 out_seq=0 2024-07-24T21:50:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b23fff640 1 -- 172.21.15.60:0/760791618 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed2ed171d0 con 0x55ed2edbbea0 2024-07-24T21:50:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b23fff640 10 client.25927 handle_command_reply: tid=0 2024-07-24T21:50:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 client.25927 shutdown 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 msgr2=0x55ed2eda9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2eda9380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b381029e0 tx=0x7f0b38030810 comp rx=0 tx=0).stop 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 shutdown_connections 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2edbbea0 0x55ed2edbe320 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2edac350 0x55ed2eda9380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2eda03c0 0x55ed2eda8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 --2- 172.21.15.60:0/760791618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.170+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 >> 172.21.15.60:0/760791618 conn(0x55ed2ed9d8d0 msgr2=0x55ed2eda3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.171+0000 7f0b23fff640 0 client.25927 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.171+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 shutdown_connections 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.171+0000 7f0b4a037740 1 -- 172.21.15.60:0/760791618 wait complete. 2024-07-24T21:50:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.171+0000 7f0b4a037740 20 client.25927 trim_cache size 0 max 16384 2024-07-24T21:50:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.203+0000 7f0b4a037740 1 Processor -- start 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.203+0000 7f0b4a037740 1 -- start start 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 0x55ed2eda8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 0x55ed2edbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c7010 con 0x55ed2eda03c0 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c7740 con 0x55ed2edac350 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b4a037740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c4ff0 con 0x55ed2ed9faa0 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 0x55ed2edbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b44f9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 0x55ed2eda8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54232/0 (socket says 172.21.15.60:54232) 2024-07-24T21:50:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 -- 172.21.15.60:0/3808109608 learned_addr learned my addr 172.21.15.60:0/3808109608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b44f9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 0x55ed2eda8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37272/0 (socket says 172.21.15.60:37272) 2024-07-24T21:50:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 -- 172.21.15.60:0/3808109608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 msgr2=0x55ed2eda8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 0x55ed2eda8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.204+0000 7f0b457a0640 1 -- 172.21.15.60:0/3808109608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 msgr2=0x55ed2edbdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b457a0640 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 0x55ed2edbdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b457a0640 1 -- 172.21.15.60:0/3808109608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2eda01f0 con 0x55ed2edac350 2024-07-24T21:50:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 0x55ed2edbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b457a0640 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cbba0 tx=0x7f0b38102a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54d5b477e15c6ec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b2e7fc640 1 -- 172.21.15.60:0/3808109608 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b380cbc70 con 0x55ed2edac350 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b2e7fc640 1 -- 172.21.15.60:0/3808109608 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b380dc070 con 0x55ed2edac350 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.205+0000 7f0b2e7fc640 1 -- 172.21.15.60:0/3808109608 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b380d0260 con 0x55ed2edac350 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 -- 172.21.15.60:0/3808109608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 msgr2=0x55ed2edc3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cbba0 tx=0x7f0b38102a10 comp rx=0 tx=0).stop 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 -- 172.21.15.60:0/3808109608 shutdown_connections 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edc3bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2eda03c0 0x55ed2edbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 --2- 172.21.15.60:0/3808109608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2ed9faa0 0x55ed2eda8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 -- 172.21.15.60:0/3808109608 >> 172.21.15.60:0/3808109608 conn(0x55ed2ed9d8d0 msgr2=0x55ed2ed9e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 -- 172.21.15.60:0/3808109608 shutdown_connections 2024-07-24T21:50:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.207+0000 7f0b4a037740 1 -- 172.21.15.60:0/3808109608 wait complete. 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.208+0000 7f0b4a037740 1 -- start start 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.208+0000 7f0b4a037740 1 Processor -- start 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.209+0000 7f0b4a037740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.209+0000 7f0b44f9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.209+0000 7f0b44f9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43340/0 (socket says 172.21.15.60:43340) 2024-07-24T21:50:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.210+0000 7f0b44f9f640 1 -- 172.21.15.60:0/514489456 learned_addr learned my addr 172.21.15.60:0/514489456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.210+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2eda03c0 0x55ed2edb43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.210+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.211+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c4920 con 0x55ed2ed9faa0 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.211+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c3510 con 0x55ed2edac350 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.211+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401c3b80 con 0x55ed2eda03c0 2024-07-24T21:50:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.211+0000 7f0b3ffff640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2eda03c0 0x55ed2edb43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.211+0000 7f0b457a0640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edb4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 -- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2eda03c0 msgr2=0x55ed2edb43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2eda03c0 0x55ed2edb43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 -- 172.21.15.60:0/514489456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 msgr2=0x55ed2edae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2edc3a70 con 0x55ed2edac350 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b457a0640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edb4b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cbba0 tx=0x7f0b380e1270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e3449fdfd48b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.212+0000 7f0b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b23fff640 1 -- 172.21.15.60:0/514489456 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b380cbc70 con 0x55ed2edac350 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b23fff640 1 -- 172.21.15.60:0/514489456 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0b380cb100 con 0x55ed2edac350 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 10 client.0 _ll_register_callbacks cb 0x7f0b38005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b380dc070 con 0x55ed2edac350 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b23fff640 1 -- 172.21.15.60:0/514489456 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b38004840 con 0x55ed2edac350 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 10 client.25919 fetch_fsmap learned FSMap version 30 2024-07-24T21:50:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55ed2ee24bb0 con 0x55ed2edac350 2024-07-24T21:50:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b23fff640 1 -- 172.21.15.60:0/514489456 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 30) ==== 7757+0+0 (secure 0 0 0) 0x7f0b380d9040 con 0x55ed2edac350 2024-07-24T21:50:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 10 client.25919 fetch_fsmap finished waiting for FSMap version 30 2024-07-24T21:50:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 20 client.25919 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.213+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2ee24fc0 0x55ed2ee27480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.214+0000 7f0b4a037740 4 client.25919 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.214+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed2ed12c70 con 0x55ed2ee24fc0 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.214+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2ee24fc0 0x55ed2ee27480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.214+0000 7f0b23fff640 10 client.25919 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:21.214+0000 7f0b44f9f640 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2ee24fc0 0x55ed2ee27480 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9268472dae92ba2 server_cookie=6b0630511a852070 in_seq=0 out_seq=0 2024-07-24T21:50:21.492 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:21 smithi123 podman[72369]: 2024-07-24 21:50:21.018624924 +0000 UTC m=+1.395758625 container create c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T21:50:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:21 smithi123 ceph-mon[37143]: pgmap v487: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-07-24T21:50:22.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:21 smithi060 ceph-mon[33000]: pgmap v487: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-07-24T21:50:22.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:21 smithi086 ceph-mon[37128]: pgmap v487: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-07-24T21:50:22.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:22 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:50:22.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:22 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:50:22.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:22 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 -- 172.21.15.60:0/872125139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204100f90 msgr2=0x7f3204101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 --2- 172.21.15.60:0/872125139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204100f90 0x7f3204101370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009900 tx=0x7f31f402a410 comp rx=0 tx=0).stop 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 -- 172.21.15.60:0/872125139 shutdown_connections 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 --2- 172.21.15.60:0/872125139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3204107120 0x7f320410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 --2- 172.21.15.60:0/872125139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32041067a0 0x7f3204106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 --2- 172.21.15.60:0/872125139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204100f90 0x7f3204101370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 -- 172.21.15.60:0/872125139 >> 172.21.15.60:0/872125139 conn(0x7f32040fc3f0 msgr2=0x7f32040fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 -- 172.21.15.60:0/872125139 shutdown_connections 2024-07-24T21:50:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.891+0000 7f320a7bd640 1 -- 172.21.15.60:0/872125139 wait complete. 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 Processor -- start 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 -- start start 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 0x7f32041991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 0x7f320419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3204110580 con 0x7f3204100f90 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3204110400 con 0x7f32041067a0 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f320a7bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3204110700 con 0x7f3204107120 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f3203fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f3203fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43436/0 (socket says 172.21.15.60:43436) 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f3203fff640 1 -- 172.21.15.60:0/2442274098 learned_addr learned my addr 172.21.15.60:0/2442274098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f31fbfff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 0x7f32041991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.892+0000 7f3208d5a640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 0x7f320419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 -- 172.21.15.60:0/2442274098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 msgr2=0x7f320419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 0x7f320419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 -- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 msgr2=0x7f3204198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 -- 172.21.15.60:0/2442274098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419ed10 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f3208d5a640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 0x7f320419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f3203fff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f31fbfff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 0x7f32041991a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00ea90 tx=0x7f31ec00ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec01b070 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419f000 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ec00cc80 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00b5a0 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.893+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3204196930 con 0x7f32041067a0 2024-07-24T21:50:22.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.894+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204102040 con 0x7f32041067a0 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.896+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f31ec01e070 con 0x7f32041067a0 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.897+0000 7f3201ffb640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 0x7f31f00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.897+0000 7f3203fff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 0x7f31f00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.897+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f31ec0e3fe0 con 0x7f32041067a0 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.897+0000 7f3203fff640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 0x7f31f00b6200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f31f4031440 tx=0x7f31f40058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:22.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.900+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f31ec0aaca0 con 0x7f32041067a0 2024-07-24T21:50:22.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:22 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:50:22.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:22 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:50:22.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:22 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:50:23.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:22 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:50:23.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:22 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:50:23.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:22 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:50:23.052 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:50:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:50:23.052 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 -- 172.21.15.60:0/672748951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18103f80 msgr2=0x7fbf1810c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 --2- 172.21.15.60:0/672748951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18103f80 0x7fbf1810c470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009860 tx=0x7fbf0802a340 comp rx=0 tx=0).stop 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 -- 172.21.15.60:0/672748951 shutdown_connections 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 --2- 172.21.15.60:0/672748951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf1810cac0 0x7fbf1810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 --2- 172.21.15.60:0/672748951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18103f80 0x7fbf1810c470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 --2- 172.21.15.60:0/672748951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103660 0x7fbf18103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 -- 172.21.15.60:0/672748951 >> 172.21.15.60:0/672748951 conn(0x7fbf180fd0a0 msgr2=0x7fbf180ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 -- 172.21.15.60:0/672748951 shutdown_connections 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.784+0000 7fbf207dd640 1 -- 172.21.15.60:0/672748951 wait complete. 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 Processor -- start 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 -- start start 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf18103660 0x7fbf18198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 0x7fbf1819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18112720 con 0x7fbf18103f80 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181125a0 con 0x7fbf1810cac0 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf207dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181128a0 con 0x7fbf18103660 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43372/0 (socket says 172.21.15.60:43372) 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf1e579640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf18103660 0x7fbf18198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf1dd78640 1 -- 172.21.15.60:0/784937441 learned_addr learned my addr 172.21.15.60:0/784937441 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.786+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 0x7fbf1819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/784937441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf18103660 msgr2=0x7fbf18198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf18103660 0x7fbf18198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 msgr2=0x7fbf181991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/784937441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1819ed20 con 0x7fbf1810cac0 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.787+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 0x7fbf1819e550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009fd0 tx=0x7fbf1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.788+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/784937441 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000ce20 con 0x7fbf1810cac0 2024-07-24T21:50:23.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.788+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/784937441 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf10005020 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.788+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/784937441 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10010690 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.788+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1819f010 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.788+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf181013c0 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.791+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/784937441 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fbf100054d0 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.792+0000 7fbf0f7fe640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 0x7fbf040b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.792+0000 7fbf1e579640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 0x7fbf040b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.792+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/784937441 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbf100e4680 con 0x7fbf1810cac0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.793+0000 7fbf22b8e740 1 Processor -- start 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.793+0000 7fbf22b8e740 1 -- start start 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.793+0000 7fbf1e579640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 0x7fbf040b6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009530 tx=0x7fbf14009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 0x558495c5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf1e579640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 0x558495c5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf1e579640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 0x558495c5fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43384/0 (socket says 172.21.15.60:43384) 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54266/0 (socket says 172.21.15.60:54266) 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf1e579640 1 -- 172.21.15.60:0/1694554949 learned_addr learned my addr 172.21.15.60:0/1694554949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558495bd6990 con 0x558495c5f6b0 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558495bc5e90 con 0x558495c5ffd0 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.794+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558495bc9ed0 con 0x558495c6bf60 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 -- 172.21.15.60:0/1694554949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 msgr2=0x558495c6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 -- 172.21.15.60:0/1694554949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 msgr2=0x558495c5fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 0x558495c5fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 -- 172.21.15.60:0/1694554949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558495b4f300 con 0x558495c5ffd0 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf0802a930 tx=0x7fbf08005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3eb8a802a32d70 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf0cff9640 1 -- 172.21.15.60:0/1694554949 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08041020 con 0x558495c5ffd0 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf0cff9640 1 -- 172.21.15.60:0/1694554949 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf08030dd0 con 0x558495c5ffd0 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.795+0000 7fbf0cff9640 1 -- 172.21.15.60:0/1694554949 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08037040 con 0x558495c5ffd0 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 msgr2=0x558495c6ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf0802a930 tx=0x7fbf08005dc0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 shutdown_connections 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c6ba20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1694554949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5f6b0 0x558495c5fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 >> 172.21.15.60:0/1694554949 conn(0x558495c5d4e0 msgr2=0x558495c5e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 shutdown_connections 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.797+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1694554949 wait complete. 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 -- start start 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 Processor -- start 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c6b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558495a29700 con 0x558495c5ffd0 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558495b685d0 con 0x558495c5f6b0 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558495c6eb80 con 0x558495c6bf60 2024-07-24T21:50:23.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.798+0000 7fbf22b8e740 10 client.0 _ll_register_callbacks cb 0x558495b88940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37302/0 (socket says 172.21.15.60:37302) 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/4042108122 learned_addr learned my addr 172.21.15.60:0/4042108122 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/4042108122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 msgr2=0x558495c6b9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c6b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/4042108122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 msgr2=0x558495c68e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c68e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/4042108122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558495b5b660 con 0x558495c6bf60 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c760 tx=0x7fbf100ab9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caa3191e0916fda server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100b42e0 con 0x558495c6bf60 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf08030dd0 con 0x558495c6bf60 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.799+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbf100b4060 con 0x558495c6bf60 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf100cc360 con 0x558495c6bf60 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 10 client.25945 fetch_fsmap learned FSMap version 30 2024-07-24T21:50:23.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558495926650 con 0x558495c6bf60 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf08030dd0 con 0x558495c6bf60 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 7757+0+0 (secure 0 0 0) 0x7fbf100a6cc0 con 0x558495c6bf60 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 10 client.25945 fetch_fsmap finished waiting for FSMap version 30 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 10 client.25945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 20 client.25945 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.800+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495c7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.801+0000 7fbf22b8e740 4 client.25945 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.801+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558495bd88e0 con 0x558495c7ba00 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.801+0000 7fbf1e579640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495c7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.801+0000 7fbf1e579640 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495c7de50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c07ef47a9be67d0 server_cookie=5809bf4b6fb64f3a in_seq=0 out_seq=0 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.801+0000 7fbf01ffb640 10 client.25945 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.803+0000 7fbf01ffb640 1 -- 172.21.15.60:0/4042108122 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558495bd88e0 con 0x558495c7ba00 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.803+0000 7fbf01ffb640 10 client.25945 handle_command_reply: tid=0 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.803+0000 7fbf22b8e740 1 client.25945 shutdown 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 msgr2=0x558495c695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c760 tx=0x7fbf100ab9a0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 shutdown_connections 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495c7de50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c695f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf01ffb640 0 client.25945 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:23.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/4042108122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c6b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 >> 172.21.15.60:0/4042108122 conn(0x558495c5d4e0 msgr2=0x558495c63080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 shutdown_connections 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 1 -- 172.21.15.60:0/4042108122 wait complete. 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.804+0000 7fbf22b8e740 20 client.25945 trim_cache size 0 max 16384 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 Processor -- start 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 -- start start 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 0x558495c63800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181b1dc0 con 0x558495c6bf60 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181b24f0 con 0x558495c5ffd0 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf22b8e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181afda0 con 0x558495c5f6b0 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1ed7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1ed7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43402/0 (socket says 172.21.15.60:43402) 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54282/0 (socket says 172.21.15.60:54282) 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/1152964306 learned_addr learned my addr 172.21.15.60:0/1152964306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.837+0000 7fbf1e579640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 0x558495c63800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/1152964306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 msgr2=0x558495c63800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 0x558495c63800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/1152964306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 msgr2=0x558495c83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 -- 172.21.15.60:0/1152964306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558495c5fe00 con 0x558495c6bf60 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f80 tx=0x7fbf10005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce78dde9888e13c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf03fff640 1 -- 172.21.15.60:0/1152964306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100cc500 con 0x558495c6bf60 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf1e579640 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 0x558495c63800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf03fff640 1 -- 172.21.15.60:0/1152964306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf100c5070 con 0x558495c6bf60 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.838+0000 7fbf03fff640 1 -- 172.21.15.60:0/1152964306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100e40b0 con 0x558495c6bf60 2024-07-24T21:50:23.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1152964306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 msgr2=0x558495c83c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f80 tx=0x7fbf10005870 comp rx=0 tx=0).stop 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1152964306 shutdown_connections 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c6bf60 0x558495c83c60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5ffd0 0x558495c83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/1152964306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c5f6b0 0x558495c63800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1152964306 >> 172.21.15.60:0/1152964306 conn(0x558495c5d4e0 msgr2=0x558495c6ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1152964306 shutdown_connections 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- 172.21.15.60:0/1152964306 wait complete. 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 -- start start 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.840+0000 7fbf22b8e740 1 Processor -- start 2024-07-24T21:50:23.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54296/0 (socket says 172.21.15.60:54296) 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1dd78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43416/0 (socket says 172.21.15.60:43416) 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 -- 172.21.15.60:0/341229924 learned_addr learned my addr 172.21.15.60:0/341229924 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181af6d0 con 0x558495c5ffd0 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181ae2c0 con 0x558495c5f6b0 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181ae930 con 0x558495c6bf60 2024-07-24T21:50:23.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1ed7a640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 10 client.0 _ll_register_callbacks cb 0x7fbf100aa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 -- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 msgr2=0x558495c6f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 -- 172.21.15.60:0/341229924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 msgr2=0x558495c746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1e579640 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558495c70070 con 0x558495c5f6b0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf100c5070 con 0x558495c5f6b0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.841+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf1e579640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005fd0 tx=0x7fbf14013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78714c9acaf53998 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf01ffb640 10 client.25934 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:23.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf01ffb640 1 -- 172.21.15.60:0/341229924 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1402d020 con 0x558495c5f6b0 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf01ffb640 1 -- 172.21.15.60:0/341229924 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbf1401b6c0 con 0x558495c5f6b0 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf01ffb640 1 -- 172.21.15.60:0/341229924 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf1401b8e0 con 0x558495c5f6b0 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf22b8e740 10 client.25934 fetch_fsmap learned FSMap version 30 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.842+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558495ce63a0 con 0x558495c5f6b0 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf01ffb640 1 -- 172.21.15.60:0/341229924 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 30) ==== 7757+0+0 (secure 0 0 0) 0x7fbf1403a020 con 0x558495c5f6b0 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 10 client.25934 fetch_fsmap finished waiting for FSMap version 30 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 20 client.25934 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495ce8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:23.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 4 client.25934 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558495bd4380 con 0x558495c7ba00 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495ce8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf1dd78640 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495ce8ad0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf45f1a714e3624f server_cookie=bfb75ef8409d6f2d in_seq=0 out_seq=0 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.843+0000 7fbf01ffb640 10 client.25934 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf01ffb640 1 -- 172.21.15.60:0/341229924 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558495bd4380 con 0x558495c7ba00 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf01ffb640 10 client.25934 handle_command_reply: tid=0 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 client.25934 shutdown 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 msgr2=0x558495c74170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005fd0 tx=0x7fbf14013040 comp rx=0 tx=0).stop 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 shutdown_connections 2024-07-24T21:50:23.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558495c7ba00 0x558495ce8ad0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558495c6bf60 0x558495c6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558495c5ffd0 0x558495c746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf01ffb640 0 client.25934 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 --2- 172.21.15.60:0/341229924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558495c5f6b0 0x558495c74170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 >> 172.21.15.60:0/341229924 conn(0x558495c5d4e0 msgr2=0x558495c5bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 shutdown_connections 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 1 -- 172.21.15.60:0/341229924 wait complete. 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.846+0000 7fbf22b8e740 20 client.25934 trim_cache size 0 max 16384 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 msgr2=0x7fbf040b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 0x7fbf040b6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009530 tx=0x7fbf14009260 comp rx=0 tx=0).stop 2024-07-24T21:50:23.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 msgr2=0x7fbf1819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 0x7fbf1819e550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009fd0 tx=0x7fbf1000ef10 comp rx=0 tx=0).stop 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 shutdown_connections 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbf040b3dc0 0x7fbf040b6280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf1810cac0 0x7fbf1819e550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf18103f80 0x7fbf181991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 --2- 172.21.15.60:0/784937441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf18103660 0x7fbf18198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 >> 172.21.15.60:0/784937441 conn(0x7fbf180fd0a0 msgr2=0x7fbf180fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 shutdown_connections 2024-07-24T21:50:23.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:22.847+0000 7fbf207dd640 1 -- 172.21.15.60:0/784937441 wait complete. 2024-07-24T21:50:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.384+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3204101370 con 0x7f32041067a0 2024-07-24T21:50:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.385+0000 7f3201ffb640 1 -- 172.21.15.60:0/2442274098 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11090 (secure 0 0 0) 0x7f31ec0af1f0 con 0x7f32041067a0 2024-07-24T21:50:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:50:23.388 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"btime":"2024-07-24T21:50:17:948743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:50:17.948711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24556":{"gid":24556,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:50:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-07-24T21:50:23.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.388+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 msgr2=0x7f31f00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.388+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 0x7f31f00b6200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f31f4031440 tx=0x7f31f40058d0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 msgr2=0x7f32041991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 0x7f32041991a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00ea90 tx=0x7f31ec00ef50 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 shutdown_connections 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f31f00b3d40 0x7f31f00b6200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204107120 0x7f320419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32041067a0 0x7f32041991a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 --2- 172.21.15.60:0/2442274098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3204100f90 0x7f3204198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 >> 172.21.15.60:0/2442274098 conn(0x7f32040fc3f0 msgr2=0x7f32040fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 shutdown_connections 2024-07-24T21:50:23.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:23.389+0000 7f320a7bd640 1 -- 172.21.15.60:0/2442274098 wait complete. 2024-07-24T21:50:23.492 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:23 smithi123 podman[72369]: 2024-07-24 21:50:23.239260091 +0000 UTC m=+3.616393788 container init c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, io.buildah.version=1.36.0) 2024-07-24T21:50:23.493 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:23 smithi123 podman[72369]: 2024-07-24 21:50:23.243055635 +0000 UTC m=+3.620189334 container start c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=main-86385b1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-24T21:50:23.493 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:23 smithi123 ceph-mds[72523]: -- 172.21.15.123:0/2576755330 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ed296b860 con 0x555ed2948800 2024-07-24T21:50:23.493 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: starting mds.f at 2024-07-24T21:50:23.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:23 smithi060 ceph-mon[33000]: pgmap v488: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 767 KiB/s wr, 207 op/s 2024-07-24T21:50:23.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:23 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2442274098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:23.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:50:23.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:50:23.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:23 smithi123 ceph-mon[37143]: pgmap v488: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 767 KiB/s wr, 207 op/s 2024-07-24T21:50:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:23 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2442274098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:23 smithi086 ceph-mon[37128]: pgmap v488: 97 pgs: 97 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 767 KiB/s wr, 207 op/s 2024-07-24T21:50:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:23 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2442274098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:24.492 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:24 smithi123 bash[72369]: c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 2024-07-24T21:50:24.493 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:50:24 smithi123 systemd[1]: Started Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:50:24.708 INFO:tasks.workunit.client.0.smithi060.stdout: 190 1324004 1041 892522 1013 327565 752 2024-07-24T21:50:24.735 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-24T21:50:24.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-24T21:50:24.796 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:24.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:24 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6834/3836191659,v1:172.21.15.123:6835/3836191659] up:boot 2024-07-24T21:50:24.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:24 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:50:24.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:24 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:50:24.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:24 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6834/3836191659,v1:172.21.15.123:6835/3836191659] up:boot 2024-07-24T21:50:24.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:24 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:50:24.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:24 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:50:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:24 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6834/3836191659,v1:172.21.15.123:6835/3836191659] up:boot 2024-07-24T21:50:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:24 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:50:25.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:24 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:50:25.644 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:25.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:25 smithi060 ceph-mon[33000]: pgmap v489: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-07-24T21:50:25.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:25 smithi123 ceph-mon[37143]: pgmap v489: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-07-24T21:50:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:25 smithi086 ceph-mon[37128]: pgmap v489: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/3288666495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34101650 msgr2=0x7f8f34101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/3288666495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34101650 0x7f8f34101a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009860 tx=0x7f8f3002a340 comp rx=0 tx=0).stop 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/3288666495 shutdown_connections 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/3288666495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f3410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/3288666495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34107060 0x7f8f341074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/3288666495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34101650 0x7f8f34101a30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/3288666495 >> 172.21.15.60:0/3288666495 conn(0x7f8f340fd0c0 msgr2=0x7f8f340ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/3288666495 shutdown_connections 2024-07-24T21:50:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.398+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/3288666495 wait complete. 2024-07-24T21:50:27.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 Processor -- start 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 -- start start 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34107060 0x7f8f3419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f341a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341115f0 con 0x7f8f341079e0 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34111470 con 0x7f8f34107060 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3b5c3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34111770 con 0x7f8f34101650 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3935f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f38b5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34107060 0x7f8f3419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3935f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59448/0 (socket says 172.21.15.60:59448) 2024-07-24T21:50:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f3935f640 1 -- 172.21.15.60:0/686341916 learned_addr learned my addr 172.21.15.60:0/686341916 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.399+0000 7f8f39b60640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f341a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 -- 172.21.15.60:0/686341916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34107060 msgr2=0x7f8f3419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34107060 0x7f8f3419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 -- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 msgr2=0x7f8f341a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f341a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 -- 172.21.15.60:0/686341916 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341a0f10 con 0x7f8f34101650 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f39b60640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f341a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3935f640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f300303f0 tx=0x7f8f30032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30040020 con 0x7f8f34101650 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f30030dd0 con 0x7f8f34101650 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f341a11a0 con 0x7f8f34101650 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a78c0 con 0x7f8f34101650 2024-07-24T21:50:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.400+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30037040 con 0x7f8f34101650 2024-07-24T21:50:27.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.401+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f34101a30 con 0x7f8f34101650 2024-07-24T21:50:27.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.403+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8f30043020 con 0x7f8f34101650 2024-07-24T21:50:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.404+0000 7f8f2a7fc640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 0x7f8f200b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.404+0000 7f8f38b5e640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 0x7f8f200b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.404+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8f301043a0 con 0x7f8f34101650 2024-07-24T21:50:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.405+0000 7f8f38b5e640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 0x7f8f200b6200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f34106da0 tx=0x7f8f24009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.407+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8f3004a020 con 0x7f8f34101650 2024-07-24T21:50:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.681+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f341a7b50 con 0x7f8f34101650 2024-07-24T21:50:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.681+0000 7f8f2a7fc640 1 -- 172.21.15.60:0/686341916 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11919 (secure 0 0 0) 0x7f8f300cb760 con 0x7f8f34101650 2024-07-24T21:50:27.683 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:50:27.683 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":31,"btime":"2024-07-24T21:50:23:599643+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":25963,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3836191659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3836191659},{"type":"v1","addr":"172.21.15.123:6835","nonce":3836191659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:50:17.948711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24556":{"gid":24556,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:50:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 31 2024-07-24T21:50:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 msgr2=0x7f8f200b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 0x7f8f200b6200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8f34106da0 tx=0x7f8f24009260 comp rx=0 tx=0).stop 2024-07-24T21:50:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 msgr2=0x7f8f3419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f300303f0 tx=0x7f8f30032da0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 shutdown_connections 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8f200b3d40 0x7f8f200b6200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f341079e0 0x7f8f341a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f34107060 0x7f8f3419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 --2- 172.21.15.60:0/686341916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f34101650 0x7f8f3419ae60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 >> 172.21.15.60:0/686341916 conn(0x7f8f340fd0c0 msgr2=0x7f8f3406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 shutdown_connections 2024-07-24T21:50:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:27.684+0000 7f8f3b5c3640 1 -- 172.21.15.60:0/686341916 wait complete. 2024-07-24T21:50:27.959 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:27 smithi060 ceph-mon[33000]: pgmap v490: 97 pgs: 97 active+clean; 744 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-07-24T21:50:27.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:27 smithi123 ceph-mon[37143]: pgmap v490: 97 pgs: 97 active+clean; 744 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-07-24T21:50:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:27 smithi086 ceph-mon[37128]: pgmap v490: 97 pgs: 97 active+clean; 744 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 353 op/s 2024-07-24T21:50:28.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:50:28.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:28 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/686341916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:28 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/686341916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:29.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:28 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/686341916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:29.690 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.694+0000 7f05b1a10640 1 -- 172.21.15.60:0/1115856871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 msgr2=0x7f05ac102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.694+0000 7f05b1a10640 1 --2- 172.21.15.60:0/1115856871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac102740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f059c009930 tx=0x7f059c02a420 comp rx=0 tx=0).stop 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 -- 172.21.15.60:0/1115856871 shutdown_connections 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 --2- 172.21.15.60:0/1115856871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 0x7f05ac10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 --2- 172.21.15.60:0/1115856871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 --2- 172.21.15.60:0/1115856871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac102740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 -- 172.21.15.60:0/1115856871 >> 172.21.15.60:0/1115856871 conn(0x7f05ac076890 msgr2=0x7f05ac076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 -- 172.21.15.60:0/1115856871 shutdown_connections 2024-07-24T21:50:29.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.695+0000 7f05b1a10640 1 -- 172.21.15.60:0/1115856871 wait complete. 2024-07-24T21:50:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.696+0000 7f05b1a10640 1 Processor -- start 2024-07-24T21:50:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.696+0000 7f05b1a10640 1 -- start start 2024-07-24T21:50:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 0x7f05ac19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac111640 con 0x7f05ac102360 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1114c0 con 0x7f05ac107150 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05b1a10640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1117c0 con 0x7f05ac107ad0 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52402/0 (socket says 172.21.15.60:52402) 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 -- 172.21.15.60:0/4202673007 learned_addr learned my addr 172.21.15.60:0/4202673007 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36908/0 (socket says 172.21.15.60:36908) 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 0x7f05ac19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 -- 172.21.15.60:0/4202673007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 msgr2=0x7f05ac19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 0x7f05ac19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 -- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 msgr2=0x7f05ac199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aa7fc640 1 -- 172.21.15.60:0/4202673007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac1a4ef0 con 0x7f05ac107150 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.697+0000 7f05aaffd640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e9f0 tx=0x7f05a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05b0a0e640 1 -- 172.21.15.60:0/4202673007 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000cd80 con 0x7f05ac107150 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac1a51e0 con 0x7f05ac107150 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05b0a0e640 1 -- 172.21.15.60:0/4202673007 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a00108e0 con 0x7f05ac107150 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05b0a0e640 1 -- 172.21.15.60:0/4202673007 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0018980 con 0x7f05ac107150 2024-07-24T21:50:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.698+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac197900 con 0x7f05ac107150 2024-07-24T21:50:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.700+0000 7f05b3dc1740 1 Processor -- start 2024-07-24T21:50:29.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:29 smithi060 ceph-mon[33000]: pgmap v491: 97 pgs: 97 active+clean; 777 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 491 op/s 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 -- start start 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf0b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf1fdb0 0x55c61cf221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c61ce81670 con 0x55c61cf0cca0 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c61ce70b70 con 0x55c61cf1fdb0 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c61ce74bb0 con 0x55c61cf0b770 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf0b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59458/0 (socket says 172.21.15.60:59458) 2024-07-24T21:50:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.701+0000 7f05aaffd640 1 -- 172.21.15.60:0/1609459091 learned_addr learned my addr 172.21.15.60:0/1609459091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:29.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 -- 172.21.15.60:0/1609459091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 msgr2=0x55c61cf0bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 -- 172.21.15.60:0/1609459091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf1fdb0 msgr2=0x55c61cf221a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf1fdb0 0x55c61cf221a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 -- 172.21.15.60:0/1609459091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61ce08d30 con 0x55c61cf0cca0 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf0b230 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005fd0 tx=0x7f05a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabb66787f9ea8c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f05b0a0e640 1 -- 172.21.15.60:0/4202673007 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f05a0010430 con 0x7f05ac107150 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f0591ffb640 1 -- 172.21.15.60:0/1609459091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4004d20 con 0x55c61cf0cca0 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f0591ffb640 1 -- 172.21.15.60:0/1609459091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a4004030 con 0x55c61cf0cca0 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.702+0000 7f0591ffb640 1 -- 172.21.15.60:0/1609459091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4005260 con 0x55c61cf0cca0 2024-07-24T21:50:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b0a0e640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 0x7f05940b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05aaffd640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 0x7f05940b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1609459091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 msgr2=0x55c61cf0b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf0b230 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005fd0 tx=0x7f05a4011810 comp rx=0 tx=0).stop 2024-07-24T21:50:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b0a0e640 1 -- 172.21.15.60:0/4202673007 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f05a00e42a0 con 0x7f05ac107150 2024-07-24T21:50:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1609459091 shutdown_connections 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf1fdb0 0x55c61cf221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf0b770 0x55c61cf0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1609459091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf0b230 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.703+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1609459091 >> 172.21.15.60:0/1609459091 conn(0x55c61ced5720 msgr2=0x55c61cf09060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.704+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1609459091 shutdown_connections 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.704+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1609459091 wait complete. 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.704+0000 7f05b3dc1740 1 -- start start 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.704+0000 7f05b3dc1740 1 Processor -- start 2024-07-24T21:50:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.704+0000 7f05aaffd640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 0x7f05940b6280 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f059c000c00 tx=0x7f059c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf1d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c61ccd47a0 con 0x55c61cf0cca0 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c61ce12ff0 con 0x55c61cf0b770 2024-07-24T21:50:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aa7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf1d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52408/0 (socket says 172.21.15.60:52408) 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c61cf22c40 con 0x55c61cf1fdb0 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 -- 172.21.15.60:0/2377702128 learned_addr learned my addr 172.21.15.60:0/2377702128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05b3dc1740 10 client.0 _ll_register_callbacks cb 0x55c61cb7e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 -- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 msgr2=0x55c61cf1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 -- 172.21.15.60:0/2377702128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 msgr2=0x55c61cf1d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf1d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aaffd640 1 -- 172.21.15.60:0/2377702128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61ce170a0 con 0x55c61cf0b770 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.705+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf1d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f05aaffd640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f059c009230 tx=0x7f059c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d46c816cf05a40 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f058affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c035040 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f059c0037f0 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a4004030 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f059c004f10 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f05b3dc1740 10 client.25961 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c61cbe0b90 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.706+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a4004030 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f059c044030 con 0x55c61cf0b770 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 10 client.25961 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 10 client.25961 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 20 client.25961 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf2fef0 0x55c61cf322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 4 client.25961 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61ce835c0 con 0x55c61cf2fef0 2024-07-24T21:50:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf2fef0 0x55c61cf322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.707+0000 7f058affd640 10 client.25961 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.708+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf2fef0 0x55c61cf322e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5280e3bdcf6297 server_cookie=5d9d4c510d872aaf in_seq=0 out_seq=0 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.709+0000 7f058affd640 1 -- 172.21.15.60:0/2377702128 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c61ce835c0 con 0x55c61cf2fef0 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.709+0000 7f058affd640 10 client.25961 handle_command_reply: tid=0 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.709+0000 7f05b3dc1740 1 client.25961 shutdown 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 msgr2=0x55c61cf1cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f059c009230 tx=0x7f059c009260 comp rx=0 tx=0).stop 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 shutdown_connections 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf2fef0 0x55c61cf322e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0cca0 0x55c61cf1d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/2377702128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0b770 0x55c61cf1cce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 >> 172.21.15.60:0/2377702128 conn(0x55c61ced5720 msgr2=0x55c61cf0dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f058affd640 0 client.25961 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 shutdown_connections 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 1 -- 172.21.15.60:0/2377702128 wait complete. 2024-07-24T21:50:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.710+0000 7f05b3dc1740 20 client.25961 trim_cache size 0 max 16384 2024-07-24T21:50:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 Processor -- start 2024-07-24T21:50:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 -- start start 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf21ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1ae8b0 con 0x55c61cf0b770 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1ac120 con 0x55c61cf0cca0 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1a9f70 con 0x55c61cf1fdb0 2024-07-24T21:50:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36932/0 (socket says 172.21.15.60:36932) 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05aaffd640 1 -- 172.21.15.60:0/1568498698 learned_addr learned my addr 172.21.15.60:0/1568498698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.735+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf21ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 -- 172.21.15.60:0/1568498698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 msgr2=0x55c61cf1ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 -- 172.21.15.60:0/1568498698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 msgr2=0x55c61cf21ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf21ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 -- 172.21.15.60:0/1568498698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61cf1b1e0 con 0x55c61cf0b770 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f059c02a460 tx=0x7f059c051680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8406de20ada929dd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05917fa640 1 -- 172.21.15.60:0/1568498698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0314a0 con 0x55c61cf0b770 2024-07-24T21:50:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05917fa640 1 -- 172.21.15.60:0/1568498698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c05d020 con 0x55c61cf0b770 2024-07-24T21:50:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.736+0000 7f05917fa640 1 -- 172.21.15.60:0/1568498698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0035f0 con 0x55c61cf0b770 2024-07-24T21:50:29.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1568498698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 msgr2=0x55c61cf07b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:29.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f059c02a460 tx=0x7f059c051680 comp rx=0 tx=0).stop 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1568498698 shutdown_connections 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf1ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf21ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1568498698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf07b50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1568498698 >> 172.21.15.60:0/1568498698 conn(0x55c61ced5720 msgr2=0x55c61cf1eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1568498698 shutdown_connections 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.737+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1568498698 wait complete. 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.738+0000 7f05b3dc1740 1 -- start start 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.738+0000 7f05b3dc1740 1 Processor -- start 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.738+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.738+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1aa5e0 con 0x55c61cf0b770 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1aba50 con 0x55c61cf0cca0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac1aacb0 con 0x55c61cf1fdb0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 10 client.0 _ll_register_callbacks cb 0x7f059c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36938/0 (socket says 172.21.15.60:36938) 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 -- 172.21.15.60:0/1665465627 learned_addr learned my addr 172.21.15.60:0/1665465627 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05ab7fe640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 -- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 msgr2=0x55c61cf08870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf08870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 -- 172.21.15.60:0/1665465627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 msgr2=0x55c61cf284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 -- 172.21.15.60:0/1665465627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61cf08db0 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05aaffd640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f059c02a940 tx=0x7f059c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4048c6b252b7cdfd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f058affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c066020 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.739+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059c05d020 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f059c031a90 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f059c071050 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 10 client.16494 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c61cf37360 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c05d020 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f059c044070 con 0x55c61cf0b770 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 10 client.16494 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 10 client.16494 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 20 client.16494 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf9bbd0 0x55c61cf9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 4 client.16494 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.740+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c61ce7f060 con 0x55c61cf9bbd0 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.742+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf9bbd0 0x55c61cf9dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.742+0000 7f05aa7fc640 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf9bbd0 0x55c61cf9dfc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6eb05fe8404b611 server_cookie=700a9b50508cd1f1 in_seq=0 out_seq=0 2024-07-24T21:50:29.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:29.742+0000 7f058affd640 10 client.16494 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:29.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:29 smithi123 ceph-mon[37143]: pgmap v491: 97 pgs: 97 active+clean; 777 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 491 op/s 2024-07-24T21:50:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:29 smithi086 ceph-mon[37128]: pgmap v491: 97 pgs: 97 active+clean; 777 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 491 op/s 2024-07-24T21:50:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b23fff640 1 -- 172.21.15.60:0/514489456 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed2ed12c70 con 0x55ed2ee24fc0 2024-07-24T21:50:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b23fff640 10 client.25919 handle_command_reply: tid=0 2024-07-24T21:50:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 client.25919 shutdown 2024-07-24T21:50:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 msgr2=0x55ed2edb4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edb4b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cbba0 tx=0x7f0b380e1270 comp rx=0 tx=0).stop 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 shutdown_connections 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ed2ee24fc0 0x55ed2ee27480 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed2edac350 0x55ed2edb4b40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed2eda03c0 0x55ed2edb43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.068+0000 7f0b4a037740 1 --2- 172.21.15.60:0/514489456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ed2ed9faa0 0x55ed2edae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.069+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 >> 172.21.15.60:0/514489456 conn(0x55ed2ed9d8d0 msgr2=0x55ed2eda3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.069+0000 7f0b23fff640 0 client.25919 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.069+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 shutdown_connections 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.069+0000 7f0b4a037740 1 -- 172.21.15.60:0/514489456 wait complete. 2024-07-24T21:50:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.069+0000 7f0b4a037740 20 client.25919 trim_cache size 0 max 16384 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 msgr2=0x7f0b280b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 0x7f0b280b6410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b34006ca0 tx=0x7f0b34009060 comp rx=0 tx=0).stop 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 msgr2=0x7f0b40114280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003ba0 tx=0x7f0b38003bd0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 shutdown_connections 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0b280b3f50 0x7f0b280b6410 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b4011ba90 0x7f0b40114280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b40109f20 0x7f0b4011b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 --2- 172.21.15.60:0/76475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b40109b40 0x7f0b4011aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 >> 172.21.15.60:0/76475664 conn(0x7f0b4006d5a0 msgr2=0x7f0b400720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 shutdown_connections 2024-07-24T21:50:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:30.070+0000 7f0b47203640 1 -- 172.21.15.60:0/76475664 wait complete. 2024-07-24T21:50:31.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 -- 172.21.15.60:0/556817597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6438103ef0 msgr2=0x7f643810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 --2- 172.21.15.60:0/556817597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6438103ef0 0x7f643810c4d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6428009900 tx=0x7f642802a3f0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 -- 172.21.15.60:0/556817597 shutdown_connections 2024-07-24T21:50:31.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 --2- 172.21.15.60:0/556817597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f643810caa0 0x7f643810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 --2- 172.21.15.60:0/556817597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6438103ef0 0x7f643810c4d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 --2- 172.21.15.60:0/556817597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64381034f0 0x7f64381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 -- 172.21.15.60:0/556817597 >> 172.21.15.60:0/556817597 conn(0x7f64380fd0c0 msgr2=0x7f64380ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 -- 172.21.15.60:0/556817597 shutdown_connections 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.586+0000 7f643ff70640 1 -- 172.21.15.60:0/556817597 wait complete. 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 Processor -- start 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 -- start start 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438103ef0 0x7f64381018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 0x7f6438101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f64381126f0 con 0x7f64381034f0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6438112570 con 0x7f643810caa0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643ff70640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6438112870 con 0x7f6438103ef0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643e50d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 0x7f6438101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36944/0 (socket says 172.21.15.60:36944) 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.587+0000 7f643dd0c640 1 -- 172.21.15.60:0/2687398048 learned_addr learned my addr 172.21.15.60:0/2687398048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643d50b640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438103ef0 0x7f64381018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 -- 172.21.15.60:0/2687398048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438103ef0 msgr2=0x7f64381018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438103ef0 0x7f64381018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 -- 172.21.15.60:0/2687398048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 msgr2=0x7f6438101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 0x7f6438101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 -- 172.21.15.60:0/2687398048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381a7090 con 0x7f64381034f0 2024-07-24T21:50:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643e50d640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 0x7f6438101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643dd0c640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f643400e9f0 tx=0x7f643400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f642effd640 1 -- 172.21.15.60:0/2687398048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643400cd80 con 0x7f64381034f0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64381a7380 con 0x7f64381034f0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f642effd640 1 -- 172.21.15.60:0/2687398048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64340108e0 con 0x7f64381034f0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f642effd640 1 -- 172.21.15.60:0/2687398048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434018980 con 0x7f64381034f0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.588+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381a7890 con 0x7f64381034f0 2024-07-24T21:50:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.591+0000 7f642effd640 1 -- 172.21.15.60:0/2687398048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6434010430 con 0x7f64381034f0 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.592+0000 7f6442321740 1 Processor -- start 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.592+0000 7f642effd640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 0x7f64240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.592+0000 7f643e50d640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 0x7f64240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.592+0000 7f642effd640 1 -- 172.21.15.60:0/2687398048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f64340e43c0 con 0x7f64381034f0 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.592+0000 7f6442321740 1 -- start start 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b686ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b695370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56152b5fd9b0 con 0x56152b6866d0 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56152b5eceb0 con 0x56152b686ff0 2024-07-24T21:50:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f6442321740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56152b5f0ef0 con 0x56152b692f80 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b686ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643e50d640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 0x7f64240b6200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6430009fd0 tx=0x7f6430009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52464/0 (socket says 172.21.15.60:52464) 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 -- 172.21.15.60:0/566293874 learned_addr learned my addr 172.21.15.60:0/566293874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643e50d640 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b695370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 -- 172.21.15.60:0/566293874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 msgr2=0x56152b695370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b695370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 -- 172.21.15.60:0/566293874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 msgr2=0x56152b686ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b686ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 -- 172.21.15.60:0/566293874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152b576320 con 0x56152b686ff0 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643dd0c640 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b686ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.593+0000 7f643d50b640 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6428009900 tx=0x7f6428032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d03dfdffeb3a5f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.594+0000 7f6423fff640 1 -- 172.21.15.60:0/566293874 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642803f070 con 0x56152b686ff0 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.594+0000 7f6423fff640 1 -- 172.21.15.60:0/566293874 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6428032e40 con 0x56152b686ff0 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.594+0000 7f6423fff640 1 -- 172.21.15.60:0/566293874 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428035070 con 0x56152b686ff0 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.594+0000 7f6442321740 1 -- 172.21.15.60:0/566293874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 msgr2=0x56152b692a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.594+0000 7f6442321740 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6428009900 tx=0x7f6428032b90 comp rx=0 tx=0).stop 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 -- 172.21.15.60:0/566293874 shutdown_connections 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b695370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b692a40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 --2- 172.21.15.60:0/566293874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b686ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 -- 172.21.15.60:0/566293874 >> 172.21.15.60:0/566293874 conn(0x56152b684500 msgr2=0x56152b6856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 -- 172.21.15.60:0/566293874 shutdown_connections 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 -- 172.21.15.60:0/566293874 wait complete. 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 -- start start 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.595+0000 7f6442321740 1 Processor -- start 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b686ff0 0x56152b690480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b6909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56152b450700 con 0x56152b686ff0 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56152b58f5f0 con 0x56152b6866d0 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56152b695ba0 con 0x56152b692f80 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f6442321740 10 client.0 _ll_register_callbacks cb 0x56152b5af960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643d50b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b686ff0 0x56152b690480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643e50d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b6909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52470/0 (socket says 172.21.15.60:52470) 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 -- 172.21.15.60:0/1212877998 learned_addr learned my addr 172.21.15.60:0/1212877998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 -- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 msgr2=0x56152b6909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b6909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 -- 172.21.15.60:0/1212877998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b686ff0 msgr2=0x56152b690480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b686ff0 0x56152b690480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.596+0000 7f643dd0c640 1 -- 172.21.15.60:0/1212877998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152b582680 con 0x56152b6866d0 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f643dd0c640 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f64340a7050 tx=0x7f64340aac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8449c2d3fc2e5cc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340b4080 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f64340aa440 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56152b5ff900 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6434018b20 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 10 client.25979 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56152b34d650 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56152b5ff900 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f64340a9ca0 con 0x56152b6866d0 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 10 client.25979 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 10 client.25979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.597+0000 7f6442321740 20 client.25979 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.598+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b6a2a90 0x56152b6a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.598+0000 7f6442321740 4 client.25979 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.598+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56152b5fb3a0 con 0x56152b6a2a90 2024-07-24T21:50:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.598+0000 7f643d50b640 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b6a2a90 0x56152b6a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.599+0000 7f643d50b640 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b6a2a90 0x56152b6a4ec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d590b747c9a23e5e server_cookie=2fb3ec1edbd6d24 in_seq=0 out_seq=0 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.599+0000 7f64217fa640 10 client.25979 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f64217fa640 1 -- 172.21.15.60:0/1212877998 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56152b5fb3a0 con 0x56152b6a2a90 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f64217fa640 10 client.25979 handle_command_reply: tid=0 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 client.25979 shutdown 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 msgr2=0x56152b68ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f64340a7050 tx=0x7f64340aac20 comp rx=0 tx=0).stop 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 shutdown_connections 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b6a2a90 0x56152b6a4ec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b6909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b686ff0 0x56152b690480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 --2- 172.21.15.60:0/1212877998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b68ff40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f64217fa640 0 client.25979 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 >> 172.21.15.60:0/1212877998 conn(0x56152b684500 msgr2=0x56152b68a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 shutdown_connections 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 1 -- 172.21.15.60:0/1212877998 wait complete. 2024-07-24T21:50:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.604+0000 7f6442321740 20 client.25979 trim_cache size 0 max 16384 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 Processor -- start 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 -- start start 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b6aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b682d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b2cb0 con 0x56152b6866d0 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b0520 con 0x56152b686ff0 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f6442321740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64381ae370 con 0x56152b692f80 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.621+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36966/0 (socket says 172.21.15.60:36966) 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 -- 172.21.15.60:0/3616427450 learned_addr learned my addr 172.21.15.60:0/3616427450 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643d50b640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b6aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643e50d640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b682d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 -- 172.21.15.60:0/3616427450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 msgr2=0x56152b682d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b682d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 -- 172.21.15.60:0/3616427450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 msgr2=0x56152b6aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b6aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 -- 172.21.15.60:0/3616427450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152b6a5540 con 0x56152b6866d0 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643d50b640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b6aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f643dd0c640 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f643400c6a0 tx=0x7f64340aaa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99c5455465e52b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f64237fe640 1 -- 172.21.15.60:0/3616427450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340aafd0 con 0x56152b6866d0 2024-07-24T21:50:31.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f64237fe640 1 -- 172.21.15.60:0/3616427450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64340aa440 con 0x56152b6866d0 2024-07-24T21:50:31.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.622+0000 7f64237fe640 1 -- 172.21.15.60:0/3616427450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340a6940 con 0x56152b6866d0 2024-07-24T21:50:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 -- 172.21.15.60:0/3616427450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 msgr2=0x56152b6aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f643400c6a0 tx=0x7f64340aaa50 comp rx=0 tx=0).stop 2024-07-24T21:50:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 -- 172.21.15.60:0/3616427450 shutdown_connections 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b692f80 0x56152b682d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b686ff0 0x56152b6aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 --2- 172.21.15.60:0/3616427450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b6866d0 0x56152b6aa790 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 -- 172.21.15.60:0/3616427450 >> 172.21.15.60:0/3616427450 conn(0x56152b684500 msgr2=0x56152b691d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 -- 172.21.15.60:0/3616427450 shutdown_connections 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.623+0000 7f6442321740 1 -- 172.21.15.60:0/3616427450 wait complete. 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 -- start start 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 Processor -- start 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b69b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b686ff0 0x56152b696750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f64381ae9e0 con 0x56152b692f80 2024-07-24T21:50:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381afe50 con 0x56152b6866d0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64381af0b0 con 0x56152b686ff0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f6442321740 10 client.0 _ll_register_callbacks cb 0x7f64340a6f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f643e50d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f643dd0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b69b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f643e50d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36976/0 (socket says 172.21.15.60:36976) 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f643e50d640 1 -- 172.21.15.60:0/673794342 learned_addr learned my addr 172.21.15.60:0/673794342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.624+0000 7f643d50b640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b686ff0 0x56152b696750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 -- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b686ff0 msgr2=0x56152b696750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b686ff0 0x56152b696750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 -- 172.21.15.60:0/673794342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 msgr2=0x56152b69b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b69b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 -- 172.21.15.60:0/673794342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152b697220 con 0x56152b692f80 2024-07-24T21:50:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643dd0c640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b69b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f643e50d640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f6430014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0dca01de5f30e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f64217fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643002e060 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64340aa440 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f643001f830 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6430012e20 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f6442321740 10 client.16512 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.625+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56152b70d580 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64340aa440 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f643000f040 con 0x56152b692f80 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 10 client.16512 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 10 client.16512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 20 client.16512 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b70dd60 0x56152b710180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 4 client.16512 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:50:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56152b5f8ae0 con 0x56152b70dd60 2024-07-24T21:50:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f643dd0c640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b70dd60 0x56152b710180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f64217fa640 10 client.16512 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:50:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.626+0000 7f643dd0c640 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b70dd60 0x56152b710180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecad0e0e783b4099 server_cookie=741140ba611de4b6 in_seq=0 out_seq=0 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f64217fa640 1 -- 172.21.15.60:0/673794342 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56152b5f8ae0 con 0x56152b70dd60 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f64217fa640 10 client.16512 handle_command_reply: tid=0 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 client.16512 shutdown 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 msgr2=0x56152b696c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f6430014070 comp rx=0 tx=0).stop 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 shutdown_connections 2024-07-24T21:50:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56152b70dd60 0x56152b710180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56152b692f80 0x56152b696c90 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f64217fa640 0 client.16512 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56152b686ff0 0x56152b696750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 --2- 172.21.15.60:0/673794342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56152b6866d0 0x56152b69b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 >> 172.21.15.60:0/673794342 conn(0x56152b684500 msgr2=0x56152b691570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 shutdown_connections 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 1 -- 172.21.15.60:0/673794342 wait complete. 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.677+0000 7f6442321740 20 client.16512 trim_cache size 0 max 16384 2024-07-24T21:50:31.680 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 msgr2=0x7f64240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 0x7f64240b6200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6430009fd0 tx=0x7f6430009600 comp rx=0 tx=0).stop 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 msgr2=0x7f6438101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f643400e9f0 tx=0x7f643400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 shutdown_connections 2024-07-24T21:50:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f64240b3d40 0x7f64240b6200 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810caa0 0x7f6438101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6438103ef0 0x7f64381018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 --2- 172.21.15.60:0/2687398048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64381034f0 0x7f6438101370 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 >> 172.21.15.60:0/2687398048 conn(0x7f64380fd0c0 msgr2=0x7f64380fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 shutdown_connections 2024-07-24T21:50:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:31.678+0000 7f643ff70640 1 -- 172.21.15.60:0/2687398048 wait complete. 2024-07-24T21:50:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:31 smithi123 ceph-mon[37143]: pgmap v492: 97 pgs: 97 active+clean; 797 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.4 MiB/s wr, 547 op/s 2024-07-24T21:50:32.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:31 smithi060 ceph-mon[33000]: pgmap v492: 97 pgs: 97 active+clean; 797 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.4 MiB/s wr, 547 op/s 2024-07-24T21:50:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:31 smithi086 ceph-mon[37128]: pgmap v492: 97 pgs: 97 active+clean; 797 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.4 MiB/s wr, 547 op/s 2024-07-24T21:50:32.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:50:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:33.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:33 smithi060 ceph-mon[33000]: pgmap v493: 97 pgs: 97 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 558 op/s 2024-07-24T21:50:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:33 smithi123 ceph-mon[37143]: pgmap v493: 97 pgs: 97 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 558 op/s 2024-07-24T21:50:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:33 smithi086 ceph-mon[37128]: pgmap v493: 97 pgs: 97 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 558 op/s 2024-07-24T21:50:34.708 INFO:tasks.workunit.client.0.smithi060.stdout: 218 383051 1369 258169 1516 93541 870 2024-07-24T21:50:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/347810416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 msgr2=0x7fb8f8108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/347810416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f8108600 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009900 tx=0x7fb8ec02a410 comp rx=0 tx=0).stop 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/347810416 shutdown_connections 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/347810416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/347810416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f8108600 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/347810416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/347810416 >> 172.21.15.60:0/347810416 conn(0x7fb8f80fe200 msgr2=0x7fb8f8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/347810416 shutdown_connections 2024-07-24T21:50:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.733+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/347810416 wait complete. 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 Processor -- start 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 -- start start 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f8112770 con 0x7fb8f81081c0 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f81125f0 con 0x7fb8f8108b40 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.734+0000 7fb8fdbf5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f81128f0 con 0x7fb8f81027b0 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36994/0 (socket says 172.21.15.60:36994) 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/1479322501 learned_addr learned my addr 172.21.15.60:0/1479322501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/1479322501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 msgr2=0x7fb8f8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/1479322501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 msgr2=0x7fb8f819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/1479322501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f819ed30 con 0x7fb8f81081c0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005940 tx=0x7fb8ec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.735+0000 7fb8f4ff9640 1 -- 172.21.15.60:0/1479322501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec03f030 con 0x7fb8f81081c0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.736+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f819efc0 con 0x7fb8f81081c0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.736+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f8074d70 con 0x7fb8f81081c0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.736+0000 7fb8f4ff9640 1 -- 172.21.15.60:0/1479322501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec0044c0 con 0x7fb8f81081c0 2024-07-24T21:50:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.736+0000 7fb8f4ff9640 1 -- 172.21.15.60:0/1479322501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec049060 con 0x7fb8f81081c0 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 Processor -- start 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f4ff9640 1 -- 172.21.15.60:0/1479322501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fb8ec042030 con 0x7fb8f81081c0 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 -- start start 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae6aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae6d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5588eadd59b0 con 0x5588eae6af80 2024-07-24T21:50:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588eadc4eb0 con 0x5588eae5e6d0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588eadc8ef0 con 0x5588eae5eff0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae6aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52518/0 (socket says 172.21.15.60:52518) 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 -- 172.21.15.60:0/36596078 learned_addr learned my addr 172.21.15.60:0/36596078 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae6d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 -- 172.21.15.60:0/36596078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 msgr2=0x5588eae6aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae6aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 -- 172.21.15.60:0/36596078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 msgr2=0x5588eae6d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae6d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.739+0000 7fb8f7fff640 1 -- 172.21.15.60:0/36596078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ead4e320 con 0x5588eae5e6d0 2024-07-24T21:50:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8f4ff9640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 0x7fb8e00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011810 tx=0x7fb8f0011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc6fab9edd207f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 0x7fb8e00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8f4ff9640 1 -- 172.21.15.60:0/1479322501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb8ec103f70 con 0x7fb8f81081c0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8ddffb640 1 -- 172.21.15.60:0/36596078 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0011310 con 0x5588eae5e6d0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8ddffb640 1 -- 172.21.15.60:0/36596078 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f000f9a0 con 0x5588eae5e6d0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.740+0000 7fb8ddffb640 1 -- 172.21.15.60:0/36596078 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0005650 con 0x5588eae5e6d0 2024-07-24T21:50:34.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 -- 172.21.15.60:0/36596078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 msgr2=0x5588eae5eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011810 tx=0x7fb8f0011cd0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 -- 172.21.15.60:0/36596078 shutdown_connections 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae6d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae6aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/36596078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae5eab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8fffa6740 1 -- 172.21.15.60:0/36596078 >> 172.21.15.60:0/36596078 conn(0x5588eae5c500 msgr2=0x5588eae5d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.741+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 0x7fb8e00b6200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5588eae5d3e0 tx=0x7fb8e8012040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.742+0000 7fb8fffa6740 1 -- 172.21.15.60:0/36596078 shutdown_connections 2024-07-24T21:50:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.742+0000 7fb8fffa6740 1 -- 172.21.15.60:0/36596078 wait complete. 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.742+0000 7fb8fffa6740 1 -- start start 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.742+0000 7fb8fffa6740 1 Processor -- start 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae67e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5588eac28700 con 0x5588eae6af80 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588ead675f0 con 0x5588eae5e6d0 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588eae6dc50 con 0x5588eae5eff0 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8fffa6740 10 client.0 _ll_register_callbacks cb 0x5588ead87960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae6aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52528/0 (socket says 172.21.15.60:52528) 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1615361566 learned_addr learned my addr 172.21.15.60:0/1615361566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae67e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 -- 172.21.15.60:0/1615361566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 msgr2=0x5588eae67e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae67e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 -- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 msgr2=0x5588eae6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 -- 172.21.15.60:0/1615361566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ead5a680 con 0x5588eae6af80 2024-07-24T21:50:34.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae685e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011810 tx=0x7fb8f0011340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48cc4f61980dc9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.743+0000 7fb8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0028020 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb8f0004c40 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f000f9a0 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb8f0033050 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 10 client.16524 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588eab25650 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f000f9a0 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb8f0018070 con 0x5588eae6af80 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 10 client.16524 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:50:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 20 client.16524 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eae7a9d0 0x5588eae7cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.744+0000 7fb8fffa6740 4 client.16524 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.745+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588eadd7900 con 0x5588eae7a9d0 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.745+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eae7a9d0 0x5588eae7cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.745+0000 7fb8d6ffd640 10 client.16524 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:50:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.745+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eae7a9d0 0x5588eae7cdc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96438af27fa2a746 server_cookie=1d68a26c8b5d1a54 in_seq=0 out_seq=0 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.746+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/1615361566 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588eadd7900 con 0x5588eae7a9d0 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.746+0000 7fb8d6ffd640 10 client.16524 handle_command_reply: tid=0 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.746+0000 7fb8fffa6740 1 client.16524 shutdown 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 msgr2=0x5588eae685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae685e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0011810 tx=0x7fb8f0011340 comp rx=0 tx=0).stop 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 shutdown_connections 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eae7a9d0 0x5588eae7cdc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae685e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae67e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8d6ffd640 0 client.16524 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/1615361566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5e6d0 0x5588eae6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 >> 172.21.15.60:0/1615361566 conn(0x5588eae5c500 msgr2=0x5588eae620a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 shutdown_connections 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 1 -- 172.21.15.60:0/1615361566 wait complete. 2024-07-24T21:50:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.747+0000 7fb8fffa6740 20 client.16524 trim_cache size 0 max 16384 2024-07-24T21:50:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.762+0000 7fb8fffa6740 1 Processor -- start 2024-07-24T21:50:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.762+0000 7fb8fffa6740 1 -- start start 2024-07-24T21:50:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5e6d0 0x5588eae82650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4011140 con 0x5588eae6af80 2024-07-24T21:50:34.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400e9b0 con 0x5588eae5eff0 2024-07-24T21:50:34.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400c800 con 0x5588eae5e6d0 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5e6d0 0x5588eae82650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52534/0 (socket says 172.21.15.60:52534) 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37034/0 (socket says 172.21.15.60:37034) 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/584303390 learned_addr learned my addr 172.21.15.60:0/584303390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/584303390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5e6d0 msgr2=0x5588eae82650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5e6d0 0x5588eae82650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/584303390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 msgr2=0x5588eae65060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 -- 172.21.15.60:0/584303390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588eae5d1e0 con 0x5588eae5eff0 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.763+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0c7df0 tx=0x7fb8ec005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2575841676905bba server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8dd7fa640 1 -- 172.21.15.60:0/584303390 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec035070 con 0x5588eae5eff0 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8dd7fa640 1 -- 172.21.15.60:0/584303390 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec0ca690 con 0x5588eae5eff0 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8dd7fa640 1 -- 172.21.15.60:0/584303390 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0cf180 con 0x5588eae5eff0 2024-07-24T21:50:34.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 -- 172.21.15.60:0/584303390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 msgr2=0x5588eae82b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0c7df0 tx=0x7fb8ec005fd0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 -- 172.21.15.60:0/584303390 shutdown_connections 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae6af80 0x5588eae65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae5eff0 0x5588eae82b90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/584303390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5e6d0 0x5588eae82650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.764+0000 7fb8fffa6740 1 -- 172.21.15.60:0/584303390 >> 172.21.15.60:0/584303390 conn(0x5588eae5c500 msgr2=0x5588eae69ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- 172.21.15.60:0/584303390 shutdown_connections 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- 172.21.15.60:0/584303390 wait complete. 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- start start 2024-07-24T21:50:34.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 Processor -- start 2024-07-24T21:50:34.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 0x5588eae659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400ce70 con 0x5588eae5e6d0 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400e2e0 con 0x5588eae6af80 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8fffa6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400d540 con 0x5588eae5eff0 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.765+0000 7fb8f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37042/0 (socket says 172.21.15.60:37042) 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f77fe640 1 -- 172.21.15.60:0/3135653095 learned_addr learned my addr 172.21.15.60:0/3135653095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 0x5588eae659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f6ffd640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8fffa6740 10 client.0 _ll_register_callbacks cb 0x7fb8ec107990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 -- 172.21.15.60:0/3135653095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 msgr2=0x5588eae654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 -- 172.21.15.60:0/3135653095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 msgr2=0x5588eae64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 -- 172.21.15.60:0/3135653095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588eae726b0 con 0x5588eae6af80 2024-07-24T21:50:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8f7fff640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 0x5588eae659e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0010010 tx=0x7fb8f0009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e3333e74e72d57 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0028040 con 0x5588eae6af80 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb8f00113b0 con 0x5588eae6af80 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.766+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ec0ca690 con 0x5588eae6af80 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb8f0024420 con 0x5588eae6af80 2024-07-24T21:50:34.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 10 client.26003 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588eaee59a0 con 0x5588eae6af80 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec0ca690 con 0x5588eae6af80 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb8f0018070 con 0x5588eae6af80 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 10 client.26003 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 10 client.26003 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:50:34.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 20 client.26003 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eaee60f0 0x5588eaee8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 4 client.26003 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.767+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588eadd33a0 con 0x5588eaee60f0 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.768+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eaee60f0 0x5588eaee8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.768+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eaee60f0 0x5588eaee8510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bc5657229fa5fbe server_cookie=3d526df0e816dcdd in_seq=0 out_seq=0 2024-07-24T21:50:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.768+0000 7fb8d6ffd640 10 client.26003 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.769+0000 7fb8d6ffd640 1 -- 172.21.15.60:0/3135653095 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588eadd33a0 con 0x5588eaee60f0 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.769+0000 7fb8d6ffd640 10 client.26003 handle_command_reply: tid=0 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.769+0000 7fb8fffa6740 1 client.26003 shutdown 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 msgr2=0x5588eae659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 0x5588eae659e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0010010 tx=0x7fb8f0009520 comp rx=0 tx=0).stop 2024-07-24T21:50:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 shutdown_connections 2024-07-24T21:50:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5588eaee60f0 0x5588eaee8510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eae6af80 0x5588eae659e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588eae5eff0 0x5588eae654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 --2- 172.21.15.60:0/3135653095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588eae5e6d0 0x5588eae64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 >> 172.21.15.60:0/3135653095 conn(0x5588eae5c500 msgr2=0x5588eae61810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8d6ffd640 0 client.26003 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:50:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 shutdown_connections 2024-07-24T21:50:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 1 -- 172.21.15.60:0/3135653095 wait complete. 2024-07-24T21:50:34.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.770+0000 7fb8fffa6740 20 client.26003 trim_cache size 0 max 16384 2024-07-24T21:50:34.782 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 msgr2=0x7fb8e00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 0x7fb8e00b6200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5588eae5d3e0 tx=0x7fb8e8012040 comp rx=0 tx=0).stop 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 msgr2=0x7fb8f81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005940 tx=0x7fb8ec0040c0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 shutdown_connections 2024-07-24T21:50:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb8e00b3d40 0x7fb8e00b6200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f8108b40 0x7fb8f819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f81081c0 0x7fb8f81991c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 --2- 172.21.15.60:0/1479322501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f81027b0 0x7fb8f8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 >> 172.21.15.60:0/1479322501 conn(0x7fb8f80fe200 msgr2=0x7fb8f81005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 shutdown_connections 2024-07-24T21:50:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:34.771+0000 7fb8fdbf5640 1 -- 172.21.15.60:0/1479322501 wait complete. 2024-07-24T21:50:35.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:50:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:35 smithi123 ceph-mon[37143]: pgmap v494: 97 pgs: 97 active+clean; 795 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.7 MiB/s wr, 536 op/s 2024-07-24T21:50:36.005 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:35 smithi086 ceph-mon[37128]: pgmap v494: 97 pgs: 97 active+clean; 795 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.7 MiB/s wr, 536 op/s 2024-07-24T21:50:36.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:35 smithi060 ceph-mon[33000]: pgmap v494: 97 pgs: 97 active+clean; 795 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.7 MiB/s wr, 536 op/s 2024-07-24T21:50:37.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:37 smithi060 ceph-mon[33000]: pgmap v495: 97 pgs: 97 active+clean; 798 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-07-24T21:50:37.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 -- 172.21.15.60:0/2156921699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c1036a0 msgr2=0x7f0d6c103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/2156921699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c103a80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009900 tx=0x7f0d5c02a410 comp rx=0 tx=0).stop 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 -- 172.21.15.60:0/2156921699 shutdown_connections 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/2156921699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c10cb00 0x7f0d6c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/2156921699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/2156921699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c103a80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 -- 172.21.15.60:0/2156921699 >> 172.21.15.60:0/2156921699 conn(0x7f0d6c0fd100 msgr2=0x7f0d6c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 -- 172.21.15.60:0/2156921699 shutdown_connections 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.875+0000 7f0d70bd0640 1 -- 172.21.15.60:0/2156921699 wait complete. 2024-07-24T21:50:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 Processor -- start 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 -- start start 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c10cb00 0x7f0d6c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c112760 con 0x7f0d6c103fc0 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1125e0 con 0x7f0d6c10cb00 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d70bd0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1128e0 con 0x7f0d6c1036a0 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54326/0 (socket says 172.21.15.60:54326) 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d63fff640 1 -- 172.21.15.60:0/119265911 learned_addr learned my addr 172.21.15.60:0/119265911 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.876+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c10cb00 0x7f0d6c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 -- 172.21.15.60:0/119265911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 msgr2=0x7f0d6c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 -- 172.21.15.60:0/119265911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c10cb00 msgr2=0x7f0d6c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c10cb00 0x7f0d6c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 -- 172.21.15.60:0/119265911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c19ed30 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d63fff640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d740 tx=0x7f0d5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d637fe640 1 -- 172.21.15.60:0/119265911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54017020 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d637fe640 1 -- 172.21.15.60:0/119265911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5400de50 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d637fe640 1 -- 172.21.15.60:0/119265911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400f620 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6c19f020 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.877+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6c1a56e0 con 0x7f0d6c103fc0 2024-07-24T21:50:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d637fe640 1 -- 172.21.15.60:0/119265911 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0d5401a020 con 0x7f0d6c103fc0 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 Processor -- start 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 -- start start 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x557273483ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 0x55727348fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273492370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5572733fa9b0 con 0x557273483ff0 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572733e9eb0 con 0x5572734836d0 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d72f81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572733edef0 con 0x55727348ff80 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 0x55727348fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d6ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x557273483ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 0x55727348fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54336/0 (socket says 172.21.15.60:54336) 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d6a59c640 1 -- 172.21.15.60:0/2567307928 learned_addr learned my addr 172.21.15.60:0/2567307928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.880+0000 7f0d63fff640 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273492370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 -- 172.21.15.60:0/2567307928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 msgr2=0x557273492370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d637fe640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 0x7f0d580b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273492370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 -- 172.21.15.60:0/2567307928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 msgr2=0x55727348fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 0x55727348fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 -- 172.21.15.60:0/2567307928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557273373320 con 0x5572734836d0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d637fe640 1 -- 172.21.15.60:0/119265911 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0d540e3e30 con 0x7f0d6c103fc0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 0x7f0d580b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x557273483ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009f00 tx=0x7f0d64011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebee4cef2456835 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d60ff9640 1 -- 172.21.15.60:0/2567307928 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004250 con 0x5572734836d0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d60ff9640 1 -- 172.21.15.60:0/2567307928 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6401be00 con 0x5572734836d0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.881+0000 7f0d60ff9640 1 -- 172.21.15.60:0/2567307928 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64019a80 con 0x5572734836d0 2024-07-24T21:50:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- 172.21.15.60:0/2567307928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 msgr2=0x557273483ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x557273483ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009f00 tx=0x7f0d64011810 comp rx=0 tx=0).stop 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 0x7f0d580b6200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d64020000 tx=0x7f0d64021000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- 172.21.15.60:0/2567307928 shutdown_connections 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273492370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557273483ff0 0x55727348fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 --2- 172.21.15.60:0/2567307928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x557273483ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- 172.21.15.60:0/2567307928 >> 172.21.15.60:0/2567307928 conn(0x557273481500 msgr2=0x5572734826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- 172.21.15.60:0/2567307928 shutdown_connections 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- 172.21.15.60:0/2567307928 wait complete. 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.882+0000 7f0d72f81740 1 -- start start 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 Processor -- start 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557273483ff0 0x55727348d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55727348ff80 0x55727348d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55727324d700 con 0x55727348ff80 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55727338c5f0 con 0x5572734836d0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557273492c00 con 0x557273483ff0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d72f81740 10 client.0 _ll_register_callbacks cb 0x5572733ac960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53560/0 (socket says 172.21.15.60:53560) 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 -- 172.21.15.60:0/3910709272 learned_addr learned my addr 172.21.15.60:0/3910709272 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55727348ff80 0x55727348d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d63fff640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557273483ff0 0x55727348d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 -- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557273483ff0 msgr2=0x55727348d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557273483ff0 0x55727348d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 -- 172.21.15.60:0/3910709272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55727348ff80 msgr2=0x55727348d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55727348ff80 0x55727348d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.883+0000 7f0d6a59c640 1 -- 172.21.15.60:0/3910709272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727337f680 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c033000 tx=0x7f0d5c0317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee3f686350e9551 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c035040 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6401be00 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0d5c004240 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0d5c031910 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d72f81740 10 client.26015 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55727314a650 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6401be00 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.884+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0d5c004b80 con 0x5572734836d0 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 10 client.26015 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 20 client.26015 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727349faa0 0x5572734a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 4 client.26015 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572733fc900 con 0x55727349faa0 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d63fff640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727349faa0 0x5572734a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d63fff640 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727349faa0 0x5572734a1eb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f001c16a46ebec server_cookie=4d45c8630bb7eb7a in_seq=0 out_seq=0 2024-07-24T21:50:37.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.885+0000 7f0d51ffb640 10 client.26015 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d51ffb640 1 -- 172.21.15.60:0/3910709272 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572733fc900 con 0x55727349faa0 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d51ffb640 10 client.26015 handle_command_reply: tid=0 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 client.26015 shutdown 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 msgr2=0x55727348cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c033000 tx=0x7f0d5c0317a0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 shutdown_connections 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727349faa0 0x5572734a1eb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55727348ff80 0x55727348d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557273483ff0 0x55727348d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d51ffb640 0 client.26015 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 --2- 172.21.15.60:0/3910709272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572734836d0 0x55727348cea0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 >> 172.21.15.60:0/3910709272 conn(0x557273481500 msgr2=0x5572734870a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 shutdown_connections 2024-07-24T21:50:37.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 1 -- 172.21.15.60:0/3910709272 wait complete. 2024-07-24T21:50:37.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.886+0000 7f0d72f81740 20 client.26015 trim_cache size 0 max 16384 2024-07-24T21:50:37.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 Processor -- start 2024-07-24T21:50:37.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 -- start start 2024-07-24T21:50:37.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x55727347fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1b0a90 con 0x5572734836d0 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1ae300 con 0x557273483ff0 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d72f81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1ac150 con 0x55727348ff80 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53566/0 (socket says 172.21.15.60:53566) 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d6a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54368/0 (socket says 172.21.15.60:54368) 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.900+0000 7f0d63fff640 1 -- 172.21.15.60:0/4087643992 learned_addr learned my addr 172.21.15.60:0/4087643992 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x55727347fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 -- 172.21.15.60:0/4087643992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 msgr2=0x55727347fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x55727347fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 -- 172.21.15.60:0/4087643992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 msgr2=0x5572734a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 -- 172.21.15.60:0/4087643992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557273483e20 con 0x557273483ff0 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d63fff640 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d540aa740 tx=0x7f0d540e6340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07355c0863c13ec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d53fff640 1 -- 172.21.15.60:0/4087643992 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540b3430 con 0x557273483ff0 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d53fff640 1 -- 172.21.15.60:0/4087643992 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d540aa550 con 0x557273483ff0 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.901+0000 7f0d53fff640 1 -- 172.21.15.60:0/4087643992 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54009c30 con 0x557273483ff0 2024-07-24T21:50:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- 172.21.15.60:0/4087643992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 msgr2=0x5572734a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d540aa740 tx=0x7f0d540e6340 comp rx=0 tx=0).stop 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- 172.21.15.60:0/4087643992 shutdown_connections 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x55727347fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x5572734a7d00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 --2- 172.21.15.60:0/4087643992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x5572734a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- 172.21.15.60:0/4087643992 >> 172.21.15.60:0/4087643992 conn(0x557273481500 msgr2=0x55727348ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- 172.21.15.60:0/4087643992 shutdown_connections 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- 172.21.15.60:0/4087643992 wait complete. 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.902+0000 7f0d72f81740 1 -- start start 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 Processor -- start 2024-07-24T21:50:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x557273493cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273494750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1ac7c0 con 0x5572734836d0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1adc30 con 0x557273483ff0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1ace90 con 0x55727348ff80 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d72f81740 10 client.0 _ll_register_callbacks cb 0x7f0d5c005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53576/0 (socket says 172.21.15.60:53576) 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d63fff640 1 -- 172.21.15.60:0/557641362 learned_addr learned my addr 172.21.15.60:0/557641362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x557273493cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.903+0000 7f0d6ad9d640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273494750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 -- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 msgr2=0x557273494750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273494750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 -- 172.21.15.60:0/557641362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 msgr2=0x557273493cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x557273493cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 -- 172.21.15.60:0/557641362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572735097c0 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x557273493cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d63fff640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d54010c20 tx=0x7f0d54010c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706fb9d4ac653080 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540aaa40 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0d540e1800 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d540aa550 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0d540ab140 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d72f81740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.904+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55727350aac0 con 0x557273483ff0 2024-07-24T21:50:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d540af040 con 0x557273483ff0 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0d540a64d0 con 0x557273483ff0 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 20 client.26021 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727350b160 0x55727350d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 4 client.26021 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0d540aa550 con 0x55727350b160 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727350b160 0x55727350d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d6a59c640 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727350b160 0x55727350d580 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eaa75f977a72481 server_cookie=fd60f1fed9df46c0 in_seq=0 out_seq=0 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d51ffb640 10 client.26021 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d51ffb640 1 -- 172.21.15.60:0/557641362 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0d540aa550 con 0x55727350b160 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d51ffb640 10 client.26021 handle_command_reply: tid=0 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.905+0000 7f0d72f81740 1 client.26021 shutdown 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 msgr2=0x557273494210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d54010c20 tx=0x7f0d54010c70 comp rx=0 tx=0).stop 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 shutdown_connections 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55727350b160 0x55727350d580 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55727348ff80 0x557273494750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557273483ff0 0x557273494210 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 --2- 172.21.15.60:0/557641362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572734836d0 0x557273493cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d51ffb640 0 client.26021 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 >> 172.21.15.60:0/557641362 conn(0x557273481500 msgr2=0x557273486810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 shutdown_connections 2024-07-24T21:50:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 1 -- 172.21.15.60:0/557641362 wait complete. 2024-07-24T21:50:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d72f81740 20 client.26021 trim_cache size 0 max 16384 2024-07-24T21:50:37.918 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:50:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 msgr2=0x7f0d580b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 0x7f0d580b6200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d64020000 tx=0x7f0d64021000 comp rx=0 tx=0).stop 2024-07-24T21:50:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 msgr2=0x7f0d6c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d740 tx=0x7f0d5400dc00 comp rx=0 tx=0).stop 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 shutdown_connections 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0d580b3d40 0x7f0d580b6200 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.906+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d6c10cb00 0x7f0d6c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.907+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d6c103fc0 0x7f0d6c1991c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.907+0000 7f0d70bd0640 1 --2- 172.21.15.60:0/119265911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d6c1036a0 0x7f0d6c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:37.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.907+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 >> 172.21.15.60:0/119265911 conn(0x7f0d6c0fd100 msgr2=0x7f0d6c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:37.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.907+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 shutdown_connections 2024-07-24T21:50:37.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:37.907+0000 7f0d70bd0640 1 -- 172.21.15.60:0/119265911 wait complete. 2024-07-24T21:50:37.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:37 smithi123 ceph-mon[37143]: pgmap v495: 97 pgs: 97 active+clean; 798 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-07-24T21:50:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:37 smithi086 ceph-mon[37128]: pgmap v495: 97 pgs: 97 active+clean; 798 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 686 op/s 2024-07-24T21:50:38.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f058affd640 1 -- 172.21.15.60:0/1665465627 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c61ce7f060 con 0x55c61cf9bbd0 2024-07-24T21:50:38.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f058affd640 10 client.16494 handle_command_reply: tid=0 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 client.16494 shutdown 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 msgr2=0x55c61cf27f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f059c02a940 tx=0x7f059c0045e0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 shutdown_connections 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c61cf9bbd0 0x55c61cf9dfc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c61cf1fdb0 0x55c61cf08870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c61cf0cca0 0x55c61cf284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f058affd640 0 client.16494 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 --2- 172.21.15.60:0/1665465627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c61cf0b770 0x55c61cf27f60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 >> 172.21.15.60:0/1665465627 conn(0x55c61ced5720 msgr2=0x55c61cf1cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 shutdown_connections 2024-07-24T21:50:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 1 -- 172.21.15.60:0/1665465627 wait complete. 2024-07-24T21:50:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b3dc1740 20 client.16494 trim_cache size 0 max 16384 2024-07-24T21:50:38.120 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 30, 2024-07-24T21:50:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:50:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 14, 2024-07-24T21:50:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:50:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "62499c77": { 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 14, 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:50:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:50:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:38.124 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 msgr2=0x7f05940b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 0x7f05940b6280 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f059c000c00 tx=0x7f059c005a20 comp rx=0 tx=0).stop 2024-07-24T21:50:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 msgr2=0x7f05ac19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e9f0 tx=0x7f05a000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 shutdown_connections 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05940b3dc0 0x7f05940b6280 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac107ad0 0x7f05ac19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ac107150 0x7f05ac19a050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 --2- 172.21.15.60:0/4202673007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ac102360 0x7f05ac199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 >> 172.21.15.60:0/4202673007 conn(0x7f05ac076890 msgr2=0x7f05ac0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 shutdown_connections 2024-07-24T21:50:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:38.115+0000 7f05b1a10640 1 -- 172.21.15.60:0/4202673007 wait complete. 2024-07-24T21:50:39.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:38 smithi123 ceph-mon[37143]: pgmap v496: 97 pgs: 97 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-07-24T21:50:39.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:38 smithi086 ceph-mon[37128]: pgmap v496: 97 pgs: 97 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-07-24T21:50:39.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:38 smithi060 ceph-mon[33000]: pgmap v496: 97 pgs: 97 active+clean; 804 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-07-24T21:50:39.651 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '62499c77', quiesce took 21.3 seconds. Will release after: 8.7 2024-07-24T21:50:40.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:50:40.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:50:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:41.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:41 smithi060 ceph-mon[33000]: pgmap v497: 97 pgs: 97 active+clean; 813 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 587 op/s 2024-07-24T21:50:41.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:41 smithi123 ceph-mon[37143]: pgmap v497: 97 pgs: 97 active+clean; 813 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 587 op/s 2024-07-24T21:50:41.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:41 smithi086 ceph-mon[37128]: pgmap v497: 97 pgs: 97 active+clean; 813 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 587 op/s 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.241+0000 7f305c97b640 1 -- 172.21.15.60:0/702865894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30581079e0 msgr2=0x7f305810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.241+0000 7f305c97b640 1 --2- 172.21.15.60:0/702865894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30581079e0 0x7f305810e1e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3050009ca0 tx=0x7f305002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 -- 172.21.15.60:0/702865894 shutdown_connections 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 --2- 172.21.15.60:0/702865894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30581079e0 0x7f305810e1e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 --2- 172.21.15.60:0/702865894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058107060 0x7f30581074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 --2- 172.21.15.60:0/702865894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3058101650 0x7f3058101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 -- 172.21.15.60:0/702865894 >> 172.21.15.60:0/702865894 conn(0x7f30580fd0c0 msgr2=0x7f30580ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 -- 172.21.15.60:0/702865894 shutdown_connections 2024-07-24T21:50:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.242+0000 7f305c97b640 1 -- 172.21.15.60:0/702865894 wait complete. 2024-07-24T21:50:42.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 Processor -- start 2024-07-24T21:50:42.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 -- start start 2024-07-24T21:50:42.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 0x7f305819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 0x7f30581a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3058111660 con 0x7f3058107060 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30581114e0 con 0x7f30581079e0 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305c97b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30581117e0 con 0x7f3058101650 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54398/0 (socket says 172.21.15.60:54398) 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f3055d9b640 1 -- 172.21.15.60:0/1540380494 learned_addr learned my addr 172.21.15.60:0/1540380494 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.243+0000 7f305659c640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 0x7f305819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 0x7f30581a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 -- 172.21.15.60:0/1540380494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 msgr2=0x7f305819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 0x7f305819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 -- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 msgr2=0x7f305819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 -- 172.21.15.60:0/1540380494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30581a0eb0 con 0x7f30581079e0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f305659c640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 0x7f305819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3055d9b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f3056d9d640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 0x7f30581a06e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3050005e30 tx=0x7f30500317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.244+0000 7f304f7fe640 1 -- 172.21.15.60:0/1540380494 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050003b70 con 0x7f30581079e0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.245+0000 7f304f7fe640 1 -- 172.21.15.60:0/1540380494 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3050003d10 con 0x7f30581079e0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.245+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581a1140 con 0x7f30581079e0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.245+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30581a7860 con 0x7f30581079e0 2024-07-24T21:50:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.245+0000 7f304f7fe640 1 -- 172.21.15.60:0/1540380494 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305002f970 con 0x7f30581079e0 2024-07-24T21:50:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 Processor -- start 2024-07-24T21:50:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f304f7fe640 1 -- 172.21.15.60:0/1540380494 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f305002fb10 con 0x7f30581079e0 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 -- start start 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 0x55f49b5be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f49b5359f0 con 0x55f49b5ca7d0 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f49b524ef0 con 0x55f49b5bdf20 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.248+0000 7f305ed2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f49b528f30 con 0x55f49b5be840 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 0x55f49b5be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 0x55f49b5be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53598/0 (socket says 172.21.15.60:53598) 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f3055d9b640 1 -- 172.21.15.60:0/323981486 learned_addr learned my addr 172.21.15.60:0/323981486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f3055d9b640 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 -- 172.21.15.60:0/323981486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 msgr2=0x55f49b5be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 0x55f49b5be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 -- 172.21.15.60:0/323981486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 msgr2=0x55f49b5ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 -- 172.21.15.60:0/323981486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49b285930 con 0x55f49b5be840 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f3055d9b640 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f305659c640 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5ca290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3048009fd0 tx=0x7f30480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a9e5a2019efc0b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f304f7fe640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 0x7f30440b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.249+0000 7f304cff9640 1 -- 172.21.15.60:0/323981486 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048004020 con 0x55f49b5be840 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f3055d9b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 0x7f30440b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f304cff9640 1 -- 172.21.15.60:0/323981486 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30480054d0 con 0x55f49b5be840 2024-07-24T21:50:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f304cff9640 1 -- 172.21.15.60:0/323981486 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048004c10 con 0x55f49b5be840 2024-07-24T21:50:42.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f304f7fe640 1 -- 172.21.15.60:0/1540380494 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f30501050f0 con 0x7f30581079e0 2024-07-24T21:50:42.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f305ed2c740 1 -- 172.21.15.60:0/323981486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 msgr2=0x55f49b5ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f305ed2c740 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5ca290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3048009fd0 tx=0x7f30480118d0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.250+0000 7f3055d9b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 0x7f30440b6280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3040009fd0 tx=0x7f3040013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- 172.21.15.60:0/323981486 shutdown_connections 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5ca290 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- 172.21.15.60:0/323981486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5bdf20 0x55f49b5be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- 172.21.15.60:0/323981486 >> 172.21.15.60:0/323981486 conn(0x55f49b5bbd50 msgr2=0x55f49b5bcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- 172.21.15.60:0/323981486 shutdown_connections 2024-07-24T21:50:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- 172.21.15.60:0/323981486 wait complete. 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- start start 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 Processor -- start 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f49b388890 con 0x55f49b5ca7d0 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f49b4c7390 con 0x55f49b5be840 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.251+0000 7f305ed2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f49b5cd450 con 0x55f49b5bdf20 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305ed2c740 10 client.0 _ll_register_callbacks cb 0x55f49b4d1500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53606/0 (socket says 172.21.15.60:53606) 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3055d9b640 1 -- 172.21.15.60:0/722046361 learned_addr learned my addr 172.21.15.60:0/722046361 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3056d9d640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 -- 172.21.15.60:0/722046361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 msgr2=0x55f49b5c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 -- 172.21.15.60:0/722046361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 msgr2=0x55f49b5c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 -- 172.21.15.60:0/722046361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49b4c1d60 con 0x55f49b5bdf20 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3055d9b640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f3056d9d640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f305659c640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5ca210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f304801b040 tx=0x7f304800fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7cccc056dc0fdf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f303dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.252+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048028020 con 0x55f49b5bdf20 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3048023470 con 0x55f49b5bdf20 2024-07-24T21:50:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30480054d0 con 0x55f49b5bdf20 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3048033050 con 0x55f49b5bdf20 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 10 client.26053 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f49b5d9300 con 0x55f49b5bdf20 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30480043c0 con 0x55f49b5bdf20 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f304802f4b0 con 0x55f49b5bdf20 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 10 client.26053 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 20 client.26053 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b5da760 0x55f49b5dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 4 client.26053 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30480054d0 con 0x55f49b5da760 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.253+0000 7f3055d9b640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b5da760 0x55f49b5dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.254+0000 7f3055d9b640 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b5da760 0x55f49b5dcb50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cb72cb268e60f2 server_cookie=b10cbeabb343a2df in_seq=0 out_seq=0 2024-07-24T21:50:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.254+0000 7f303dffb640 10 client.26053 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f303dffb640 1 -- 172.21.15.60:0/722046361 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f30480054d0 con 0x55f49b5da760 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f303dffb640 10 client.26053 handle_command_reply: tid=0 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 client.26053 shutdown 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 msgr2=0x55f49b5ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5ca210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f304801b040 tx=0x7f304800fcf0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 shutdown_connections 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b5da760 0x55f49b5dcb50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 --2- 172.21.15.60:0/722046361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5ca210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 >> 172.21.15.60:0/722046361 conn(0x55f49b5bbd50 msgr2=0x55f49b5c18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f303dffb640 0 client.26053 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 shutdown_connections 2024-07-24T21:50:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 1 -- 172.21.15.60:0/722046361 wait complete. 2024-07-24T21:50:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.256+0000 7f305ed2c740 20 client.26053 trim_cache size 0 max 16384 2024-07-24T21:50:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 Processor -- start 2024-07-24T21:50:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 -- start start 2024-07-24T21:50:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5ca7d0 0x55f49b5bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3038011150 con 0x55f49b5bdf20 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f303800e9c0 con 0x55f49b5ca7d0 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.278+0000 7f305ed2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f303800c810 con 0x55f49b5be840 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54430/0 (socket says 172.21.15.60:54430) 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 -- 172.21.15.60:0/3146342003 learned_addr learned my addr 172.21.15.60:0/3146342003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f3055d9b640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f3056d9d640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5ca7d0 0x55f49b5bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 -- 172.21.15.60:0/3146342003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 msgr2=0x55f49b5c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 -- 172.21.15.60:0/3146342003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5ca7d0 msgr2=0x55f49b5bd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5ca7d0 0x55f49b5bd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 -- 172.21.15.60:0/3146342003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49b5bec20 con 0x55f49b5bdf20 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f3055d9b640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f305659c640 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe1dcffe363afe9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.279+0000 7f303ffff640 1 -- 172.21.15.60:0/3146342003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304801d040 con 0x55f49b5bdf20 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.280+0000 7f303ffff640 1 -- 172.21.15.60:0/3146342003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048005590 con 0x55f49b5bdf20 2024-07-24T21:50:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.280+0000 7f303ffff640 1 -- 172.21.15.60:0/3146342003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480035f0 con 0x55f49b5bdf20 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.280+0000 7f305ed2c740 1 -- 172.21.15.60:0/3146342003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 msgr2=0x55f49b5c2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.280+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005970 comp rx=0 tx=0).stop 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.280+0000 7f305ed2c740 1 -- 172.21.15.60:0/3146342003 shutdown_connections 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5ca7d0 0x55f49b5bd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5be840 0x55f49b5c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3146342003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5bdf20 0x55f49b5c2070 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- 172.21.15.60:0/3146342003 >> 172.21.15.60:0/3146342003 conn(0x55f49b5bbd50 msgr2=0x55f49b5bca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- 172.21.15.60:0/3146342003 shutdown_connections 2024-07-24T21:50:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- 172.21.15.60:0/3146342003 wait complete. 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- start start 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 Processor -- start 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f303800ce80 con 0x55f49b5ca7d0 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f303800e2f0 con 0x55f49b5be840 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.281+0000 7f305ed2c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f303800d550 con 0x55f49b5bdf20 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f305ed2c740 10 client.0 _ll_register_callbacks cb 0x7f304800f980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f305659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f305659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58494/0 (socket says 172.21.15.60:58494) 2024-07-24T21:50:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f305659c640 1 -- 172.21.15.60:0/3437955229 learned_addr learned my addr 172.21.15.60:0/3437955229 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3055d9b640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 -- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 msgr2=0x55f49b5d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5d2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 -- 172.21.15.60:0/3437955229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 msgr2=0x55f49b5ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 -- 172.21.15.60:0/3437955229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49b5ceb70 con 0x55f49b5ca7d0 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f3056d9d640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ce5b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3050102950 tx=0x7f30500cb580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe920665cf7fe2bf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f303dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.282+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500e5070 con 0x55f49b5ca7d0 2024-07-24T21:50:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f30500d4a30 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f49b5ccfa0 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f30500cbe10 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 10 client.16566 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f49b645240 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f49b5ccfa0 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f30500cb190 con 0x55f49b5ca7d0 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 10 client.16566 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 20 client.16566 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b645a40 0x55f49b647e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 4 client.16566 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.283+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f49b537940 con 0x55f49b645a40 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.284+0000 7f305659c640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b645a40 0x55f49b647e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.284+0000 7f305659c640 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b645a40 0x55f49b647e60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e58b2c6a94b7b9 server_cookie=927ab157203bfc2 in_seq=0 out_seq=0 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.284+0000 7f303dffb640 10 client.16566 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f303dffb640 1 -- 172.21.15.60:0/3437955229 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f49b537940 con 0x55f49b645a40 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f303dffb640 10 client.16566 handle_command_reply: tid=0 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 client.16566 shutdown 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 msgr2=0x55f49b5ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ce5b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3050102950 tx=0x7f30500cb580 comp rx=0 tx=0).stop 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 shutdown_connections 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f49b645a40 0x55f49b647e60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f303dffb640 0 client.16566 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f49b5ca7d0 0x55f49b5ce5b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f49b5be840 0x55f49b5ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 --2- 172.21.15.60:0/3437955229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f49b5bdf20 0x55f49b5d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 >> 172.21.15.60:0/3437955229 conn(0x55f49b5bbd50 msgr2=0x55f49b5bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 shutdown_connections 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 1 -- 172.21.15.60:0/3437955229 wait complete. 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.285+0000 7f305ed2c740 20 client.16566 trim_cache size 0 max 16384 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "f0085476-1f73-4156-8226-3ba9628a3514", 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:50:42.294 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 msgr2=0x7f30440b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 0x7f30440b6280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3040009fd0 tx=0x7f3040013040 comp rx=0 tx=0).stop 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 msgr2=0x7f30581a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 0x7f30581a06e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3050005e30 tx=0x7f30500317c0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 shutdown_connections 2024-07-24T21:50:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30440b3dc0 0x7f30440b6280 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30581079e0 0x7f30581a06e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3058107060 0x7f305819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 --2- 172.21.15.60:0/1540380494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3058101650 0x7f305819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 >> 172.21.15.60:0/1540380494 conn(0x7f30580fd0c0 msgr2=0x7f305806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 shutdown_connections 2024-07-24T21:50:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:42.286+0000 7f305c97b640 1 -- 172.21.15.60:0/1540380494 wait complete. 2024-07-24T21:50:42.743 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:50:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:50:42.380+0000 7f827ed99640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000881 [...6,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11/ auth{1=1,2=1} fragtree_t(*^3) v138111 ap=3 f(v2 m2024-07-24T21:49:40.821482+0000 215=215+0) n(v6 rc2024-07-24T21:49:40.821482+0000 b3299152 216=215+1) old_inodes=2 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:9050 nref=3)) caps={15300=pAsLsXsFs/pAsLsXsFsxcral@74} | ptrwaiter=0 dirtyscattered=0 request=1 lock=2 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x55de91653080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000881:[<0x10000000399/blog-11 v10780>,<0x10000000208/blogtest_in v345>,<0x10000000201/src v2099>,<0x10000000006/blogbench-1.0 v1818>,<0x10000000005/tmp v50>,<0x100000001fb/client.0 v40>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v60>,<0x10000000001/sv_1 v80>,<0x10000000000/qa v69>,<0x1/volumes v66>]//[]","memoryvalue":"(2)0x10000000881:[<0x10000000399/blog-11 v138111>,<0x10000000208/blogtest_in v1490>,<0x10000000201/src v2172>,<0x10000000006/blogbench-1.0 v1894>,<0x10000000005/tmp v110>,<0x100000001fb/client.0 v106>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v110>,<0x10000000001/sv_1 v133>,<0x10000000000/qa v124>,<0x1/volumes v108>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:49:40.821482+0000 216=216+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:49:40.821482+0000 b3299152 216=215+1)","memory_value.dirstat":"f(v2 m2024-07-24T21:49:40.821482+0000 215=215+0)","memory_value.rstat":"n(v6 rc2024-07-24T21:49:40.821482+0000 b3299152 216=215+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:50:42.743 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:50:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:50:42.535+0000 7f827ed99640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000192c [...6,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-46/ auth{1=1,2=1} v136018 ap=3 f(v3 m2024-07-24T21:49:43.759454+0000 48=48+0) n(v5 rc2024-07-24T21:49:43.759454+0000 b1122323 49=48+1) old_inodes=2 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:9070 nref=3)) caps={15300=pAsLsXsFs/pAsLsXsFsxcral@8} | dirtyscattered=0 request=1 lock=2 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyrstat=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55de922ef600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x2000000192c:[<0x10000000399/blog-46 v127145>,<0x10000000208/blogtest_in v1201>,<0x10000000201/src v2125>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v82>,<0x100000001fb/client.0 v70>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v86>,<0x10000000001/sv_1 v109>,<0x10000000000/qa v100>,<0x1/volumes v94>]//[]","memoryvalue":"(2)0x2000000192c:[<0x10000000399/blog-46 v136018>,<0x10000000208/blogtest_in v1490>,<0x10000000201/src v2172>,<0x10000000006/blogbench-1.0 v1894>,<0x10000000005/tmp v110>,<0x100000001fb/client.0 v106>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v110>,<0x10000000001/sv_1 v133>,<0x10000000000/qa v124>,<0x1/volumes v108>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:49:31.401570+0000 42=42+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:49:43.759454+0000 b1122323 49=48+1)","memory_value.dirstat":"f(v3 m2024-07-24T21:49:43.759454+0000 48=48+0)","memory_value.rstat":"n(v5 rc2024-07-24T21:49:43.759454+0000 b1122323 49=48+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:50:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 -- 172.21.15.60:0/3109583572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef81035c0 msgr2=0x7fbef81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 --2- 172.21.15.60:0/3109583572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef81035c0 0x7fbef81039a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009be0 tx=0x7fbeec02a6c0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 -- 172.21.15.60:0/3109583572 shutdown_connections 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 --2- 172.21.15.60:0/3109583572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 0x7fbef810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 --2- 172.21.15.60:0/3109583572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef8103ee0 0x7fbef810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 --2- 172.21.15.60:0/3109583572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef81035c0 0x7fbef81039a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 -- 172.21.15.60:0/3109583572 >> 172.21.15.60:0/3109583572 conn(0x7fbef80fcfb0 msgr2=0x7fbef80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 -- 172.21.15.60:0/3109583572 shutdown_connections 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.185+0000 7fbefed59640 1 -- 172.21.15.60:0/3109583572 wait complete. 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 Processor -- start 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 -- start start 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 0x7fbef8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 0x7fbef819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8112680 con 0x7fbef8103ee0 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8112500 con 0x7fbef81035c0 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefed59640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8112800 con 0x7fbef810ca10 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 0x7fbef8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54456/0 (socket says 172.21.15.60:54456) 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbef7fff640 1 -- 172.21.15.60:0/1173871132 learned_addr learned my addr 172.21.15.60:0/1173871132 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.186+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 0x7fbef819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1173871132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 msgr2=0x7fbef819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 0x7fbef819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 msgr2=0x7fbef8199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1173871132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef819ed60 con 0x7fbef81035c0 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbef7fff640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 0x7fbef8198cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbeec02abe0 tx=0x7fbeec0305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbef5ffb640 1 -- 172.21.15.60:0/1173871132 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec041020 con 0x7fbef81035c0 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef819eff0 con 0x7fbef81035c0 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbef5ffb640 1 -- 172.21.15.60:0/1173871132 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec030e20 con 0x7fbef81035c0 2024-07-24T21:50:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbef5ffb640 1 -- 172.21.15.60:0/1173871132 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec036040 con 0x7fbef81035c0 2024-07-24T21:50:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.187+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef81a5710 con 0x7fbef81035c0 2024-07-24T21:50:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbef5ffb640 1 -- 172.21.15.60:0/1173871132 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fbeec044020 con 0x7fbef81035c0 2024-07-24T21:50:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 Processor -- start 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 -- start start 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368113fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 0x563681140280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681152820 0x563681154c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5636810b3280 con 0x563681152820 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636810a2780 con 0x56368113f960 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.191+0000 7fbf01b86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5636810a67c0 con 0x563681147c20 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefd2f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368113fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 0x563681140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefd2f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368113fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53660/0 (socket says 172.21.15.60:53660) 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefd2f6640 1 -- 172.21.15.60:0/536391895 learned_addr learned my addr 172.21.15.60:0/536391895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbef7fff640 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681152820 0x563681154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 -- 172.21.15.60:0/536391895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 msgr2=0x56368113fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368113fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 -- 172.21.15.60:0/536391895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681152820 msgr2=0x563681154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681152820 0x563681154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 -- 172.21.15.60:0/536391895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563680e06790 con 0x563681147c20 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbef5ffb640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 0x7fbee80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 0x563681140280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbeec102220 tx=0x7fbeec108d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326721acd80bfa02 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.192+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 0x7fbee80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.193+0000 7fbef5ffb640 1 -- 172.21.15.60:0/1173871132 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbeec104ad0 con 0x7fbef81035c0 2024-07-24T21:50:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.193+0000 7fbee2ffd640 1 -- 172.21.15.60:0/536391895 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0c75a0 con 0x563681147c20 2024-07-24T21:50:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.193+0000 7fbee2ffd640 1 -- 172.21.15.60:0/536391895 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0cb1f0 con 0x563681147c20 2024-07-24T21:50:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.193+0000 7fbee2ffd640 1 -- 172.21.15.60:0/536391895 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0cae50 con 0x563681147c20 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- 172.21.15.60:0/536391895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 msgr2=0x563681140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 0x563681140280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbeec102220 tx=0x7fbeec108d90 comp rx=0 tx=0).stop 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 0x7fbee80b6280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009e60 tx=0x7fbef0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- 172.21.15.60:0/536391895 shutdown_connections 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681152820 0x563681154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681147c20 0x563681140280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 --2- 172.21.15.60:0/536391895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368113fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- 172.21.15.60:0/536391895 >> 172.21.15.60:0/536391895 conn(0x56368113b6f0 msgr2=0x56368113c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- 172.21.15.60:0/536391895 shutdown_connections 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- 172.21.15.60:0/536391895 wait complete. 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 -- start start 2024-07-24T21:50:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.194+0000 7fbf01b86740 1 Processor -- start 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681144930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563680fdcab0 con 0x56368113f960 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56368102a9e0 con 0x563681147c20 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563681155400 con 0x563681152820 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbf01b86740 10 client.0 _ll_register_callbacks cb 0x563680db2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54472/0 (socket says 172.21.15.60:54472) 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 -- 172.21.15.60:0/666148574 learned_addr learned my addr 172.21.15.60:0/666148574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbef7fff640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 -- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 msgr2=0x563681144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 -- 172.21.15.60:0/666148574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 msgr2=0x563681144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 -- 172.21.15.60:0/666148574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563680e7f620 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbeec1022d0 tx=0x7fbeec003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505dbe47b378f033 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.195+0000 7fbed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0cb030 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbeec0c7cd0 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563681051ce0 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbeec0cb4b0 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbf01b86740 10 client.16578 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.196+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563681160bb0 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563681051ce0 con 0x56368113f960 2024-07-24T21:50:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbeec003c00 con 0x56368113f960 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 10 client.16578 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 20 client.16578 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563681161f00 0x5636811642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 4 client.16578 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636810b51d0 con 0x563681161f00 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.197+0000 7fbef7fff640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563681161f00 0x5636811642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.198+0000 7fbef7fff640 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563681161f00 0x5636811642f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6066b475ec09f597 server_cookie=3d37b10831b1480d in_seq=0 out_seq=0 2024-07-24T21:50:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.198+0000 7fbed7fff640 10 client.16578 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.200+0000 7fbed7fff640 1 -- 172.21.15.60:0/666148574 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636810b51d0 con 0x563681161f00 2024-07-24T21:50:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.200+0000 7fbed7fff640 10 client.16578 handle_command_reply: tid=0 2024-07-24T21:50:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.200+0000 7fbf01b86740 1 client.16578 shutdown 2024-07-24T21:50:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.200+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 msgr2=0x563681147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.200+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbeec1022d0 tx=0x7fbeec003b70 comp rx=0 tx=0).stop 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 shutdown_connections 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563681161f00 0x5636811642f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbed7fff640 0 client.16578 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 --2- 172.21.15.60:0/666148574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56368113f960 0x563681147450 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 >> 172.21.15.60:0/666148574 conn(0x56368113b6f0 msgr2=0x563681139f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 shutdown_connections 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 1 -- 172.21.15.60:0/666148574 wait complete. 2024-07-24T21:50:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.201+0000 7fbf01b86740 20 client.16578 trim_cache size 0 max 16384 2024-07-24T21:50:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 Processor -- start 2024-07-24T21:50:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 -- start start 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 0x56368113fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681140280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81ae8f0 con 0x563681147c20 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81ac160 con 0x56368113f960 2024-07-24T21:50:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbf01b86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81a9fb0 con 0x563681152820 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53670/0 (socket says 172.21.15.60:53670) 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.230+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1603451562 learned_addr learned my addr 172.21.15.60:0/1603451562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbef7fff640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 0x56368113fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1603451562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 msgr2=0x563681140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1603451562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 msgr2=0x56368113fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 0x56368113fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 -- 172.21.15.60:0/1603451562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636811648b0 con 0x56368113f960 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbef7fff640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 0x56368113fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009b30 tx=0x7fbeec004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee990fc6cb552b67 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.231+0000 7fbee27fc640 1 -- 172.21.15.60:0/1603451562 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0cb030 con 0x56368113f960 2024-07-24T21:50:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbee27fc640 1 -- 172.21.15.60:0/1603451562 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0cb6e0 con 0x56368113f960 2024-07-24T21:50:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbee27fc640 1 -- 172.21.15.60:0/1603451562 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0307a0 con 0x56368113f960 2024-07-24T21:50:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 -- 172.21.15.60:0/1603451562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 msgr2=0x56368115e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009b30 tx=0x7fbeec004040 comp rx=0 tx=0).stop 2024-07-24T21:50:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 -- 172.21.15.60:0/1603451562 shutdown_connections 2024-07-24T21:50:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681147c20 0x56368113fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 --2- 172.21.15.60:0/1603451562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56368113f960 0x56368115e4e0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009b30 tx=0x7fbeec004040 comp rx=0 tx=0).stop 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 -- 172.21.15.60:0/1603451562 >> 172.21.15.60:0/1603451562 conn(0x56368113b6f0 msgr2=0x56368113d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.232+0000 7fbf01b86740 1 -- 172.21.15.60:0/1603451562 shutdown_connections 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 -- 172.21.15.60:0/1603451562 wait complete. 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 -- start start 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 Processor -- start 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681169680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 0x563681159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81aa620 con 0x563681159650 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81aba90 con 0x563681147c20 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.233+0000 7fbf01b86740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbef81aacf0 con 0x563681152820 2024-07-24T21:50:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbf01b86740 10 client.0 _ll_register_callbacks cb 0x7fbeec104f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefd2f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 0x563681159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53684/0 (socket says 172.21.15.60:53684) 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 -- 172.21.15.60:0/2611474210 learned_addr learned my addr 172.21.15.60:0/2611474210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbef7fff640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 -- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 msgr2=0x563681169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 -- 172.21.15.60:0/2611474210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 msgr2=0x563681159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 0x563681159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 -- 172.21.15.60:0/2611474210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56368115a010 con 0x563681147c20 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbef7fff640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefd2f6640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 0x563681159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbefcaf5640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0cb3e0 tx=0x7fbeec030fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f011bf2ff9fa9c43 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.234+0000 7fbed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0aa060 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbeec0c7cd0 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec0cb6e0 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbeec0307a0 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbf01b86740 10 client.26057 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56368115a010 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.235+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0cfce0 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbeec0ecbe0 con 0x563681147c20 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 20 client.26057 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5636811ccc70 0x5636811cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 4 client.26057 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:50:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbeec0cb6e0 con 0x5636811ccc70 2024-07-24T21:50:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.236+0000 7fbef7fff640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5636811ccc70 0x5636811cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.237+0000 7fbef7fff640 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5636811ccc70 0x5636811cf060 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576abce6e386300a server_cookie=9b339019bfa76042 in_seq=0 out_seq=0 2024-07-24T21:50:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:43.237+0000 7fbed7fff640 10 client.26057 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:43.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: pgmap v498: 97 pgs: 97 active+clean; 817 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 763 op/s 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:43.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:43.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: pgmap v498: 97 pgs: 97 active+clean; 817 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 763 op/s 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:43.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:43.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: pgmap v498: 97 pgs: 97 active+clean; 817 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 763 op/s 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:43.613 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:44.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:44 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:44.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:44 smithi060 ceph-mon[33000]: Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:44 smithi060 ceph-mon[33000]: Scrub error on inode 0x2000000192c (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-46) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:44 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:44.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:44 smithi123 ceph-mon[37143]: Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:44 smithi123 ceph-mon[37143]: Scrub error on inode 0x2000000192c (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-46) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:44 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:44.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:44 smithi086 ceph-mon[37128]: Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:44 smithi086 ceph-mon[37128]: Scrub error on inode 0x2000000192c (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-46) see mds.i log and `damage ls` output for details 2024-07-24T21:50:44.708 INFO:tasks.workunit.client.0.smithi060.stdout: 218 45037 0 31203 0 14321 0 2024-07-24T21:50:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbed7fff640 1 -- 172.21.15.60:0/2611474210 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbeec0cb6e0 con 0x5636811ccc70 2024-07-24T21:50:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbed7fff640 10 client.26057 handle_command_reply: tid=0 2024-07-24T21:50:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 client.26057 shutdown 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 msgr2=0x563681169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0cb3e0 tx=0x7fbeec030fd0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 shutdown_connections 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5636811ccc70 0x5636811cf060 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563681159650 0x563681159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563681152820 0x563681169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbed7fff640 0 client.26057 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 --2- 172.21.15.60:0/2611474210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563681147c20 0x563681169140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 >> 172.21.15.60:0/2611474210 conn(0x56368113b6f0 msgr2=0x563681139f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 shutdown_connections 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 1 -- 172.21.15.60:0/2611474210 wait complete. 2024-07-24T21:50:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.048+0000 7fbf01b86740 20 client.26057 trim_cache size 0 max 16384 2024-07-24T21:50:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:50:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:50:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:50:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 msgr2=0x7fbee80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 0x7fbee80b6280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009e60 tx=0x7fbef0005f90 comp rx=0 tx=0).stop 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 msgr2=0x7fbef8198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 0x7fbef8198cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbeec02abe0 tx=0x7fbeec0305b0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 shutdown_connections 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbee80b3dc0 0x7fbee80b6280 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbef810ca10 0x7fbef819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbef8103ee0 0x7fbef8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 --2- 172.21.15.60:0/1173871132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef81035c0 0x7fbef8198cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 >> 172.21.15.60:0/1173871132 conn(0x7fbef80fcfb0 msgr2=0x7fbef80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 shutdown_connections 2024-07-24T21:50:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:50:45.049+0000 7fbefed59640 1 -- 172.21.15.60:0/1173871132 wait complete. 2024-07-24T21:50:45.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:45 smithi060 ceph-mon[33000]: pgmap v499: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 850 op/s 2024-07-24T21:50:45.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:45 smithi123 ceph-mon[37143]: pgmap v499: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 850 op/s 2024-07-24T21:50:45.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:45 smithi086 ceph-mon[37128]: pgmap v499: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 850 op/s 2024-07-24T21:50:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 -- 172.21.15.60:0/2592280137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 msgr2=0x7f05ec10f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 --2- 172.21.15.60:0/2592280137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec10f300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009ca0 tx=0x7f05e402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 -- 172.21.15.60:0/2592280137 shutdown_connections 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 --2- 172.21.15.60:0/2592280137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec10f300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 --2- 172.21.15.60:0/2592280137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec108180 0x7f05ec1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 --2- 172.21.15.60:0/2592280137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec102770 0x7f05ec102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.881+0000 7f05f24c6640 1 -- 172.21.15.60:0/2592280137 >> 172.21.15.60:0/2592280137 conn(0x7f05ec0fe1c0 msgr2=0x7f05ec1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.882+0000 7f05f24c6640 1 -- 172.21.15.60:0/2592280137 shutdown_connections 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.882+0000 7f05f24c6640 1 -- 172.21.15.60:0/2592280137 wait complete. 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.882+0000 7f05f24c6640 1 Processor -- start 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 -- start start 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 0x7f05ec0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec112750 con 0x7f05ec108180 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1125d0 con 0x7f05ec102770 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05f24c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1128d0 con 0x7f05ec108b00 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34958/0 (socket says 172.21.15.60:34958) 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05ebfff640 1 -- 172.21.15.60:0/3994871878 learned_addr learned my addr 172.21.15.60:0/3994871878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.883+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 0x7f05ec0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05f0a63640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 -- 172.21.15.60:0/3994871878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 msgr2=0x7f05ec075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 -- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 msgr2=0x7f05ec0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 0x7f05ec0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 -- 172.21.15.60:0/3994871878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1a7060 con 0x7f05ec102770 2024-07-24T21:50:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 0x7f05ec0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.884+0000 7f05ebfff640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9f0 tx=0x7f05dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.885+0000 7f05e97fa640 1 -- 172.21.15.60:0/3994871878 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00cd80 con 0x7f05ec102770 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.885+0000 7f05e97fa640 1 -- 172.21.15.60:0/3994871878 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc0054d0 con 0x7f05ec102770 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.885+0000 7f05e97fa640 1 -- 172.21.15.60:0/3994871878 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010690 con 0x7f05ec102770 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.885+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec1a7350 con 0x7f05ec102770 2024-07-24T21:50:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.885+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a7840 con 0x7f05ec102770 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.888+0000 7f05e97fa640 1 -- 172.21.15.60:0/3994871878 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f05dc004e90 con 0x7f05ec102770 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05e97fa640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 0x7f05d40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 Processor -- start 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 0x7f05d40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 -- start start 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05e97fa640 1 -- 172.21.15.60:0/3994871878 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f05dc0e4360 con 0x7f05ec102770 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 0x55edf07e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55edf074c960 con 0x55edf07d5fa0 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55edf073be60 con 0x55edf07e1f30 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.889+0000 7f05f4877740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55edf073fea0 con 0x55edf07d5680 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54610/0 (socket says 172.21.15.60:54610) 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05ebfff640 1 -- 172.21.15.60:0/907412456 learned_addr learned my addr 172.21.15.60:0/907412456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 0x55edf07e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 -- 172.21.15.60:0/907412456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 msgr2=0x55edf07d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 -- 172.21.15.60:0/907412456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 msgr2=0x55edf07e19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 0x55edf07e19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05f0a63640 1 -- 172.21.15.60:0/907412456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf06c52d0 con 0x55edf07e1f30 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05ebfff640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 0x55edf07e19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.890+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 0x7f05d40b6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f05ec076980 tx=0x7f05e0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.891+0000 7f05f0a63640 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e4320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05ec076f50 tx=0x7f05e4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112b6fd2b0c75d9c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.891+0000 7f05d27fc640 1 -- 172.21.15.60:0/907412456 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4031cf0 con 0x55edf07e1f30 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.891+0000 7f05d27fc640 1 -- 172.21.15.60:0/907412456 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e4036030 con 0x55edf07e1f30 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.891+0000 7f05d27fc640 1 -- 172.21.15.60:0/907412456 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e40042a0 con 0x55edf07e1f30 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.892+0000 7f05f4877740 1 -- 172.21.15.60:0/907412456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 msgr2=0x55edf07e4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.892+0000 7f05f4877740 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e4320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05ec076f50 tx=0x7f05e4005ef0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 -- 172.21.15.60:0/907412456 shutdown_connections 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e4320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5fa0 0x55edf07e19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 --2- 172.21.15.60:0/907412456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 -- 172.21.15.60:0/907412456 >> 172.21.15.60:0/907412456 conn(0x55edf07d34b0 msgr2=0x55edf07d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 -- 172.21.15.60:0/907412456 shutdown_connections 2024-07-24T21:50:45.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 -- 172.21.15.60:0/907412456 wait complete. 2024-07-24T21:50:45.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.893+0000 7f05f4877740 1 -- start start 2024-07-24T21:50:45.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 Processor -- start 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07df490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55edf059f700 con 0x55edf07d5680 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55edf06de5a0 con 0x55edf07e1f30 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55edf07e49d0 con 0x55edf07d5fa0 2024-07-24T21:50:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f4877740 10 client.0 _ll_register_callbacks cb 0x55edf06fe910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f0a63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f0a63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34978/0 (socket says 172.21.15.60:34978) 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05f0a63640 1 -- 172.21.15.60:0/1387348493 learned_addr learned my addr 172.21.15.60:0/1387348493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.894+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 -- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 msgr2=0x55edf07df490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07df490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 -- 172.21.15.60:0/1387348493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 msgr2=0x55edf07def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 -- 172.21.15.60:0/1387348493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf06d1630 con 0x55edf07e1f30 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05e4031d00 tx=0x7f05e4031d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ddd76af75c07d84 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e404a040 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f05e4031650 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.895+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e4036030 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f05e405e050 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 10 client.26069 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55edf049c650 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e4036030 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f05e404d030 con 0x55edf07e1f30 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 20 client.26069 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf07f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 4 client.26069 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:45.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf074e8b0 con 0x55edf07f1760 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.896+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf07f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.897+0000 7f05cb7fe640 10 client.26069 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.897+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf07f3b50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9f904ba0f70b72 server_cookie=fde4dc5d80ef6218 in_seq=0 out_seq=0 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.897+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1387348493 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edf074e8b0 con 0x55edf07f1760 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.897+0000 7f05cb7fe640 10 client.26069 handle_command_reply: tid=0 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.897+0000 7f05f4877740 1 client.26069 shutdown 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 msgr2=0x55edf07df9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05e4031d00 tx=0x7f05e4031d30 comp rx=0 tx=0).stop 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 shutdown_connections 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf07f3b50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05cb7fe640 0 client.26069 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07df9d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07df490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 --2- 172.21.15.60:0/1387348493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 >> 172.21.15.60:0/1387348493 conn(0x55edf07d34b0 msgr2=0x55edf07d9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 shutdown_connections 2024-07-24T21:50:45.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 1 -- 172.21.15.60:0/1387348493 wait complete. 2024-07-24T21:50:45.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.898+0000 7f05f4877740 20 client.26069 trim_cache size 0 max 16384 2024-07-24T21:50:45.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 Processor -- start 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 -- start start 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 0x55edf07f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1ae750 con 0x55edf07e1f30 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1abfc0 con 0x55edf07d5fa0 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f4877740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1a9e10 con 0x55edf07d5680 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f0a63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 0x55edf07f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05f0a63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 0x55edf07f9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43032/0 (socket says 172.21.15.60:43032) 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54624/0 (socket says 172.21.15.60:54624) 2024-07-24T21:50:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05ebfff640 1 -- 172.21.15.60:0/1537553125 learned_addr learned my addr 172.21.15.60:0/1537553125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.921+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34988/0 (socket says 172.21.15.60:34988) 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 -- 172.21.15.60:0/1537553125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 msgr2=0x55edf07f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 -- 172.21.15.60:0/1537553125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 msgr2=0x55edf07f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 0x55edf07f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 -- 172.21.15.60:0/1537553125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf07d5dd0 con 0x55edf07d5680 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0e4330 tx=0x7f05dc005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3055af2a516816aa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05d1ffb640 1 -- 172.21.15.60:0/1537553125 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0c4070 con 0x55edf07d5680 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05d1ffb640 1 -- 172.21.15.60:0/1537553125 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc0aaa80 con 0x55edf07d5680 2024-07-24T21:50:45.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.922+0000 7f05d1ffb640 1 -- 172.21.15.60:0/1537553125 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0a7200 con 0x55edf07d5680 2024-07-24T21:50:45.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- 172.21.15.60:0/1537553125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 msgr2=0x55edf07d97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0e4330 tx=0x7f05dc005700 comp rx=0 tx=0).stop 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- 172.21.15.60:0/1537553125 shutdown_connections 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07e1f30 0x55edf07f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07d5fa0 0x55edf07f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 --2- 172.21.15.60:0/1537553125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5680 0x55edf07d97d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- 172.21.15.60:0/1537553125 >> 172.21.15.60:0/1537553125 conn(0x55edf07d34b0 msgr2=0x55edf07dd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- 172.21.15.60:0/1537553125 shutdown_connections 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- 172.21.15.60:0/1537553125 wait complete. 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.923+0000 7f05f4877740 1 -- start start 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 Processor -- start 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1aa480 con 0x55edf07d5680 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1ab8f0 con 0x55edf07e1f30 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1aab50 con 0x55edf07d5fa0 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f4877740 10 client.0 _ll_register_callbacks cb 0x7f05e0009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54634/0 (socket says 172.21.15.60:54634) 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05eb7fe640 1 -- 172.21.15.60:0/1535801338 learned_addr learned my addr 172.21.15.60:0/1535801338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 -- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 msgr2=0x55edf07ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 -- 172.21.15.60:0/1535801338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 msgr2=0x55edf07e9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:45.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07e9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05f0a63640 1 -- 172.21.15.60:0/1535801338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf07e7170 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.924+0000 7f05eb7fe640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05f0a63640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e6c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05e402ade0 tx=0x7f05e4005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1719e943efdb9301 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4031860 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f05e4030730 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05dc0aaa80 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f05e4030930 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05f4877740 10 client.26075 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55edf085c8d0 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.925+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc0aaa80 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f05e404d040 con 0x55edf07e1f30 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 20 client.26075 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf085f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 4 client.26075 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edf074a350 con 0x55edf07f1760 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf085f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.926+0000 7f05cb7fe640 10 client.26075 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05ebfff640 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf085f120 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e5e36aabd3e882 server_cookie=e5ae4d3c95ec06b8 in_seq=0 out_seq=0 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05cb7fe640 1 -- 172.21.15.60:0/1535801338 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55edf074a350 con 0x55edf07f1760 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05cb7fe640 10 client.26075 handle_command_reply: tid=0 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 client.26075 shutdown 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 msgr2=0x55edf07e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e6c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05e402ade0 tx=0x7f05e4005f70 comp rx=0 tx=0).stop 2024-07-24T21:50:45.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 shutdown_connections 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55edf07f1760 0x55edf085f120 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edf07e1f30 0x55edf07e6c30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf07d5fa0 0x55edf07ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 --2- 172.21.15.60:0/1535801338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55edf07d5680 0x55edf07e9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 >> 172.21.15.60:0/1535801338 conn(0x55edf07d34b0 msgr2=0x55edf07deec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05cb7fe640 0 client.26075 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 shutdown_connections 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 1 -- 172.21.15.60:0/1535801338 wait complete. 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.927+0000 7f05f4877740 20 client.26075 trim_cache size 0 max 16384 2024-07-24T21:50:45.934 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "f0085476-1f73-4156-8226-3ba9628a3514": { 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "f0085476-1f73-4156-8226-3ba9628a3514", 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:45.935 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 msgr2=0x7f05d40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 0x7f05d40b6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f05ec076980 tx=0x7f05e0005f90 comp rx=0 tx=0).stop 2024-07-24T21:50:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 msgr2=0x7f05ec074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9f0 tx=0x7f05dc00edf0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 shutdown_connections 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05d40b3dc0 0x7f05d40b6280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ec108b00 0x7f05ec075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05ec108180 0x7f05ec0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.928+0000 7f05f24c6640 1 --2- 172.21.15.60:0/3994871878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05ec102770 0x7f05ec074e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.929+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 >> 172.21.15.60:0/3994871878 conn(0x7f05ec0fe1c0 msgr2=0x7f05ec1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.929+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 shutdown_connections 2024-07-24T21:50:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:45.929+0000 7f05f24c6640 1 -- 172.21.15.60:0/3994871878 wait complete. 2024-07-24T21:50:46.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:46 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:46.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:46 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:46.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:46 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:46.867 INFO:tasks.cephfs.filesystem:scrub status for tag:f0085476-1f73-4156-8226-3ba9628a3514 - {'path': '/', 'tag': 'f0085476-1f73-4156-8226-3ba9628a3514', 'options': 'recursive,force'} 2024-07-24T21:50:47.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:47 smithi123 ceph-mon[37143]: pgmap v500: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2024-07-24T21:50:47.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:47 smithi086 ceph-mon[37128]: pgmap v500: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2024-07-24T21:50:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:47 smithi060 ceph-mon[33000]: pgmap v500: 97 pgs: 97 active+clean; 822 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2024-07-24T21:50:48.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:50:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:49 smithi060 ceph-mon[33000]: pgmap v501: 97 pgs: 97 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.98k op/s 2024-07-24T21:50:49.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:49 smithi123 ceph-mon[37143]: pgmap v501: 97 pgs: 97 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.98k op/s 2024-07-24T21:50:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:49 smithi086 ceph-mon[37128]: pgmap v501: 97 pgs: 97 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.98k op/s 2024-07-24T21:50:50.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 -- 172.21.15.60:0/2155636206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 msgr2=0x7f1ae010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:50.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 --2- 172.21.15.60:0/2155636206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae010c4b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009900 tx=0x7f1ad402a410 comp rx=0 tx=0).stop 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 -- 172.21.15.60:0/2155636206 shutdown_connections 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 --2- 172.21.15.60:0/2155636206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae010cb00 0x7f1ae010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 --2- 172.21.15.60:0/2155636206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae010c4b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 --2- 172.21.15.60:0/2155636206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae01036a0 0x7f1ae0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.413+0000 7f1ae661f640 1 -- 172.21.15.60:0/2155636206 >> 172.21.15.60:0/2155636206 conn(0x7f1ae00fd100 msgr2=0x7f1ae00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.414+0000 7f1ae661f640 1 -- 172.21.15.60:0/2155636206 shutdown_connections 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.414+0000 7f1ae661f640 1 -- 172.21.15.60:0/2155636206 wait complete. 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.414+0000 7f1ae661f640 1 Processor -- start 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.414+0000 7f1ae661f640 1 -- start start 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01036a0 0x7f1ae0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 0x7f1ae019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01127b0 con 0x7f1ae0103fc0 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0112630 con 0x7f1ae01036a0 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae661f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0112930 con 0x7f1ae010cb00 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43058/0 (socket says 172.21.15.60:43058) 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae4bbc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 0x7f1ae019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 -- 172.21.15.60:0/3512916436 learned_addr learned my addr 172.21.15.60:0/3512916436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adffff640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01036a0 0x7f1ae0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 -- 172.21.15.60:0/3512916436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 msgr2=0x7f1ae019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 0x7f1ae019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 -- 172.21.15.60:0/3512916436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01036a0 msgr2=0x7f1ae0198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01036a0 0x7f1ae0198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 -- 172.21.15.60:0/3512916436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae019edd0 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1ae4bbc640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 0x7f1ae019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.415+0000 7f1adf7fe640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4004270 tx=0x7f1ad40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.416+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad403f030 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.416+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad40044c0 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.416+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae019f060 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.416+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4035070 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.416+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a5780 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.417+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0103a80 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.418+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1ad4042030 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.419+0000 7f1add7fa640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 0x7f1ac80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.419+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1ad4104910 con 0x7f1ae0103fc0 2024-07-24T21:50:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.419+0000 7f1adffff640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 0x7f1ac80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.419+0000 7f1adffff640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 0x7f1ac80b6280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009990 tx=0x7f1ad0009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.422+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f1ad40cbb80 con 0x7f1ae0103fc0 2024-07-24T21:50:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.657+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae01a5a10 con 0x7f1ae0103fc0 2024-07-24T21:50:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.658+0000 7f1add7fa640 1 -- 172.21.15.60:0/3512916436 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11919 (secure 0 0 0) 0x7f1ad40cb480 con 0x7f1ae0103fc0 2024-07-24T21:50:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:50:50.660 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":31,"btime":"2024-07-24T21:50:23:599643+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":25963,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3836191659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3836191659},{"type":"v1","addr":"172.21.15.123:6835","nonce":3836191659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:50:17.948711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24556":{"gid":24556,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:50:50.660 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 31 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 msgr2=0x7f1ac80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 0x7f1ac80b6280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009990 tx=0x7f1ad0009110 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 msgr2=0x7f1ae0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4004270 tx=0x7f1ad40042a0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 shutdown_connections 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1ac80b3dc0 0x7f1ac80b6280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae010cb00 0x7f1ae019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ae0103fc0 0x7f1ae0199260 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 --2- 172.21.15.60:0/3512916436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae01036a0 0x7f1ae0198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 >> 172.21.15.60:0/3512916436 conn(0x7f1ae00fd100 msgr2=0x7f1ae010ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 shutdown_connections 2024-07-24T21:50:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:50.660+0000 7f1ae661f640 1 -- 172.21.15.60:0/3512916436 wait complete. 2024-07-24T21:50:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:51 smithi123 ceph-mon[37143]: pgmap v502: 97 pgs: 97 active+clean; 825 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.7 MiB/s wr, 2.31k op/s 2024-07-24T21:50:51.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:51 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3512916436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:51.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:51 smithi060 ceph-mon[33000]: pgmap v502: 97 pgs: 97 active+clean; 825 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.7 MiB/s wr, 2.31k op/s 2024-07-24T21:50:51.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:51 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3512916436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:51 smithi086 ceph-mon[37128]: pgmap v502: 97 pgs: 97 active+clean; 825 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.7 MiB/s wr, 2.31k op/s 2024-07-24T21:50:51.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:51 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3512916436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:50:51.914 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 62499c77 --release --await' 2024-07-24T21:50:51.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 62499c77 --release --await 2024-07-24T21:50:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:50:53.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:53 smithi123 ceph-mon[37143]: pgmap v503: 97 pgs: 97 active+clean; 853 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 5.3 MiB/s wr, 2.21k op/s 2024-07-24T21:50:53.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:53 smithi086 ceph-mon[37128]: pgmap v503: 97 pgs: 97 active+clean; 853 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 5.3 MiB/s wr, 2.21k op/s 2024-07-24T21:50:53.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:53 smithi060 ceph-mon[33000]: pgmap v503: 97 pgs: 97 active+clean; 853 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 5.3 MiB/s wr, 2.21k op/s 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 -- 172.21.15.60:0/4012839506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 msgr2=0x7fd48810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 --2- 172.21.15.60:0/4012839506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48810e8a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd478009900 tx=0x7fd47802a480 comp rx=0 tx=0).stop 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 -- 172.21.15.60:0/4012839506 shutdown_connections 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 --2- 172.21.15.60:0/4012839506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48810ede0 0x7fd4881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 --2- 172.21.15.60:0/4012839506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48810e8a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 --2- 172.21.15.60:0/4012839506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd488101230 0x7fd488101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 -- 172.21.15.60:0/4012839506 >> 172.21.15.60:0/4012839506 conn(0x7fd4880fd100 msgr2=0x7fd4880ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 -- 172.21.15.60:0/4012839506 shutdown_connections 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.931+0000 7fd48f560640 1 -- 172.21.15.60:0/4012839506 wait complete. 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 Processor -- start 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 -- start start 2024-07-24T21:50:53.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd488101230 0x7fd48819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48810ede0 0x7fd4881a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd488114920 con 0x7fd488101b50 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881147a0 con 0x7fd488101230 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48f560640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd488114aa0 con 0x7fd48810ede0 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43072/0 (socket says 172.21.15.60:43072) 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48cafb640 1 -- 172.21.15.60:0/578990511 learned_addr learned my addr 172.21.15.60:0/578990511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.932+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd488101230 0x7fd48819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48dafd640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48810ede0 0x7fd4881a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 -- 172.21.15.60:0/578990511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48810ede0 msgr2=0x7fd4881a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48810ede0 0x7fd4881a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 -- 172.21.15.60:0/578990511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd488101230 msgr2=0x7fd48819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd488101230 0x7fd48819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 -- 172.21.15.60:0/578990511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4881a0f20 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48cafb640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd478005940 tx=0x7fd4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd47e7fc640 1 -- 172.21.15.60:0/578990511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47803f030 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd47e7fc640 1 -- 172.21.15.60:0/578990511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478003cb0 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4881a11b0 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd47e7fc640 1 -- 172.21.15.60:0/578990511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478035070 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.933+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4881a78d0 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 Processor -- start 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd47e7fc640 1 -- 172.21.15.60:0/578990511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd478042030 con 0x7fd488101b50 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 -- start start 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 0x557d33681e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33682350 0x557d3368dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d335f8dd0 con 0x557d3368e2e0 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d335e82d0 con 0x557d33681a30 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd491911740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d335ec310 con 0x557d33682350 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd48dafd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 0x557d33681e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43076/0 (socket says 172.21.15.60:43076) 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd48cafb640 1 -- 172.21.15.60:0/198243168 learned_addr learned my addr 172.21.15.60:0/198243168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.936+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33682350 0x557d3368dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 -- 172.21.15.60:0/198243168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33682350 msgr2=0x557d3368dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33682350 0x557d3368dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 -- 172.21.15.60:0/198243168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 msgr2=0x557d33681e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 0x557d33681e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 -- 172.21.15.60:0/198243168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d33348b00 con 0x557d3368e2e0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd47e7fc640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 0x7fd4740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48dafd640 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 0x557d33681e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48dafd640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 0x7fd4740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd48cafb640 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd478101f90 tx=0x7fd4780c7b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3610f22bccd33f7a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd47e7fc640 1 -- 172.21.15.60:0/578990511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd4781048d0 con 0x7fd488101b50 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd4737fe640 1 -- 172.21.15.60:0/198243168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478049070 con 0x557d3368e2e0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd4737fe640 1 -- 172.21.15.60:0/198243168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4780ee070 con 0x557d3368e2e0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.937+0000 7fd4737fe640 1 -- 172.21.15.60:0/198243168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780d2e60 con 0x557d3368e2e0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- 172.21.15.60:0/198243168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 msgr2=0x557d336906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd48dafd640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 0x7fd4740b6200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480009740 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd478101f90 tx=0x7fd4780c7b30 comp rx=0 tx=0).stop 2024-07-24T21:50:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- 172.21.15.60:0/198243168 shutdown_connections 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336906d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33682350 0x557d3368dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 --2- 172.21.15.60:0/198243168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33681a30 0x557d33681e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- 172.21.15.60:0/198243168 >> 172.21.15.60:0/198243168 conn(0x557d3367f860 msgr2=0x557d33680a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- 172.21.15.60:0/198243168 shutdown_connections 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- 172.21.15.60:0/198243168 wait complete. 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.938+0000 7fd491911740 1 -- start start 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 Processor -- start 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d3368b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d3368bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d3344bbb0 con 0x557d33681a30 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d3358a900 con 0x557d33682350 2024-07-24T21:50:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d33690d20 con 0x557d3368e2e0 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd491911740 10 client.0 _ll_register_callbacks cb 0x557d33594c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd48d2fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd48d2fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43086/0 (socket says 172.21.15.60:43086) 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd48d2fc640 1 -- 172.21.15.60:0/2783079044 learned_addr learned my addr 172.21.15.60:0/2783079044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd48dafd640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d3368bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.939+0000 7fd48cafb640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d3368b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 -- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 msgr2=0x557d3368bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d3368bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 -- 172.21.15.60:0/2783079044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 msgr2=0x557d3368b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d3368b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 -- 172.21.15.60:0/2783079044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d333bf960 con 0x557d33681a30 2024-07-24T21:50:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd48400bad0 tx=0x7fd48400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a372eb630dae6877 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484003e20 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd484005020 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d335fad20 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd484005220 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd491911740 10 client.16620 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557d3369c830 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.940+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d335fad20 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd4840245a0 con 0x557d33681a30 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 10 client.16620 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 10 client.16620 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 20 client.16620 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3369dce0 0x557d336a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 4 client.16620 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:50:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d335f67c0 con 0x557d3369dce0 2024-07-24T21:50:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd48cafb640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3369dce0 0x557d336a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd48cafb640 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3369dce0 0x557d336a0180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6248fee3438983ad server_cookie=3462243d5ceaab4f in_seq=0 out_seq=0 2024-07-24T21:50:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.941+0000 7fd470ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd470ff9640 1 -- 172.21.15.60:0/2783079044 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d335f67c0 con 0x557d3369dce0 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd470ff9640 10 client.16620 handle_command_reply: tid=0 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 client.16620 shutdown 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 msgr2=0x557d3368b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd48400bad0 tx=0x7fd48400bed0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 shutdown_connections 2024-07-24T21:50:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3369dce0 0x557d336a0180 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d3368bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d3368b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 --2- 172.21.15.60:0/2783079044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33681a30 0x557d3368b2a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 >> 172.21.15.60:0/2783079044 conn(0x557d3367f860 msgr2=0x557d33685400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.942+0000 7fd470ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.943+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 shutdown_connections 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.943+0000 7fd491911740 1 -- 172.21.15.60:0/2783079044 wait complete. 2024-07-24T21:50:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.943+0000 7fd491911740 20 client.16620 trim_cache size 0 max 16384 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 Processor -- start 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 -- start start 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d33685b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33686be0 0x557d336a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b2c70 con 0x557d3368e2e0 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b04e0 con 0x557d33682350 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd491911740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881ae330 con 0x557d33686be0 2024-07-24T21:50:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43102/0 (socket says 172.21.15.60:43102) 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 -- 172.21.15.60:0/959125903 learned_addr learned my addr 172.21.15.60:0/959125903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d33685b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48dafd640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33686be0 0x557d336a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 -- 172.21.15.60:0/959125903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33686be0 msgr2=0x557d336a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33686be0 0x557d336a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 -- 172.21.15.60:0/959125903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 msgr2=0x557d33685b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d33685b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 -- 172.21.15.60:0/959125903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3368a200 con 0x557d3368e2e0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d33685b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.961+0000 7fd48cafb640 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd478003fd0 tx=0x7fd478003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afccd772a37e861 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd472ffd640 1 -- 172.21.15.60:0/959125903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478049040 con 0x557d3368e2e0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd472ffd640 1 -- 172.21.15.60:0/959125903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47810a400 con 0x557d3368e2e0 2024-07-24T21:50:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd472ffd640 1 -- 172.21.15.60:0/959125903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780d2e60 con 0x557d3368e2e0 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 -- 172.21.15.60:0/959125903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 msgr2=0x557d336a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd478003fd0 tx=0x7fd478003e60 comp rx=0 tx=0).stop 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 -- 172.21.15.60:0/959125903 shutdown_connections 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d33686be0 0x557d336a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3368e2e0 0x557d336a5a50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 --2- 172.21.15.60:0/959125903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33682350 0x557d33685b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 -- 172.21.15.60:0/959125903 >> 172.21.15.60:0/959125903 conn(0x557d3367f860 msgr2=0x557d336864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 -- 172.21.15.60:0/959125903 shutdown_connections 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.962+0000 7fd491911740 1 -- 172.21.15.60:0/959125903 wait complete. 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 -- start start 2024-07-24T21:50:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 Processor -- start 2024-07-24T21:50:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 0x557d33692a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d33707a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881ae9a0 con 0x557d33682350 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881afe10 con 0x557d33686be0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881af070 con 0x557d3368e2e0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd491911740 10 client.0 _ll_register_callbacks cb 0x7fd484004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd48d2fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd48cafb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 0x557d33692a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd48d2fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43104/0 (socket says 172.21.15.60:43104) 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd48d2fc640 1 -- 172.21.15.60:0/919735586 learned_addr learned my addr 172.21.15.60:0/919735586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.963+0000 7fd48dafd640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d33707a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 -- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 msgr2=0x557d33707a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d33707a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 -- 172.21.15.60:0/919735586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 msgr2=0x557d336924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 -- 172.21.15.60:0/919735586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d33682180 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd48cafb640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 0x557d33692a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b7397ce762f6bb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478049050 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd4780c7770 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d335f3f00 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4780d2e60 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd491911740 10 client.26096 fetch_fsmap learned FSMap version 31 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.964+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557d337091e0 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d335f3f00 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd47810acd0 con 0x557d33686be0 2024-07-24T21:50:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 10 client.26096 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 10 client.26096 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 20 client.26096 populate_metadata read hostname 'smithi060' 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d337095d0 0x557d3370b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 4 client.26096 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "62499c77", "release": true, "await": true}] 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "62499c77", "release": true, "await": true}) -- 0x557d3356bec0 con 0x557d337095d0 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.965+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d337095d0 0x557d3370b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.966+0000 7fd48d2fc640 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d337095d0 0x557d3370b9c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d264ddb0ee9d1077 server_cookie=17ccfd3546d3ec08 in_seq=0 out_seq=0 2024-07-24T21:50:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:53.966+0000 7fd470ff9640 10 client.26096 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd470ff9640 1 -- 172.21.15.60:0/919735586 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557d3356bec0 con 0x557d337095d0 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd470ff9640 10 client.26096 handle_command_reply: tid=0 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 client.26096 shutdown 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 msgr2=0x557d33692a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 0x557d33692a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478032f60 comp rx=0 tx=0).stop 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 shutdown_connections 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d337095d0 0x557d3370b9c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3368e2e0 0x557d33707a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d33686be0 0x557d33692a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 --2- 172.21.15.60:0/919735586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d33682350 0x557d336924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd470ff9640 0 client.26096 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 >> 172.21.15.60:0/919735586 conn(0x557d3367f860 msgr2=0x557d3367e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 shutdown_connections 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 1 -- 172.21.15.60:0/919735586 wait complete. 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.614+0000 7fd491911740 20 client.26096 trim_cache size 0 max 16384 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 30, 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:50:54.617 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 16, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "62499c77": { 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 16, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:50:54.618 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:50:54.619 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 msgr2=0x7fd4740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 0x7fd4740b6200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480009740 comp rx=0 tx=0).stop 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 msgr2=0x7fd48819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd478005940 tx=0x7fd4780042f0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 shutdown_connections 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4740b3d40 0x7fd4740b6200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48810ede0 0x7fd4881a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd488101b50 0x7fd48819b3b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 --2- 172.21.15.60:0/578990511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd488101230 0x7fd48819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:50:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 >> 172.21.15.60:0/578990511 conn(0x7fd4880fd100 msgr2=0x7fd4880ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:50:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 shutdown_connections 2024-07-24T21:50:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:50:54.615+0000 7fd48f560640 1 -- 172.21.15.60:0/578990511 wait complete. 2024-07-24T21:50:54.708 INFO:tasks.workunit.client.0.smithi060.stdout: 218 126340 0 83329 4 33039 4 2024-07-24T21:50:55.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:55 smithi123 ceph-mon[37143]: pgmap v504: 97 pgs: 97 active+clean; 863 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.8 MiB/s wr, 2.00k op/s 2024-07-24T21:50:55.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:55 smithi060 ceph-mon[33000]: pgmap v504: 97 pgs: 97 active+clean; 863 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.8 MiB/s wr, 2.00k op/s 2024-07-24T21:50:55.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:55 smithi086 ceph-mon[37128]: pgmap v504: 97 pgs: 97 active+clean; 863 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.8 MiB/s wr, 2.00k op/s 2024-07-24T21:50:55.567 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '62499c77', total seconds elapsed: 37.3 2024-07-24T21:50:56.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:56 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:50:56.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:56 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:56 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:50:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:56 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:56 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:50:56.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:56.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:56 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:50:57.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:57 smithi086 ceph-mon[37128]: pgmap v505: 97 pgs: 97 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 1.99k op/s 2024-07-24T21:50:57.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:57 smithi060 ceph-mon[33000]: pgmap v505: 97 pgs: 97 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 1.99k op/s 2024-07-24T21:50:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:57 smithi123 ceph-mon[37143]: pgmap v505: 97 pgs: 97 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 1.99k op/s 2024-07-24T21:50:59.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:50:59 smithi086 ceph-mon[37128]: pgmap v506: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 1.33k op/s 2024-07-24T21:50:59.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:50:59 smithi060 ceph-mon[33000]: pgmap v506: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 1.33k op/s 2024-07-24T21:50:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:50:59 smithi123 ceph-mon[37143]: pgmap v506: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.2 MiB/s wr, 1.33k op/s 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/85025404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 msgr2=0x7f6ae411de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/85025404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae411de80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009900 tx=0x7f6ae002a420 comp rx=0 tx=0).stop 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/85025404 shutdown_connections 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/85025404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae411de80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/85025404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae4109f20 0x7f6ae410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/85025404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/85025404 >> 172.21.15.60:0/85025404 conn(0x7f6ae406d5a0 msgr2=0x7f6ae406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.130+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/85025404 shutdown_connections 2024-07-24T21:51:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.131+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/85025404 wait complete. 2024-07-24T21:51:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 Processor -- start 2024-07-24T21:51:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 -- start start 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4118f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae4109f20 0x7f6ae4119450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4121350 con 0x7f6ae4109b40 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41211d0 con 0x7f6ae411ba90 2024-07-24T21:51:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6aeb4f0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41214d0 con 0x7f6ae4109f20 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6ae9a8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6ae9a8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46042/0 (socket says 172.21.15.60:46042) 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2589238994 learned_addr learned my addr 172.21.15.60:0/2589238994 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.132+0000 7f6ae928c640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4118f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2589238994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae4109f20 msgr2=0x7f6ae4119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae4109f20 0x7f6ae4119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 msgr2=0x7f6ae4118f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4118f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2589238994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae4112bb0 con 0x7f6ae411ba90 2024-07-24T21:51:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae928c640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4118f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.133+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ae002a990 tx=0x7f6ae0003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.140+0000 7f6ada7fc640 1 -- 172.21.15.60:0/2589238994 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae003f070 con 0x7f6ae411ba90 2024-07-24T21:51:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.140+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae4112e40 con 0x7f6ae411ba90 2024-07-24T21:51:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.141+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae4117da0 con 0x7f6ae411ba90 2024-07-24T21:51:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.152+0000 7f6aee249740 1 Processor -- start 2024-07-24T21:51:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.152+0000 7f6aee249740 1 -- start start 2024-07-24T21:51:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.152+0000 7f6aee249740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 0x55aa29d46e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6aee249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6aee249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d55740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6aee249740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa29cbc280 con 0x55aa29d46aa0 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6aee249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa29cab780 con 0x55aa29d53350 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6aee249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa29caf7c0 con 0x55aa29d473c0 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39292/0 (socket says 172.21.15.60:39292) 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ae928c640 1 -- 172.21.15.60:0/3456031452 learned_addr learned my addr 172.21.15.60:0/3456031452 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ada7fc640 1 -- 172.21.15.60:0/2589238994 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae0031b50 con 0x7f6ae411ba90 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 0x55aa29d46e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ada7fc640 1 -- 172.21.15.60:0/2589238994 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0035030 con 0x7f6ae411ba90 2024-07-24T21:51:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.153+0000 7f6ada7fc640 1 -- 172.21.15.60:0/2589238994 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6ae0042070 con 0x7f6ae411ba90 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.154+0000 7f6ae928c640 1 -- 172.21.15.60:0/3456031452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 msgr2=0x55aa29d55740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.154+0000 7f6ae928c640 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d55740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.154+0000 7f6ae928c640 1 -- 172.21.15.60:0/3456031452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 msgr2=0x55aa29d46e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.154+0000 7f6ae928c640 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 0x55aa29d46e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.154+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 0x55aa29d46e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.155+0000 7f6ae928c640 1 -- 172.21.15.60:0/3456031452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa29a0f790 con 0x55aa29d473c0 2024-07-24T21:51:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.155+0000 7f6ae928c640 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400bd40 tx=0x7f6ad400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a6d43ec2d8ffc7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.157+0000 7f6ada7fc640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 0x7f6ad00b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.158+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 0x7f6ad00b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.158+0000 7f6ada7fc640 1 -- 172.21.15.60:0/2589238994 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6ae01057a0 con 0x7f6ae411ba90 2024-07-24T21:51:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.160+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 0x7f6ad00b6440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ae411e230 tx=0x7f6adc0095a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.172+0000 7f6acf7fe640 1 -- 172.21.15.60:0/3456031452 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4004220 con 0x55aa29d473c0 2024-07-24T21:51:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.173+0000 7f6acf7fe640 1 -- 172.21.15.60:0/3456031452 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad400be30 con 0x55aa29d473c0 2024-07-24T21:51:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.180+0000 7f6aee249740 1 -- 172.21.15.60:0/3456031452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 msgr2=0x55aa29d52e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.180+0000 7f6aee249740 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400bd40 tx=0x7f6ad400f730 comp rx=0 tx=0).stop 2024-07-24T21:51:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.186+0000 7f6aee249740 1 -- 172.21.15.60:0/3456031452 shutdown_connections 2024-07-24T21:51:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.186+0000 7f6aee249740 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d55740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.186+0000 7f6aee249740 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d473c0 0x55aa29d52e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.186+0000 7f6aee249740 1 --2- 172.21.15.60:0/3456031452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d46aa0 0x55aa29d46e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.186+0000 7f6aee249740 1 -- 172.21.15.60:0/3456031452 >> 172.21.15.60:0/3456031452 conn(0x55aa29d448d0 msgr2=0x55aa29d45a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.197+0000 7f6aee249740 1 -- 172.21.15.60:0/3456031452 shutdown_connections 2024-07-24T21:51:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.197+0000 7f6aee249740 1 -- 172.21.15.60:0/3456031452 wait complete. 2024-07-24T21:51:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.197+0000 7f6aee249740 1 -- start start 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.197+0000 7f6aee249740 1 Processor -- start 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55aa29be5ab0 con 0x55aa29d473c0 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa29c339e0 con 0x55aa29d46aa0 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa29d56080 con 0x55aa29d53350 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.198+0000 7f6aee249740 10 client.0 _ll_register_callbacks cb 0x55aa299bba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.199+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d52950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.199+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d52950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46054/0 (socket says 172.21.15.60:46054) 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.199+0000 7f6ae928c640 1 -- 172.21.15.60:0/1059367010 learned_addr learned my addr 172.21.15.60:0/1059367010 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.199+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.199+0000 7f6ae8a8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 msgr2=0x55aa29d50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1059367010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 msgr2=0x55aa29d52950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d52950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1059367010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa29a88620 con 0x55aa29d473c0 2024-07-24T21:51:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.200+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4fe30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4074740 tx=0x7f6adc004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e4433142617ed server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.236+0000 7f6accff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.236+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc027060 con 0x55aa29d473c0 2024-07-24T21:51:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.236+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa29c5ace0 con 0x55aa29d473c0 2024-07-24T21:51:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.236+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6adc012030 con 0x55aa29d473c0 2024-07-24T21:51:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.237+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6adc016460 con 0x55aa29d473c0 2024-07-24T21:51:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.237+0000 7f6aee249740 10 client.16638 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.237+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aa29d61830 con 0x55aa29d473c0 2024-07-24T21:51:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.239+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa29c5ace0 con 0x55aa29d473c0 2024-07-24T21:51:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.240+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6adc02a830 con 0x55aa29d473c0 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.240+0000 7f6aee249740 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.240+0000 7f6aee249740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.241+0000 7f6aee249740 20 client.16638 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.241+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d63160 0x55aa29d65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.241+0000 7f6aee249740 4 client.16638 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.241+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa29cbe1d0 con 0x55aa29d63160 2024-07-24T21:51:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.241+0000 7f6ae928c640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d63160 0x55aa29d65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.242+0000 7f6ae928c640 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d63160 0x55aa29d65620 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1cf03caf4cb0f52 server_cookie=fa78cfdcb94dfff1 in_seq=0 out_seq=0 2024-07-24T21:51:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.242+0000 7f6accff9640 10 client.16638 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.245+0000 7f6accff9640 1 -- 172.21.15.60:0/1059367010 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa29cbe1d0 con 0x55aa29d63160 2024-07-24T21:51:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.245+0000 7f6accff9640 10 client.16638 handle_command_reply: tid=0 2024-07-24T21:51:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.245+0000 7f6aee249740 1 client.16638 shutdown 2024-07-24T21:51:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.245+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 msgr2=0x55aa29d4fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4fe30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4074740 tx=0x7f6adc004be0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 shutdown_connections 2024-07-24T21:51:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d63160 0x55aa29d65620 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4fe30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 --2- 172.21.15.60:0/1059367010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d52950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.246+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 >> 172.21.15.60:0/1059367010 conn(0x55aa29d448d0 msgr2=0x55aa29d4a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.250+0000 7f6accff9640 0 client.16638 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.250+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 shutdown_connections 2024-07-24T21:51:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.250+0000 7f6aee249740 1 -- 172.21.15.60:0/1059367010 wait complete. 2024-07-24T21:51:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.250+0000 7f6aee249740 20 client.16638 trim_cache size 0 max 16384 2024-07-24T21:51:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.340+0000 7f6aee249740 1 Processor -- start 2024-07-24T21:51:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.340+0000 7f6aee249740 1 -- start start 2024-07-24T21:51:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.343+0000 7f6aee249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d6aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6aee249740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d6b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6aee249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d4ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d6aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae928c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d6aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46070/0 (socket says 172.21.15.60:46070) 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac8011120 con 0x55aa29d473c0 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae928c640 1 -- 172.21.15.60:0/2390989159 learned_addr learned my addr 172.21.15.60:0/2390989159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac800e990 con 0x55aa29d46aa0 2024-07-24T21:51:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac800c890 con 0x55aa29d53350 2024-07-24T21:51:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d6b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d4ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2390989159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 msgr2=0x55aa29d6aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d6aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2390989159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 msgr2=0x55aa29d6b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d6b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.344+0000 7f6ae9a8d640 1 -- 172.21.15.60:0/2390989159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa29d477a0 con 0x55aa29d53350 2024-07-24T21:51:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.345+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.345+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d4ad50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0102f10 tx=0x7f6ae00cbee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87856ab6f22ed474 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.348+0000 7f6aceffd640 1 -- 172.21.15.60:0/2390989159 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00d09d0 con 0x55aa29d53350 2024-07-24T21:51:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.348+0000 7f6aceffd640 1 -- 172.21.15.60:0/2390989159 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae00d5480 con 0x55aa29d53350 2024-07-24T21:51:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.350+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 msgr2=0x55aa29d4ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.350+0000 7f6aee249740 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d4ad50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0102f10 tx=0x7f6ae00cbee0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 shutdown_connections 2024-07-24T21:51:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d53350 0x55aa29d4ad50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 --2- 172.21.15.60:0/2390989159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d46aa0 0x55aa29d6aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 >> 172.21.15.60:0/2390989159 conn(0x55aa29d448d0 msgr2=0x55aa29d4e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 shutdown_connections 2024-07-24T21:51:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.359+0000 7f6aee249740 1 -- 172.21.15.60:0/2390989159 wait complete. 2024-07-24T21:51:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.360+0000 7f6aee249740 1 -- start start 2024-07-24T21:51:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.360+0000 7f6aee249740 1 Processor -- start 2024-07-24T21:51:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d46aa0 0x55aa29d4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d4c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac800ce50 con 0x55aa29d473c0 2024-07-24T21:51:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac800e2c0 con 0x55aa29d53350 2024-07-24T21:51:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac800d520 con 0x55aa29d46aa0 2024-07-24T21:51:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.361+0000 7f6aee249740 10 client.0 _ll_register_callbacks cb 0x7f6adc009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.362+0000 7f6ae8a8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.362+0000 7f6ae8a8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60344/0 (socket says 172.21.15.60:60344) 2024-07-24T21:51:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.362+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1206433285 learned_addr learned my addr 172.21.15.60:0/1206433285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d4c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d46aa0 msgr2=0x55aa29d4d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d46aa0 0x55aa29d4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1206433285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 msgr2=0x55aa29d4c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d4c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae8a8b640 1 -- 172.21.15.60:0/1206433285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa29d4cdb0 con 0x55aa29d473c0 2024-07-24T21:51:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.363+0000 7f6ae9a8d640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d4c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:51:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.364+0000 7f6ae8a8b640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000f80 tx=0x7f6adc02aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad5487925bddf45 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.378+0000 7f6accff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.379+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc039030 con 0x55aa29d473c0 2024-07-24T21:51:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.379+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f6adc01d040 con 0x55aa29d473c0 2024-07-24T21:51:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.379+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae00d5480 con 0x55aa29d473c0 2024-07-24T21:51:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.380+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6adc043050 con 0x55aa29d473c0 2024-07-24T21:51:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.380+0000 7f6aee249740 10 client.16644 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.380+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aa29d5b470 con 0x55aa29d473c0 2024-07-24T21:51:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae00d5480 con 0x55aa29d473c0 2024-07-24T21:51:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6adc020530 con 0x55aa29d473c0 2024-07-24T21:51:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 10 client.16644 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 20 client.16644 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d5bc80 0x55aa29dcfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 4 client.16644 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.381+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa29cb9c70 con 0x55aa29d5bc80 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.382+0000 7f6ae928c640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d5bc80 0x55aa29dcfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.383+0000 7f6ae928c640 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d5bc80 0x55aa29dcfed0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64656e20ece81ab server_cookie=7c0724cb81e7937 in_seq=0 out_seq=0 2024-07-24T21:51:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:00.383+0000 7f6accff9640 10 client.16644 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.160+0000 7f6accff9640 1 -- 172.21.15.60:0/1206433285 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa29cb9c70 con 0x55aa29d5bc80 2024-07-24T21:51:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.160+0000 7f6accff9640 10 client.16644 handle_command_reply: tid=0 2024-07-24T21:51:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.160+0000 7f6aee249740 1 client.16644 shutdown 2024-07-24T21:51:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 msgr2=0x55aa29d4c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000f80 tx=0x7f6adc02aae0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 shutdown_connections 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55aa29d5bc80 0x55aa29dcfed0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa29d53350 0x55aa29d4c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa29d473c0 0x55aa29d4c060 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 --2- 172.21.15.60:0/1206433285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29d46aa0 0x55aa29d4d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 >> 172.21.15.60:0/1206433285 conn(0x55aa29d448d0 msgr2=0x55aa29d4ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6accff9640 0 client.16644 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 shutdown_connections 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 1 -- 172.21.15.60:0/1206433285 wait complete. 2024-07-24T21:51:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.161+0000 7f6aee249740 20 client.16644 trim_cache size 0 max 16384 2024-07-24T21:51:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:51:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:51:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:51:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:51:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 msgr2=0x7f6ad00b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 0x7f6ad00b6440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ae411e230 tx=0x7f6adc0095a0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 msgr2=0x7f6ae41122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ae002a990 tx=0x7f6ae0003cd0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 shutdown_connections 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6ad00b3f80 0x7f6ad00b6440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ae411ba90 0x7f6ae41122f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae4109f20 0x7f6ae4119450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 --2- 172.21.15.60:0/2589238994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae4109b40 0x7f6ae4118f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 >> 172.21.15.60:0/2589238994 conn(0x7f6ae406d5a0 msgr2=0x7f6ae411bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 shutdown_connections 2024-07-24T21:51:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:01.162+0000 7f6aeb4f0640 1 -- 172.21.15.60:0/2589238994 wait complete. 2024-07-24T21:51:01.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:01 smithi086 ceph-mon[37128]: pgmap v507: 97 pgs: 97 active+clean; 898 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 724 op/s 2024-07-24T21:51:01.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:01 smithi060 ceph-mon[33000]: pgmap v507: 97 pgs: 97 active+clean; 898 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 724 op/s 2024-07-24T21:51:01.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:01 smithi123 ceph-mon[37143]: pgmap v507: 97 pgs: 97 active+clean; 898 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 724 op/s 2024-07-24T21:51:03.544 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:03 smithi123 ceph-mon[37143]: pgmap v508: 97 pgs: 97 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-07-24T21:51:03.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:03 smithi086 ceph-mon[37128]: pgmap v508: 97 pgs: 97 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-07-24T21:51:03.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:03 smithi060 ceph-mon[33000]: pgmap v508: 97 pgs: 97 active+clean; 897 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-07-24T21:51:04.708 INFO:tasks.workunit.client.0.smithi060.stdout: 244 1387030 1669 936865 1586 364022 804 2024-07-24T21:51:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:05 smithi086 ceph-mon[37128]: pgmap v509: 97 pgs: 97 active+clean; 902 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 591 op/s 2024-07-24T21:51:05.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:05 smithi060 ceph-mon[33000]: pgmap v509: 97 pgs: 97 active+clean; 902 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 591 op/s 2024-07-24T21:51:05.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:05 smithi123 ceph-mon[37143]: pgmap v509: 97 pgs: 97 active+clean; 902 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 591 op/s 2024-07-24T21:51:07.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:07 smithi086 ceph-mon[37128]: pgmap v510: 97 pgs: 97 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 882 op/s 2024-07-24T21:51:07.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:07 smithi060 ceph-mon[33000]: pgmap v510: 97 pgs: 97 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 882 op/s 2024-07-24T21:51:07.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:07 smithi123 ceph-mon[37143]: pgmap v510: 97 pgs: 97 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 882 op/s 2024-07-24T21:51:09.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:09 smithi086 ceph-mon[37128]: pgmap v511: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 821 op/s 2024-07-24T21:51:09.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:09 smithi060 ceph-mon[33000]: pgmap v511: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 821 op/s 2024-07-24T21:51:09.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:09 smithi123 ceph-mon[37143]: pgmap v511: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 821 op/s 2024-07-24T21:51:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:11 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:51:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:11 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:51:11.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:11 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:51:11.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:11 smithi086 ceph-mon[37128]: pgmap v512: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-07-24T21:51:11.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:11 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:51:11.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:11 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:51:11.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:11 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:51:11.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:11 smithi060 ceph-mon[33000]: pgmap v512: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-07-24T21:51:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:11 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:51:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:11 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:51:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:11 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:51:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:11 smithi123 ceph-mon[37143]: pgmap v512: 97 pgs: 97 active+clean; 909 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-07-24T21:51:13.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:13 smithi086 ceph-mon[37128]: pgmap v513: 97 pgs: 97 active+clean; 943 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 897 op/s 2024-07-24T21:51:13.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:13 smithi060 ceph-mon[33000]: pgmap v513: 97 pgs: 97 active+clean; 943 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 897 op/s 2024-07-24T21:51:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:13 smithi123 ceph-mon[37143]: pgmap v513: 97 pgs: 97 active+clean; 943 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 897 op/s 2024-07-24T21:51:14.709 INFO:tasks.workunit.client.0.smithi060.stdout: 281 1903660 1912 1301277 2049 443929 628 2024-07-24T21:51:15.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:15 smithi086 ceph-mon[37128]: pgmap v514: 97 pgs: 97 active+clean; 950 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 661 op/s 2024-07-24T21:51:15.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:15 smithi060 ceph-mon[33000]: pgmap v514: 97 pgs: 97 active+clean; 950 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 661 op/s 2024-07-24T21:51:15.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:15 smithi123 ceph-mon[37143]: pgmap v514: 97 pgs: 97 active+clean; 950 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 661 op/s 2024-07-24T21:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.328+0000 7f4a9a7fc640 1 -- 172.21.15.60:0/1740033207 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94032b00 con 0x7f4a9c109db0 2024-07-24T21:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/1740033207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c109db0 msgr2=0x7f4a9c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 --2- 172.21.15.60:0/1740033207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c109db0 0x7f4a9c10a190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005fd0 tx=0x7f4a9402b410 comp rx=0 tx=0).stop 2024-07-24T21:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/1740033207 shutdown_connections 2024-07-24T21:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 --2- 172.21.15.60:0/1740033207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c113270 0x7f4a9c1156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 --2- 172.21.15.60:0/1740033207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c10a6d0 0x7f4a9c112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 --2- 172.21.15.60:0/1740033207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c109db0 0x7f4a9c10a190 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005fd0 tx=0x7f4a9402b410 comp rx=0 tx=0).stop 2024-07-24T21:51:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.330+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/1740033207 >> 172.21.15.60:0/1740033207 conn(0x7f4a9c06d5a0 msgr2=0x7f4a9c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.334+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/1740033207 shutdown_connections 2024-07-24T21:51:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.334+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/1740033207 wait complete. 2024-07-24T21:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.335+0000 7f4aa1d4b640 1 Processor -- start 2024-07-24T21:51:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.340+0000 7f4aa1d4b640 1 -- start start 2024-07-24T21:51:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c10a6d0 0x7f4a9c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 0x7f4a9c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 0x7f4a9c1a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c118be0 con 0x7f4a9c10a6d0 2024-07-24T21:51:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c118a60 con 0x7f4a9c113270 2024-07-24T21:51:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4aa1d4b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c118d60 con 0x7f4a9c1a6e90 2024-07-24T21:51:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 0x7f4a9c1a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 0x7f4a9c1a7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46854/0 (socket says 172.21.15.60:46854) 2024-07-24T21:51:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4a9bfff640 1 -- 172.21.15.60:0/3685199396 learned_addr learned my addr 172.21.15.60:0/3685199396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.341+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 0x7f4a9c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.342+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c10a6d0 0x7f4a9c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.344+0000 7f4a9affd640 1 -- 172.21.15.60:0/3685199396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 msgr2=0x7f4a9c1a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.344+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 0x7f4a9c1a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.344+0000 7f4a9affd640 1 -- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c10a6d0 msgr2=0x7f4a9c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.344+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c10a6d0 0x7f4a9c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.344+0000 7f4a9affd640 1 -- 172.21.15.60:0/3685199396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c1a7a70 con 0x7f4a9c113270 2024-07-24T21:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.346+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 0x7f4a9c1a1b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009fd0 tx=0x7f4a8c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.379+0000 7f4a98ff9640 1 -- 172.21.15.60:0/3685199396 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c015020 con 0x7f4a9c113270 2024-07-24T21:51:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.379+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/3685199396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1ae010 con 0x7f4a9c113270 2024-07-24T21:51:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.379+0000 7f4aa1d4b640 1 -- 172.21.15.60:0/3685199396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1ae4a0 con 0x7f4a9c113270 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.383+0000 7f4aa4b7b740 1 Processor -- start 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 -- start start 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42c3e0 0x56260f43b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4a98ff9640 1 -- 172.21.15.60:0/3685199396 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8c011ce0 con 0x7f4a9c113270 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f43ba80 0x56260f43bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56260f3a3280 con 0x56260f42c3e0 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56260f392780 con 0x56260f43ba80 2024-07-24T21:51:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56260f3967c0 con 0x56260f42b9e0 2024-07-24T21:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4a98ff9640 1 -- 172.21.15.60:0/3685199396 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c010e00 con 0x7f4a9c113270 2024-07-24T21:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.384+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46860/0 (socket says 172.21.15.60:46860) 2024-07-24T21:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.385+0000 7f4a9bfff640 1 -- 172.21.15.60:0/3038185536 learned_addr learned my addr 172.21.15.60:0/3038185536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.385+0000 7f4a9bfff640 1 -- 172.21.15.60:0/3038185536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f43ba80 msgr2=0x56260f43bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.385+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f43ba80 0x56260f43bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.385+0000 7f4a9bfff640 1 -- 172.21.15.60:0/3038185536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42c3e0 msgr2=0x56260f43b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.385+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42c3e0 0x56260f43b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.386+0000 7f4a9bfff640 1 -- 172.21.15.60:0/3038185536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260f0f6790 con 0x56260f42b9e0 2024-07-24T21:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.387+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009aa0 tx=0x7f4a90009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f45e08c6c1b81e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.414+0000 7f4a897fa640 1 -- 172.21.15.60:0/3038185536 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90017070 con 0x56260f42b9e0 2024-07-24T21:51:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.414+0000 7f4a897fa640 1 -- 172.21.15.60:0/3038185536 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90013470 con 0x56260f42b9e0 2024-07-24T21:51:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.415+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3038185536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 msgr2=0x56260f42bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.415+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009aa0 tx=0x7f4a90009f60 comp rx=0 tx=0).stop 2024-07-24T21:51:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.420+0000 7f4a98ff9640 1 -- 172.21.15.60:0/3685199396 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4a8c018020 con 0x7f4a9c113270 2024-07-24T21:51:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.421+0000 7f4a98ff9640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 0x7f4a840b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.422+0000 7f4a98ff9640 1 -- 172.21.15.60:0/3685199396 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4a8c0e3360 con 0x7f4a9c113270 2024-07-24T21:51:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.424+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 0x7f4a840b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.429+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 0x7f4a840b6410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009830 tx=0x7f4a90009400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.460+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3038185536 shutdown_connections 2024-07-24T21:51:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.460+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f43ba80 0x56260f43bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.460+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42c3e0 0x56260f43b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.460+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3038185536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42b9e0 0x56260f42bdc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.460+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3038185536 >> 172.21.15.60:0/3038185536 conn(0x56260f42e140 msgr2=0x56260f42e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.465+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3038185536 shutdown_connections 2024-07-24T21:51:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.465+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3038185536 wait complete. 2024-07-24T21:51:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.466+0000 7f4aa4b7b740 1 -- start start 2024-07-24T21:51:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.466+0000 7f4aa4b7b740 1 Processor -- start 2024-07-24T21:51:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f442470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f4429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56260f2ccab0 con 0x56260f43ba80 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56260f31a9e0 con 0x56260f42b9e0 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56260f445d10 con 0x56260f42c3e0 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.467+0000 7f4aa4b7b740 10 client.0 _ll_register_callbacks cb 0x56260f0a2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42240/0 (socket says 172.21.15.60:42240) 2024-07-24T21:51:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/3806902564 learned_addr learned my addr 172.21.15.60:0/3806902564 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f4429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 msgr2=0x56260f442470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:51:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f442470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/3806902564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 msgr2=0x56260f4429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f4429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.468+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/3806902564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260f16f620 con 0x56260f42b9e0 2024-07-24T21:51:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.472+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400a040 tx=0x7f4a94007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc75b6dbd69cd1cf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.504+0000 7f4a7effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:51:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.504+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94042020 con 0x56260f42b9e0 2024-07-24T21:51:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.504+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56260f341ce0 con 0x56260f42b9e0 2024-07-24T21:51:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.504+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4a94032740 con 0x56260f42b9e0 2024-07-24T21:51:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.505+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a9404d050 con 0x56260f42b9e0 2024-07-24T21:51:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.505+0000 7f4aa4b7b740 10 client.26117 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.505+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56260f451420 con 0x56260f42b9e0 2024-07-24T21:51:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.516+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56260f341ce0 con 0x56260f42b9e0 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.516+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4a94004270 con 0x56260f42b9e0 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.516+0000 7f4aa4b7b740 10 client.26117 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.516+0000 7f4aa4b7b740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.517+0000 7f4aa4b7b740 20 client.26117 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.517+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f452e30 0x56260f4552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.517+0000 7f4aa4b7b740 4 client.26117 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.517+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260f3a51d0 con 0x56260f452e30 2024-07-24T21:51:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.517+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f452e30 0x56260f4552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.518+0000 7f4a7effd640 10 client.26117 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.518+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f452e30 0x56260f4552d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9def0f107bd239d0 server_cookie=39d314983a8ebc32 in_seq=0 out_seq=0 2024-07-24T21:51:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.519+0000 7f4a7effd640 1 -- 172.21.15.60:0/3806902564 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56260f3a51d0 con 0x56260f452e30 2024-07-24T21:51:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.519+0000 7f4a7effd640 10 client.26117 handle_command_reply: tid=0 2024-07-24T21:51:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 client.26117 shutdown 2024-07-24T21:51:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 msgr2=0x56260f444ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400a040 tx=0x7f4a94007c80 comp rx=0 tx=0).stop 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 shutdown_connections 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f452e30 0x56260f4552d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f4429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f442470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3806902564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f444ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 >> 172.21.15.60:0/3806902564 conn(0x56260f42e140 msgr2=0x56260f43c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.520+0000 7f4a7effd640 0 client.26117 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.522+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 shutdown_connections 2024-07-24T21:51:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.522+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3806902564 wait complete. 2024-07-24T21:51:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.523+0000 7f4aa4b7b740 20 client.26117 trim_cache size 0 max 16384 2024-07-24T21:51:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.778+0000 7f4aa4b7b740 1 Processor -- start 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.778+0000 7f4aa4b7b740 1 -- start start 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f441210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f43d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42250/0 (socket says 172.21.15.60:42250) 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f441210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.779+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f441210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46888/0 (socket says 172.21.15.60:46888) 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f43d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f43d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35650/0 (socket says 172.21.15.60:35650) 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/2415188021 learned_addr learned my addr 172.21.15.60:0/2415188021 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80011280 con 0x56260f43ba80 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8000eaf0 con 0x56260f42b9e0 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8000c940 con 0x56260f42c3e0 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/2415188021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 msgr2=0x56260f441210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f441210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/2415188021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 msgr2=0x56260f43d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f43d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.780+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/2415188021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260f440950 con 0x56260f42b9e0 2024-07-24T21:51:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.782+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007780 tx=0x7f4a940021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbd2cbfb2984f54 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.801+0000 7f4a88ff9640 1 -- 172.21.15.60:0/2415188021 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94042040 con 0x56260f42b9e0 2024-07-24T21:51:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.801+0000 7f4a88ff9640 1 -- 172.21.15.60:0/2415188021 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a94032e00 con 0x56260f42b9e0 2024-07-24T21:51:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.801+0000 7f4a88ff9640 1 -- 172.21.15.60:0/2415188021 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94037040 con 0x56260f42b9e0 2024-07-24T21:51:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.803+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 msgr2=0x56260f440cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.803+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007780 tx=0x7f4a940021d0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 shutdown_connections 2024-07-24T21:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f43ba80 0x56260f43d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f42c3e0 0x56260f441210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/2415188021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42b9e0 0x56260f440cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 >> 172.21.15.60:0/2415188021 conn(0x56260f42e140 msgr2=0x56260f43e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.812+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 shutdown_connections 2024-07-24T21:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.813+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/2415188021 wait complete. 2024-07-24T21:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.813+0000 7f4aa4b7b740 1 -- start start 2024-07-24T21:51:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.814+0000 7f4aa4b7b740 1 Processor -- start 2024-07-24T21:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42b9e0 0x56260f45aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42b9e0 0x56260f45aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f43ba80 0x56260f44b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42262/0 (socket says 172.21.15.60:42262) 2024-07-24T21:51:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4a9affd640 1 -- 172.21.15.60:0/3082089676 learned_addr learned my addr 172.21.15.60:0/3082089676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8000cfb0 con 0x56260f42b9e0 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8000e420 con 0x56260f42c3e0 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8000d680 con 0x56260f43ba80 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.815+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f43ba80 0x56260f44b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4aa4b7b740 10 client.0 _ll_register_callbacks cb 0x7f4a940076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4a9affd640 1 -- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f43ba80 msgr2=0x56260f44b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f43ba80 0x56260f44b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4a9affd640 1 -- 172.21.15.60:0/3082089676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42b9e0 msgr2=0x56260f45aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42b9e0 0x56260f45aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.816+0000 7f4a9affd640 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260f43ed30 con 0x56260f42c3e0 2024-07-24T21:51:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.817+0000 7f4a9affd640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0aa030 tx=0x7f4a8c004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132e3903b2899ad6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.838+0000 7f4a7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:51:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.838+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0c9060 con 0x56260f42c3e0 2024-07-24T21:51:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.838+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a94032e00 con 0x56260f42c3e0 2024-07-24T21:51:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.840+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4a8c010680 con 0x56260f42c3e0 2024-07-24T21:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.840+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0ae6a0 con 0x56260f42c3e0 2024-07-24T21:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.840+0000 7f4aa4b7b740 10 client.26123 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.841+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56260f43dc80 con 0x56260f42c3e0 2024-07-24T21:51:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.842+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8c0b3150 con 0x56260f42c3e0 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.843+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4a8c0a9300 con 0x56260f42c3e0 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.843+0000 7f4aa4b7b740 10 client.26123 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.843+0000 7f4aa4b7b740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.843+0000 7f4aa4b7b740 20 client.26123 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.843+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f4bdb70 0x56260f4bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.844+0000 7f4aa4b7b740 4 client.26123 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.844+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4a94032e00 con 0x56260f4bdb70 2024-07-24T21:51:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.844+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f4bdb70 0x56260f4bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.844+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f4bdb70 0x56260f4bff60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61395591f15ddc87 server_cookie=8b2f61697a8d7f4f in_seq=0 out_seq=0 2024-07-24T21:51:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:16.846+0000 7f4a7e7fc640 10 client.26123 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:16.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:51:17.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:17 smithi086 ceph-mon[37128]: pgmap v515: 97 pgs: 97 active+clean; 989 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 666 op/s 2024-07-24T21:51:17.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:17 smithi060 ceph-mon[33000]: pgmap v515: 97 pgs: 97 active+clean; 989 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 666 op/s 2024-07-24T21:51:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:17 smithi123 ceph-mon[37143]: pgmap v515: 97 pgs: 97 active+clean; 989 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 666 op/s 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.474+0000 7f4a7e7fc640 1 -- 172.21.15.60:0/3082089676 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4a94032e00 con 0x56260f4bdb70 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.474+0000 7f4a7e7fc640 10 client.26123 handle_command_reply: tid=0 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.474+0000 7f4aa4b7b740 1 client.26123 shutdown 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.474+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 msgr2=0x56260f45b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.474+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0aa030 tx=0x7f4a8c004700 comp rx=0 tx=0).stop 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 shutdown_connections 2024-07-24T21:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56260f4bdb70 0x56260f4bff60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56260f43ba80 0x56260f44b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56260f42c3e0 0x56260f45b0e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 --2- 172.21.15.60:0/3082089676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56260f42b9e0 0x56260f45aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 >> 172.21.15.60:0/3082089676 conn(0x56260f42e140 msgr2=0x56260f42aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4a7e7fc640 0 client.26123 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.475+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 shutdown_connections 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.476+0000 7f4aa4b7b740 1 -- 172.21.15.60:0/3082089676 wait complete. 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.476+0000 7f4aa4b7b740 20 client.26123 trim_cache size 0 max 16384 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:51:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 msgr2=0x7f4a840b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 0x7f4a840b6410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009830 tx=0x7f4a90009400 comp rx=0 tx=0).stop 2024-07-24T21:51:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 msgr2=0x7f4a9c1a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 0x7f4a9c1a1b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009fd0 tx=0x7f4a8c00b870 comp rx=0 tx=0).stop 2024-07-24T21:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 shutdown_connections 2024-07-24T21:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a840b3f50 0x7f4a840b6410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a9c1a6e90 0x7f4a9c1a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c113270 0x7f4a9c1a1b00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 --2- 172.21.15.60:0/3685199396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c10a6d0 0x7f4a9c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.550+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 >> 172.21.15.60:0/3685199396 conn(0x7f4a9c06d5a0 msgr2=0x7f4a9c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.551+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 shutdown_connections 2024-07-24T21:51:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:18.551+0000 7f4a8a7fc640 1 -- 172.21.15.60:0/3685199396 wait complete. 2024-07-24T21:51:19.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:19 smithi086 ceph-mon[37128]: pgmap v516: 97 pgs: 97 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 601 op/s 2024-07-24T21:51:19.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:19 smithi060 ceph-mon[33000]: pgmap v516: 97 pgs: 97 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 601 op/s 2024-07-24T21:51:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:19 smithi123 ceph-mon[37143]: pgmap v516: 97 pgs: 97 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 601 op/s 2024-07-24T21:51:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:21 smithi086 ceph-mon[37128]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 826 op/s 2024-07-24T21:51:21.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:21 smithi060 ceph-mon[33000]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 826 op/s 2024-07-24T21:51:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:21 smithi123 ceph-mon[37143]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 826 op/s 2024-07-24T21:51:22.686 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:23 smithi086 ceph-mon[37128]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:23.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:23 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:51:23.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:23 smithi060 ceph-mon[33000]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:23.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:23 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:51:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:23 smithi123 ceph-mon[37143]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:23 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:51:24.709 INFO:tasks.workunit.client.0.smithi060.stdout: 309 2101496 1441 1444550 1670 482394 1087 2024-07-24T21:51:25.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:25 smithi086 ceph-mon[37128]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 16 MiB/s wr, 995 op/s 2024-07-24T21:51:25.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:25 smithi060 ceph-mon[33000]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 16 MiB/s wr, 995 op/s 2024-07-24T21:51:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:25 smithi123 ceph-mon[37143]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 16 MiB/s wr, 995 op/s 2024-07-24T21:51:27.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.423+0000 7f831edd9640 1 -- 172.21.15.60:0/926750386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 msgr2=0x7f831810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.423+0000 7f831edd9640 1 --2- 172.21.15.60:0/926750386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831810e1f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8310009f00 tx=0x7f831002ec10 comp rx=0 tx=0).stop 2024-07-24T21:51:27.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.426+0000 7f831edd9640 1 -- 172.21.15.60:0/926750386 shutdown_connections 2024-07-24T21:51:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.426+0000 7f831edd9640 1 --2- 172.21.15.60:0/926750386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831810e1f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.426+0000 7f831edd9640 1 --2- 172.21.15.60:0/926750386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318107070 0x7f83181074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.426+0000 7f831edd9640 1 --2- 172.21.15.60:0/926750386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318101660 0x7f8318101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.426+0000 7f831edd9640 1 -- 172.21.15.60:0/926750386 >> 172.21.15.60:0/926750386 conn(0x7f83180fd0f0 msgr2=0x7f83180ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.429+0000 7f831edd9640 1 -- 172.21.15.60:0/926750386 shutdown_connections 2024-07-24T21:51:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.430+0000 7f831edd9640 1 -- 172.21.15.60:0/926750386 wait complete. 2024-07-24T21:51:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.431+0000 7f831edd9640 1 Processor -- start 2024-07-24T21:51:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 -- start start 2024-07-24T21:51:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318101660 0x7f8318074180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318107070 0x7f83180746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8318110570 con 0x7f83181079f0 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83181103f0 con 0x7f8318107070 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.432+0000 7f831edd9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83181106f0 con 0x7f8318101660 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.433+0000 7f831d376640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.433+0000 7f831d376640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38808/0 (socket says 172.21.15.60:38808) 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.433+0000 7f831d376640 1 -- 172.21.15.60:0/2544837328 learned_addr learned my addr 172.21.15.60:0/2544837328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.433+0000 7f8317fff640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318107070 0x7f83180746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.433+0000 7f831cb75640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318101660 0x7f8318074180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.434+0000 7f831d376640 1 -- 172.21.15.60:0/2544837328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318101660 msgr2=0x7f8318074180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.434+0000 7f831d376640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318101660 0x7f8318074180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.434+0000 7f831d376640 1 -- 172.21.15.60:0/2544837328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318107070 msgr2=0x7f83180746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.434+0000 7f831d376640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318107070 0x7f83180746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.434+0000 7f831d376640 1 -- 172.21.15.60:0/2544837328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831806ddc0 con 0x7f83181079f0 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.435+0000 7f831d376640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f831002f200 tx=0x7f8310037670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.435+0000 7f8315ffb640 1 -- 172.21.15.60:0/2544837328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831003c070 con 0x7f83181079f0 2024-07-24T21:51:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.435+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831806e050 con 0x7f83181079f0 2024-07-24T21:51:27.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.436+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8318072fc0 con 0x7f83181079f0 2024-07-24T21:51:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.444+0000 7f8315ffb640 1 -- 172.21.15.60:0/2544837328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8310003e20 con 0x7f83181079f0 2024-07-24T21:51:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.444+0000 7f8315ffb640 1 -- 172.21.15.60:0/2544837328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83100045c0 con 0x7f83181079f0 2024-07-24T21:51:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.444+0000 7f8315ffb640 1 -- 172.21.15.60:0/2544837328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f831000c500 con 0x7f83181079f0 2024-07-24T21:51:27.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.446+0000 7f8315ffb640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 0x7f83000b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.446+0000 7f8315ffb640 1 -- 172.21.15.60:0/2544837328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f8310108170 con 0x7f83181079f0 2024-07-24T21:51:27.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.449+0000 7f831cb75640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 0x7f83000b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.449+0000 7f832118a740 1 Processor -- start 2024-07-24T21:51:27.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.450+0000 7f831cb75640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 0x7f83000b6440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f83080058c0 tx=0x7f830800a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 -- start start 2024-07-24T21:51:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f02a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f11320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9e79960 con 0x55fdb9f02fa0 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9e68e60 con 0x55fdb9f02680 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.452+0000 7f832118a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9e6cea0 con 0x55fdb9f0ef30 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.453+0000 7f831cb75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f02a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.454+0000 7f831d376640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f11320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.454+0000 7f8317fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.454+0000 7f8317fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38824/0 (socket says 172.21.15.60:38824) 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.454+0000 7f8317fff640 1 -- 172.21.15.60:0/653841435 learned_addr learned my addr 172.21.15.60:0/653841435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 -- 172.21.15.60:0/653841435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ef30 msgr2=0x55fdb9f11320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f11320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 -- 172.21.15.60:0/653841435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 msgr2=0x55fdb9f02a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f02a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 -- 172.21.15.60:0/653841435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb9df22d0 con 0x55fdb9f02fa0 2024-07-24T21:51:27.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.455+0000 7f8317fff640 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f830c00cbc0 tx=0x7f830c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3e6d7c3c13d11e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.456+0000 7f82feffd640 1 -- 172.21.15.60:0/653841435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c010630 con 0x55fdb9f02fa0 2024-07-24T21:51:27.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.456+0000 7f82feffd640 1 -- 172.21.15.60:0/653841435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c010c80 con 0x55fdb9f02fa0 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.457+0000 7f832118a740 1 -- 172.21.15.60:0/653841435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 msgr2=0x55fdb9f0e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.457+0000 7f832118a740 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f830c00cbc0 tx=0x7f830c00e720 comp rx=0 tx=0).stop 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.458+0000 7f832118a740 1 -- 172.21.15.60:0/653841435 shutdown_connections 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.458+0000 7f832118a740 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f11320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.458+0000 7f832118a740 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02fa0 0x55fdb9f0e9f0 secure :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f830c00cbc0 tx=0x7f830c00e720 comp rx=0 tx=0).stop 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.458+0000 7f832118a740 1 --2- 172.21.15.60:0/653841435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f02a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.458+0000 7f832118a740 1 -- 172.21.15.60:0/653841435 >> 172.21.15.60:0/653841435 conn(0x55fdb9f004b0 msgr2=0x55fdb9f01650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.459+0000 7f832118a740 1 -- 172.21.15.60:0/653841435 shutdown_connections 2024-07-24T21:51:27.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.459+0000 7f832118a740 1 -- 172.21.15.60:0/653841435 wait complete. 2024-07-24T21:51:27.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.459+0000 7f832118a740 1 -- start start 2024-07-24T21:51:27.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 Processor -- start 2024-07-24T21:51:27.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:27 smithi060 ceph-mon[33000]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f1b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9ccc700 con 0x55fdb9f02680 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9e0b5a0 con 0x55fdb9f0ef30 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.460+0000 7f832118a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fdb9f11c00 con 0x55fdb9f0ca80 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.461+0000 7f832118a740 10 client.0 _ll_register_callbacks cb 0x55fdb9e2b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.461+0000 7f831cb75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.461+0000 7f831cb75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38830/0 (socket says 172.21.15.60:38830) 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.461+0000 7f831cb75640 1 -- 172.21.15.60:0/3535675557 learned_addr learned my addr 172.21.15.60:0/3535675557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 -- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 msgr2=0x55fdb9f1b010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f1b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 -- 172.21.15.60:0/3535675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 msgr2=0x55fdb9f0c540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 -- 172.21.15.60:0/3535675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb9dfe630 con 0x55fdb9f02680 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.462+0000 7f831cb75640 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8308010a60 tx=0x7f83080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5d89d5bc952637 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.463+0000 7f82f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.464+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830800fb40 con 0x55fdb9f02680 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.464+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830c010c80 con 0x55fdb9f02680 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.464+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f830800fce0 con 0x55fdb9f02680 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.465+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f830801d420 con 0x55fdb9f02680 2024-07-24T21:51:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.465+0000 7f832118a740 10 client.16668 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:27.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.465+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdb9bc9650 con 0x55fdb9f02680 2024-07-24T21:51:27.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c010c80 con 0x55fdb9f02680 2024-07-24T21:51:27.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8308025750 con 0x55fdb9f02680 2024-07-24T21:51:27.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 10 client.16668 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:27.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 20 client.16668 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f1f590 0x55fdb9f21a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 4 client.16668 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.467+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb9e7b8b0 con 0x55fdb9f1f590 2024-07-24T21:51:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.469+0000 7f8317fff640 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f1f590 0x55fdb9f21a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.473+0000 7f8317fff640 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f1f590 0x55fdb9f21a30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b373c4821dc2c9 server_cookie=7eb51424c9a88be9 in_seq=0 out_seq=0 2024-07-24T21:51:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.473+0000 7f82f7fff640 10 client.16668 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.476+0000 7f82f7fff640 1 -- 172.21.15.60:0/3535675557 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdb9e7b8b0 con 0x55fdb9f1f590 2024-07-24T21:51:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.476+0000 7f82f7fff640 10 client.16668 handle_command_reply: tid=0 2024-07-24T21:51:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.476+0000 7f832118a740 1 client.16668 shutdown 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 msgr2=0x55fdb9f0c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8308010a60 tx=0x7f83080046c0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 shutdown_connections 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f1f590 0x55fdb9f21a30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f1b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 --2- 172.21.15.60:0/3535675557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f02680 0x55fdb9f0c000 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 >> 172.21.15.60:0/3535675557 conn(0x55fdb9f004b0 msgr2=0x55fdb9f06050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.477+0000 7f82f7fff640 0 client.16668 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.478+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 shutdown_connections 2024-07-24T21:51:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.478+0000 7f832118a740 1 -- 172.21.15.60:0/3535675557 wait complete. 2024-07-24T21:51:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.478+0000 7f832118a740 20 client.16668 trim_cache size 0 max 16384 2024-07-24T21:51:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.526+0000 7f832118a740 1 Processor -- start 2024-07-24T21:51:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 -- start start 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f0a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8304011330 con 0x55fdb9f0ef30 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f830400eba0 con 0x55fdb9f02680 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f832118a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f830400c9f0 con 0x55fdb9f0ca80 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.527+0000 7f831cb75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831cb75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44682/0 (socket says 172.21.15.60:44682) 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831cb75640 1 -- 172.21.15.60:0/2770943118 learned_addr learned my addr 172.21.15.60:0/2770943118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 -- 172.21.15.60:0/2770943118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 msgr2=0x55fdb9f0a880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:51:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f0a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 -- 172.21.15.60:0/2770943118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 msgr2=0x55fdb9f1ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831d376640 1 -- 172.21.15.60:0/2770943118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb9f0a500 con 0x55fdb9f0ef30 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.528+0000 7f831cb75640 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.529+0000 7f831d376640 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0adc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8310102460 tx=0x7f831002ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feba68c30581a73 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.530+0000 7f82fe7fc640 1 -- 172.21.15.60:0/2770943118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310109060 con 0x55fdb9f0ef30 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.530+0000 7f82fe7fc640 1 -- 172.21.15.60:0/2770943118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83100098d0 con 0x55fdb9f0ef30 2024-07-24T21:51:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.530+0000 7f82fe7fc640 1 -- 172.21.15.60:0/2770943118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83100cb920 con 0x55fdb9f0ef30 2024-07-24T21:51:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.531+0000 7f832118a740 1 -- 172.21.15.60:0/2770943118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 msgr2=0x55fdb9f0adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.531+0000 7f832118a740 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0adc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8310102460 tx=0x7f831002ec10 comp rx=0 tx=0).stop 2024-07-24T21:51:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.533+0000 7f832118a740 1 -- 172.21.15.60:0/2770943118 shutdown_connections 2024-07-24T21:51:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.533+0000 7f832118a740 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f0adc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.533+0000 7f832118a740 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f0a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.533+0000 7f832118a740 1 --2- 172.21.15.60:0/2770943118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f02680 0x55fdb9f1ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.533+0000 7f832118a740 1 -- 172.21.15.60:0/2770943118 >> 172.21.15.60:0/2770943118 conn(0x55fdb9f004b0 msgr2=0x55fdb9f0bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.536+0000 7f832118a740 1 -- 172.21.15.60:0/2770943118 shutdown_connections 2024-07-24T21:51:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.536+0000 7f832118a740 1 -- 172.21.15.60:0/2770943118 wait complete. 2024-07-24T21:51:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.537+0000 7f832118a740 1 -- start start 2024-07-24T21:51:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.537+0000 7f832118a740 1 Processor -- start 2024-07-24T21:51:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.538+0000 7f832118a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f02680 0x55fdb9f12700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.538+0000 7f832118a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f832118a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f832118a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f830400d060 con 0x55fdb9f0ca80 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f8317fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f8317fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38836/0 (socket says 172.21.15.60:38836) 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f8317fff640 1 -- 172.21.15.60:0/1942284617 learned_addr learned my addr 172.21.15.60:0/1942284617 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f831d376640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f831cb75640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f02680 0x55fdb9f12700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f832118a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f830400e4d0 con 0x55fdb9f0ef30 2024-07-24T21:51:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f830400d730 con 0x55fdb9f02680 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.539+0000 7f832118a740 10 client.0 _ll_register_callbacks cb 0x7f8308014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 -- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f02680 msgr2=0x55fdb9f12700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f02680 0x55fdb9f12700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 -- 172.21.15.60:0/1942284617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 msgr2=0x55fdb9f88300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f88300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 -- 172.21.15.60:0/1942284617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb9f88a10 con 0x55fdb9f0ca80 2024-07-24T21:51:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f8317fff640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f830c01a770 tx=0x7f830c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420bf9a638376654 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.540+0000 7f82f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.543+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c01a410 con 0x55fdb9f0ca80 2024-07-24T21:51:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.543+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdb9f0b860 con 0x55fdb9f0ca80 2024-07-24T21:51:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.543+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f830c005070 con 0x55fdb9f0ca80 2024-07-24T21:51:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.547+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f830c005270 con 0x55fdb9f0ca80 2024-07-24T21:51:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.547+0000 7f832118a740 10 client.16674 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.547+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdb9f89d00 con 0x55fdb9f0ca80 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fdb9f0b860 con 0x55fdb9f0ca80 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f830c0283e0 con 0x55fdb9f0ca80 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 10 client.16674 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 20 client.16674 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f8a3e0 0x55fdb9f8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 4 client.16674 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.548+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdb9e77350 con 0x55fdb9f8a3e0 2024-07-24T21:51:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.549+0000 7f831cb75640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f8a3e0 0x55fdb9f8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.566+0000 7f831cb75640 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f8a3e0 0x55fdb9f8c800 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc61c75d7d4c4e8b server_cookie=e902c08fc358d920 in_seq=0 out_seq=0 2024-07-24T21:51:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.566+0000 7f82f7fff640 10 client.16674 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.569+0000 7f82f7fff640 1 -- 172.21.15.60:0/1942284617 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fdb9e77350 con 0x55fdb9f8a3e0 2024-07-24T21:51:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.569+0000 7f82f7fff640 10 client.16674 handle_command_reply: tid=0 2024-07-24T21:51:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.569+0000 7f832118a740 1 client.16674 shutdown 2024-07-24T21:51:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.570+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 msgr2=0x55fdb9f12c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.570+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f830c01a770 tx=0x7f830c005ce0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.572+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 shutdown_connections 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fdb9f8a3e0 0x55fdb9f8c800 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb9f0ef30 0x55fdb9f88300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fdb9f0ca80 0x55fdb9f12c80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f82f7fff640 0 client.16674 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f832118a740 1 --2- 172.21.15.60:0/1942284617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fdb9f02680 0x55fdb9f12700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.573+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 >> 172.21.15.60:0/1942284617 conn(0x55fdb9f004b0 msgr2=0x55fdb9f0a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.583+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 shutdown_connections 2024-07-24T21:51:27.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.584+0000 7f832118a740 1 -- 172.21.15.60:0/1942284617 wait complete. 2024-07-24T21:51:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.589+0000 7f832118a740 20 client.16674 trim_cache size 0 max 16384 2024-07-24T21:51:27.594 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:51:27.594 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:51:27.594 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:51:27.594 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:51:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.592+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 msgr2=0x7f83000b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.592+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 0x7f83000b6440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f83080058c0 tx=0x7f830800a4c0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.592+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 msgr2=0x7f831806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.592+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f831002f200 tx=0x7f8310037670 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 shutdown_connections 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f83000b3f80 0x7f83000b6440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83181079f0 0x7f831806d500 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8318107070 0x7f83180746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 --2- 172.21.15.60:0/2544837328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8318101660 0x7f8318074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.593+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 >> 172.21.15.60:0/2544837328 conn(0x7f83180fd0f0 msgr2=0x7f83180fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.596+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 shutdown_connections 2024-07-24T21:51:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:27.596+0000 7f831edd9640 1 -- 172.21.15.60:0/2544837328 wait complete. 2024-07-24T21:51:27.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:27 smithi123 ceph-mon[37143]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-07-24T21:51:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:27 smithi086 ceph-mon[37128]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-07-24T21:51:29.043 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:51:29.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:51:29.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:29 smithi060 ceph-mon[33000]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-07-24T21:51:29.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:29 smithi123 ceph-mon[37143]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-07-24T21:51:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:29 smithi086 ceph-mon[37128]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-07-24T21:51:31.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:31 smithi060 ceph-mon[33000]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 865 op/s 2024-07-24T21:51:31.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:31 smithi123 ceph-mon[37143]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 865 op/s 2024-07-24T21:51:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:31 smithi086 ceph-mon[37128]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 865 op/s 2024-07-24T21:51:33.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-07-24T21:51:33.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T21:51:33.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T21:51:33.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:51:33.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:51:33.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:33 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:51:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:33 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 683 op/s 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T21:51:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:51:33.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:51:33.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:33 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:51:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.938+0000 7f0171e80640 1 -- 172.21.15.60:0/178653291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109b40 msgr2=0x7f016c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.938+0000 7f0171e80640 1 --2- 172.21.15.60:0/178653291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109b40 0x7f016c110ba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0164009ca0 tx=0x7f016402a8f0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 -- 172.21.15.60:0/178653291 shutdown_connections 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 --2- 172.21.15.60:0/178653291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c11ba90 0x7f016c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 --2- 172.21.15.60:0/178653291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c109f20 0x7f016c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 --2- 172.21.15.60:0/178653291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109b40 0x7f016c110ba0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 -- 172.21.15.60:0/178653291 >> 172.21.15.60:0/178653291 conn(0x7f016c06d5a0 msgr2=0x7f016c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 -- 172.21.15.60:0/178653291 shutdown_connections 2024-07-24T21:51:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.939+0000 7f0171e80640 1 -- 172.21.15.60:0/178653291 wait complete. 2024-07-24T21:51:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.941+0000 7f0171e80640 1 Processor -- start 2024-07-24T21:51:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 -- start start 2024-07-24T21:51:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c109b40 0x7f016c118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 0x7f016c119400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 0x7f016c112250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f016c121310 con 0x7f016c109b40 2024-07-24T21:51:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f016c121190 con 0x7f016c109f20 2024-07-24T21:51:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.942+0000 7f0171e80640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f016c121490 con 0x7f016c11ba90 2024-07-24T21:51:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.943+0000 7f016bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 0x7f016c112250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.943+0000 7f016bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 0x7f016c112250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46764/0 (socket says 172.21.15.60:46764) 2024-07-24T21:51:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.943+0000 7f016bfff640 1 -- 172.21.15.60:0/3617197499 learned_addr learned my addr 172.21.15.60:0/3617197499 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.943+0000 7f016affd640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 0x7f016c119400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.944+0000 7f016affd640 1 -- 172.21.15.60:0/3617197499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 msgr2=0x7f016c112250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.944+0000 7f016affd640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 0x7f016c112250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.944+0000 7f016affd640 1 -- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c109b40 msgr2=0x7f016c118ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:51:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.958+0000 7f016b7fe640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c109b40 0x7f016c118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.958+0000 7f016affd640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c109b40 0x7f016c118ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.958+0000 7f016affd640 1 -- 172.21.15.60:0/3617197499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c112b10 con 0x7f016c109f20 2024-07-24T21:51:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.958+0000 7f016affd640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 0x7f016c119400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f016000c710 tx=0x7f016000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:33.997+0000 7f0168ff9640 1 -- 172.21.15.60:0/3617197499 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160004220 con 0x7f016c109f20 2024-07-24T21:51:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.001+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c112e00 con 0x7f016c109f20 2024-07-24T21:51:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.001+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c117db0 con 0x7f016c109f20 2024-07-24T21:51:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.005+0000 7f0174bcd740 1 Processor -- start 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 -- start start 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e69cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 0x55883e6ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55883e612280 con 0x55883e69d3e0 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55883e601780 con 0x55883e69cac0 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f0174bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55883e6057c0 con 0x55883e6a9370 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38858/0 (socket says 172.21.15.60:38858) 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f016b7fe640 1 -- 172.21.15.60:0/1851813414 learned_addr learned my addr 172.21.15.60:0/1851813414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f016bfff640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e69cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.006+0000 7f016affd640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 0x55883e6ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016b7fe640 1 -- 172.21.15.60:0/1851813414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 msgr2=0x55883e6ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016b7fe640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 0x55883e6ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016b7fe640 1 -- 172.21.15.60:0/1851813414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 msgr2=0x55883e69cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016b7fe640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e69cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016b7fe640 1 -- 172.21.15.60:0/1851813414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883e365790 con 0x55883e69d3e0 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016affd640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 0x55883e6ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f016bfff640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e69cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f0168ff9640 1 -- 172.21.15.60:0/3617197499 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016000ce20 con 0x7f016c109f20 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.007+0000 7f0168ff9640 1 -- 172.21.15.60:0/3617197499 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160003da0 con 0x7f016c109f20 2024-07-24T21:51:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.008+0000 7f016b7fe640 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0164000f80 tx=0x7f0164031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc3b8a1160ec7e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.017+0000 7f0168ff9640 1 -- 172.21.15.60:0/3617197499 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f01600079b0 con 0x7f016c109f20 2024-07-24T21:51:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.022+0000 7f0168ff9640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 0x7f01540b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.023+0000 7f016bfff640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 0x7f01540b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.024+0000 7f016bfff640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 0x7f01540b6410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f015c0058c0 tx=0x7f015c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.024+0000 7f0168ff9640 1 -- 172.21.15.60:0/3617197499 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f01600e3ac0 con 0x7f016c109f20 2024-07-24T21:51:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.028+0000 7f0159ffb640 1 -- 172.21.15.60:0/1851813414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164031480 con 0x55883e69d3e0 2024-07-24T21:51:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.028+0000 7f0159ffb640 1 -- 172.21.15.60:0/1851813414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0164036070 con 0x55883e69d3e0 2024-07-24T21:51:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 -- 172.21.15.60:0/1851813414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 msgr2=0x55883e6a8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0164000f80 tx=0x7f0164031b10 comp rx=0 tx=0).stop 2024-07-24T21:51:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 -- 172.21.15.60:0/1851813414 shutdown_connections 2024-07-24T21:51:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a9370 0x55883e6ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69d3e0 0x55883e6a8e30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 --2- 172.21.15.60:0/1851813414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e69cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 -- 172.21.15.60:0/1851813414 >> 172.21.15.60:0/1851813414 conn(0x55883e69a8f0 msgr2=0x55883e69ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 -- 172.21.15.60:0/1851813414 shutdown_connections 2024-07-24T21:51:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.030+0000 7f0174bcd740 1 -- 172.21.15.60:0/1851813414 wait complete. 2024-07-24T21:51:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.031+0000 7f0174bcd740 1 -- start start 2024-07-24T21:51:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.031+0000 7f0174bcd740 1 Processor -- start 2024-07-24T21:51:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55883e53bab0 con 0x55883e69cac0 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55883e5899e0 con 0x55883e6a9370 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55883e6ce050 con 0x55883e6a6510 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.032+0000 7f0174bcd740 10 client.0 _ll_register_callbacks cb 0x55883e311a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38866/0 (socket says 172.21.15.60:38866) 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 -- 172.21.15.60:0/2188062307 learned_addr learned my addr 172.21.15.60:0/2188062307 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016bfff640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016affd640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 -- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 msgr2=0x55883e6a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 -- 172.21.15.60:0/2188062307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 msgr2=0x55883e6a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.033+0000 7f016b7fe640 1 -- 172.21.15.60:0/2188062307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883e3de620 con 0x55883e69cac0 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.034+0000 7f016affd640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.034+0000 7f016bfff640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.034+0000 7f016b7fe640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0164009a60 tx=0x7f01640098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e7da7a05d5512d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.070+0000 7f014effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.070+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164031480 con 0x55883e69cac0 2024-07-24T21:51:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.070+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55883e5b0ce0 con 0x55883e69cac0 2024-07-24T21:51:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.070+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0164049040 con 0x55883e69cac0 2024-07-24T21:51:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.071+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0164031620 con 0x55883e69cac0 2024-07-24T21:51:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.071+0000 7f0174bcd740 10 client.16680 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.071+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55883e6d9d80 con 0x55883e69cac0 2024-07-24T21:51:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.071+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55883e5b0ce0 con 0x55883e69cac0 2024-07-24T21:51:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0164030a10 con 0x55883e69cac0 2024-07-24T21:51:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 10 client.16680 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 20 client.16680 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e6db2e0 0x55883e6dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 4 client.16680 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883e6141d0 con 0x55883e6db2e0 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.072+0000 7f016affd640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e6db2e0 0x55883e6dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.073+0000 7f016affd640 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e6db2e0 0x55883e6dd700 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecd0664918d5fb2 server_cookie=d74cc74d8631b23c in_seq=0 out_seq=0 2024-07-24T21:51:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f014effd640 10 client.16680 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f014effd640 1 -- 172.21.15.60:0/2188062307 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55883e6141d0 con 0x55883e6db2e0 2024-07-24T21:51:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f014effd640 10 client.16680 handle_command_reply: tid=0 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f0174bcd740 1 client.16680 shutdown 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 msgr2=0x55883e6a8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.075+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0164009a60 tx=0x7f01640098b0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 shutdown_connections 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e6db2e0 0x55883e6dd700 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2188062307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e69cac0 0x55883e6a8b90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.076+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 >> 172.21.15.60:0/2188062307 conn(0x55883e69a8f0 msgr2=0x55883e6a0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.079+0000 7f014effd640 0 client.16680 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.079+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 shutdown_connections 2024-07-24T21:51:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.079+0000 7f0174bcd740 1 -- 172.21.15.60:0/2188062307 wait complete. 2024-07-24T21:51:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.079+0000 7f0174bcd740 20 client.16680 trim_cache size 0 max 16384 2024-07-24T21:51:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.372+0000 7f0174bcd740 1 Processor -- start 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 -- start start 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 0x55883e6a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0150011290 con 0x55883e6a6510 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f015000eb00 con 0x55883e6a9370 2024-07-24T21:51:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f0174bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f015000c950 con 0x55883e69cac0 2024-07-24T21:51:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46792/0 (socket says 172.21.15.60:46792) 2024-07-24T21:51:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f016b7fe640 1 -- 172.21.15.60:0/2045117545 learned_addr learned my addr 172.21.15.60:0/2045117545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.394+0000 7f016affd640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 0x55883e6a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.395+0000 7f016bfff640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016bfff640 1 -- 172.21.15.60:0/2045117545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 msgr2=0x55883e69c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016bfff640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016bfff640 1 -- 172.21.15.60:0/2045117545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 msgr2=0x55883e6a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016bfff640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 0x55883e6a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016bfff640 1 -- 172.21.15.60:0/2045117545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883e69d7c0 con 0x55883e6a9370 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.401+0000 7f016affd640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 0x55883e6a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.402+0000 7f016b7fe640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.402+0000 7f016bfff640 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a8610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f015c010c90 tx=0x7f015c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19334af0882d1685 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.405+0000 7f01597fa640 1 -- 172.21.15.60:0/2045117545 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c01ccd0 con 0x55883e6a9370 2024-07-24T21:51:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.406+0000 7f01597fa640 1 -- 172.21.15.60:0/2045117545 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c02d070 con 0x55883e6a9370 2024-07-24T21:51:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.406+0000 7f01597fa640 1 -- 172.21.15.60:0/2045117545 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00f710 con 0x55883e6a9370 2024-07-24T21:51:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.412+0000 7f0174bcd740 1 -- 172.21.15.60:0/2045117545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 msgr2=0x55883e6a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.412+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a8610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f015c010c90 tx=0x7f015c019000 comp rx=0 tx=0).stop 2024-07-24T21:51:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 -- 172.21.15.60:0/2045117545 shutdown_connections 2024-07-24T21:51:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e6a9370 0x55883e6a8610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a6510 0x55883e6a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 --2- 172.21.15.60:0/2045117545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e69cac0 0x55883e69c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 -- 172.21.15.60:0/2045117545 >> 172.21.15.60:0/2045117545 conn(0x55883e69a8f0 msgr2=0x55883e6a0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 -- 172.21.15.60:0/2045117545 shutdown_connections 2024-07-24T21:51:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 -- 172.21.15.60:0/2045117545 wait complete. 2024-07-24T21:51:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.425+0000 7f0174bcd740 1 -- start start 2024-07-24T21:51:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.426+0000 7f0174bcd740 1 Processor -- start 2024-07-24T21:51:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e6d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 0x55883e6ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f015000cfc0 con 0x55883e6a9370 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f015000e430 con 0x55883e69cac0 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f015000d690 con 0x55883e6a6510 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46808/0 (socket says 172.21.15.60:46808) 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016affd640 1 -- 172.21.15.60:0/4161749996 learned_addr learned my addr 172.21.15.60:0/4161749996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f0174bcd740 10 client.0 _ll_register_callbacks cb 0x7f015c00a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016bfff640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 0x55883e6ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016b7fe640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e6d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016affd640 1 -- 172.21.15.60:0/4161749996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 msgr2=0x55883e6d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.439+0000 7f016affd640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e6d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.440+0000 7f016affd640 1 -- 172.21.15.60:0/4161749996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 msgr2=0x55883e6ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.440+0000 7f016affd640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 0x55883e6ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.440+0000 7f016affd640 1 -- 172.21.15.60:0/4161749996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883e6ce8c0 con 0x55883e6a6510 2024-07-24T21:51:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.440+0000 7f016bfff640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 0x55883e6ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.440+0000 7f016affd640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f016000cfd0 tx=0x7f01600aa4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e982ac1d60300f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.458+0000 7f014effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:51:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.458+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016001d040 con 0x55883e6a6510 2024-07-24T21:51:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.458+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0160007ca0 con 0x55883e6a6510 2024-07-24T21:51:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.458+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c02d070 con 0x55883e6a6510 2024-07-24T21:51:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.460+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0160003610 con 0x55883e6a6510 2024-07-24T21:51:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.460+0000 7f0174bcd740 10 client.26158 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.461+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55883e744800 con 0x55883e6a6510 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c02d070 con 0x55883e6a6510 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f01600c9070 con 0x55883e6a6510 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f0174bcd740 10 client.26158 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f0174bcd740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f0174bcd740 20 client.26158 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e744f90 0x55883e7473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.462+0000 7f0174bcd740 4 client.26158 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:51:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.463+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55883e60fc70 con 0x55883e744f90 2024-07-24T21:51:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.463+0000 7f016b7fe640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e744f90 0x55883e7473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.463+0000 7f016b7fe640 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e744f90 0x55883e7473b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801732cd65b5e530 server_cookie=faf81b5101b62bfb in_seq=0 out_seq=0 2024-07-24T21:51:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:34.464+0000 7f014effd640 10 client.26158 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:34.709 INFO:tasks.workunit.client.0.smithi060.stdout: 347 2231714 1982 1570605 2264 523074 1864 2024-07-24T21:51:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:35 smithi060 ceph-mon[33000]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-07-24T21:51:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:35.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:35 smithi123 ceph-mon[37143]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-07-24T21:51:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:35 smithi086 ceph-mon[37128]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-07-24T21:51:37.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:37 smithi060 ceph-mon[33000]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-07-24T21:51:37.597 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:37 smithi086 ceph-mon[37128]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-07-24T21:51:37.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:37 smithi123 ceph-mon[37143]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 544 op/s 2024-07-24T21:51:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.871+0000 7f014effd640 1 -- 172.21.15.60:0/4161749996 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55883e60fc70 con 0x55883e744f90 2024-07-24T21:51:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.871+0000 7f014effd640 10 client.26158 handle_command_reply: tid=0 2024-07-24T21:51:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.871+0000 7f0174bcd740 1 client.26158 shutdown 2024-07-24T21:51:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 msgr2=0x55883e6d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f016000cfd0 tx=0x7f01600aa4b0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 shutdown_connections 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55883e744f90 0x55883e7473b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55883e6a9370 0x55883e6ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55883e6a6510 0x55883e6d3910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 --2- 172.21.15.60:0/4161749996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55883e69cac0 0x55883e6d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 >> 172.21.15.60:0/4161749996 conn(0x55883e69a8f0 msgr2=0x55883e6a5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f014effd640 0 client.26158 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 shutdown_connections 2024-07-24T21:51:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.874+0000 7f0174bcd740 1 -- 172.21.15.60:0/4161749996 wait complete. 2024-07-24T21:51:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.875+0000 7f0174bcd740 20 client.26158 trim_cache size 0 max 16384 2024-07-24T21:51:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:51:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:51:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:51:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:51:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.885+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 msgr2=0x7f01540b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.885+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 0x7f01540b6410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f015c0058c0 tx=0x7f015c005f90 comp rx=0 tx=0).stop 2024-07-24T21:51:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.885+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 msgr2=0x7f016c119400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.885+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 0x7f016c119400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f016000c710 tx=0x7f016000cbd0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 shutdown_connections 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f01540b3f50 0x7f01540b6410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c11ba90 0x7f016c112250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f016c109f20 0x7f016c119400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 --2- 172.21.15.60:0/3617197499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016c109b40 0x7f016c118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.886+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 >> 172.21.15.60:0/3617197499 conn(0x7f016c06d5a0 msgr2=0x7f016c11bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.887+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 shutdown_connections 2024-07-24T21:51:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:37.887+0000 7f0171e80640 1 -- 172.21.15.60:0/3617197499 wait complete. 2024-07-24T21:51:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 -- 172.21.15.60:0/891837211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc109b40 msgr2=0x7fdebc110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 --2- 172.21.15.60:0/891837211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc109b40 0x7fdebc110ba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb802a4d0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 -- 172.21.15.60:0/891837211 shutdown_connections 2024-07-24T21:51:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 --2- 172.21.15.60:0/891837211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc11ba90 0x7fdebc11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 --2- 172.21.15.60:0/891837211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109f20 0x7fdebc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 --2- 172.21.15.60:0/891837211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc109b40 0x7fdebc110ba0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.265+0000 7fdec3a86640 1 -- 172.21.15.60:0/891837211 >> 172.21.15.60:0/891837211 conn(0x7fdebc06d980 msgr2=0x7fdebc06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.267+0000 7fdec3a86640 1 -- 172.21.15.60:0/891837211 shutdown_connections 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.267+0000 7fdec3a86640 1 -- 172.21.15.60:0/891837211 wait complete. 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.268+0000 7fdec3a86640 1 Processor -- start 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 -- start start 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc109f20 0x7fdebc119450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc11ba90 0x7fdebc112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:38.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc121670 con 0x7fdebc11ba90 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1214f0 con 0x7fdebc109f20 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec3a86640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1217f0 con 0x7fdebc109b40 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.271+0000 7fdec1822640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53422/0 (socket says 172.21.15.60:53422) 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 -- 172.21.15.60:0/778950155 learned_addr learned my addr 172.21.15.60:0/778950155 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 -- 172.21.15.60:0/778950155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc109f20 msgr2=0x7fdebc119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc109f20 0x7fdebc119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 -- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc11ba90 msgr2=0x7fdebc112300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:38.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc11ba90 0x7fdebc112300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.272+0000 7fdec1822640 1 -- 172.21.15.60:0/778950155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc112bc0 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.278+0000 7fdec1822640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802aa40 tx=0x7fdeb8003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.278+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb803f030 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.278+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb8032bf0 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.279+0000 7fdec3a86640 1 -- 172.21.15.60:0/778950155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc112e50 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.279+0000 7fdec3a86640 1 -- 172.21.15.60:0/778950155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc117e20 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.279+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035070 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.281+0000 7fdec3a86640 1 -- 172.21.15.60:0/778950155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea0005290 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.286+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fdeb8042030 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.287+0000 7fdeb2ffd640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 0x7fdea80b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.288+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fdeb81045e0 con 0x7fdebc109b40 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.288+0000 7fdec1021640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 0x7fdea80b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.290+0000 7fdec1021640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 0x7fdea80b6410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdebc11e210 tx=0x7fdeb4009250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:38.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.295+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdeb8049030 con 0x7fdebc109b40 2024-07-24T21:51:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.834+0000 7fdec3a86640 1 -- 172.21.15.60:0/778950155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdea0005850 con 0x7fdebc109b40 2024-07-24T21:51:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.835+0000 7fdeb2ffd640 1 -- 172.21.15.60:0/778950155 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11919 (secure 0 0 0) 0x7fdeb80cb890 con 0x7fdebc109b40 2024-07-24T21:51:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:51:38.838 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":31,"btime":"2024-07-24T21:50:23:599643+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":25963,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3836191659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3836191659},{"type":"v1","addr":"172.21.15.123:6835","nonce":3836191659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:50:17.948711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24556":{"gid":24556,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1405683512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1405683512},{"type":"v1","addr":"172.21.15.123:6833","nonce":1405683512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:51:38.838 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 31 2024-07-24T21:51:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.846+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 msgr2=0x7fdea80b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.846+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 0x7fdea80b6410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdebc11e210 tx=0x7fdeb4009250 comp rx=0 tx=0).stop 2024-07-24T21:51:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.846+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 msgr2=0x7fdebc118f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.846+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802aa40 tx=0x7fdeb8003e30 comp rx=0 tx=0).stop 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 shutdown_connections 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdea80b3f50 0x7fdea80b6410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdebc11ba90 0x7fdebc112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc109f20 0x7fdebc119450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 --2- 172.21.15.60:0/778950155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdebc109b40 0x7fdebc118f10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 >> 172.21.15.60:0/778950155 conn(0x7fdebc06d980 msgr2=0x7fdebc11c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.850+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 shutdown_connections 2024-07-24T21:51:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:38.853+0000 7fdeb0f39640 1 -- 172.21.15.60:0/778950155 wait complete. 2024-07-24T21:51:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:39 smithi060 ceph-mon[33000]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-07-24T21:51:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/778950155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:51:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:39 smithi123 ceph-mon[37143]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-07-24T21:51:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/778950155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:51:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:39 smithi086 ceph-mon[37128]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-07-24T21:51:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/778950155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:51:40.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:51:41.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:41 smithi060 ceph-mon[33000]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-07-24T21:51:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:41 smithi123 ceph-mon[37143]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-07-24T21:51:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:41 smithi086 ceph-mon[37128]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-07-24T21:51:43.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:43 smithi123 ceph-mon[37143]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-07-24T21:51:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:43 smithi086 ceph-mon[37128]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-07-24T21:51:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:43 smithi060 ceph-mon[33000]: pgmap v528: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-07-24T21:51:44.712 INFO:tasks.workunit.client.0.smithi060.stdout: 361 1517895 995 1074513 839 373296 841 2024-07-24T21:51:45.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:45 smithi060 ceph-mon[33000]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-07-24T21:51:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:45 smithi123 ceph-mon[37143]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-07-24T21:51:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:45 smithi086 ceph-mon[37128]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-07-24T21:51:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:47.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:47 smithi060 ceph-mon[33000]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-07-24T21:51:47.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:47 smithi123 ceph-mon[37143]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-07-24T21:51:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:47 smithi086 ceph-mon[37128]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-07-24T21:51:49.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:49 smithi060 ceph-mon[33000]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-07-24T21:51:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:49 smithi123 ceph-mon[37143]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-07-24T21:51:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:49 smithi086 ceph-mon[37128]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-07-24T21:51:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 -- 172.21.15.60:0/815171420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 msgr2=0x7f21081074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 --2- 172.21.15.60:0/815171420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081074a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009900 tx=0x7f20f802a410 comp rx=0 tx=0).stop 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 -- 172.21.15.60:0/815171420 shutdown_connections 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 --2- 172.21.15.60:0/815171420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 0x7f210810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 --2- 172.21.15.60:0/815171420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081074a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 --2- 172.21.15.60:0/815171420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.300+0000 7f211048e640 1 -- 172.21.15.60:0/815171420 >> 172.21.15.60:0/815171420 conn(0x7f21080fd0c0 msgr2=0x7f21080ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.301+0000 7f211048e640 1 -- 172.21.15.60:0/815171420 shutdown_connections 2024-07-24T21:51:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.301+0000 7f211048e640 1 -- 172.21.15.60:0/815171420 wait complete. 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.301+0000 7f211048e640 1 Processor -- start 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.301+0000 7f211048e640 1 -- start start 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 0x7f210819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2108111600 con 0x7f2108107060 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2108111480 con 0x7f21081079e0 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f211048e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2108111780 con 0x7f2108101650 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46976/0 (socket says 172.21.15.60:46976) 2024-07-24T21:51:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 -- 172.21.15.60:0/4170568097 learned_addr learned my addr 172.21.15.60:0/4170568097 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210da29640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210ea2b640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 0x7f210819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 -- 172.21.15.60:0/4170568097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 msgr2=0x7f210819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 0x7f210819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.302+0000 7f210e22a640 1 -- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 msgr2=0x7f21081991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f210e22a640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f210e22a640 1 -- 172.21.15.60:0/4170568097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819ed30 con 0x7f2108101650 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f210da29640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f210e22a640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f210400e9f0 tx=0x7f210400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f20ff7fe640 1 -- 172.21.15.60:0/4170568097 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210400ccc0 con 0x7f2108101650 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210819f020 con 0x7f2108101650 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f20ff7fe640 1 -- 172.21.15.60:0/4170568097 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f210400ce60 con 0x7f2108101650 2024-07-24T21:51:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f20ff7fe640 1 -- 172.21.15.60:0/4170568097 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104010430 con 0x7f2108101650 2024-07-24T21:51:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.303+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a56e0 con 0x7f2108101650 2024-07-24T21:51:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 Processor -- start 2024-07-24T21:51:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 -- start start 2024-07-24T21:51:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f20ff7fe640 1 -- 172.21.15.60:0/4170568097 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f21040046c0 con 0x7f2108101650 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112cc3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cd6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b112c3a9b0 con 0x55b112cc3ff0 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b112c29eb0 con 0x55b112cc36d0 2024-07-24T21:51:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f211283f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b112c2def0 con 0x55b112cd4390 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f210e22a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f210e22a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47478/0 (socket says 172.21.15.60:47478) 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f210e22a640 1 -- 172.21.15.60:0/3404997250 learned_addr learned my addr 172.21.15.60:0/3404997250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.306+0000 7f210da29640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cd6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210ea2b640 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112cc3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 -- 172.21.15.60:0/3404997250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 msgr2=0x55b112cd6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cd6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 -- 172.21.15.60:0/3404997250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 msgr2=0x55b112cc3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112cc3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 -- 172.21.15.60:0/3404997250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b112bb3320 con 0x55b112cc3ff0 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210da29640 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cd6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210e22a640 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f21040e1a00 tx=0x7f21040ec470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d930d8d2c7267ed server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f20ff7fe640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 0x7f20f40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.307+0000 7f210ea2b640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 0x7f20f40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.308+0000 7f20ff7fe640 1 -- 172.21.15.60:0/4170568097 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f21040e4340 con 0x7f2108101650 2024-07-24T21:51:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.308+0000 7f20fcff9640 1 -- 172.21.15.60:0/3404997250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040f8070 con 0x55b112cc3ff0 2024-07-24T21:51:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.308+0000 7f20fcff9640 1 -- 172.21.15.60:0/3404997250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21040aaa60 con 0x55b112cc3ff0 2024-07-24T21:51:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.308+0000 7f20fcff9640 1 -- 172.21.15.60:0/3404997250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040aa8a0 con 0x55b112cc3ff0 2024-07-24T21:51:50.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- 172.21.15.60:0/3404997250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 msgr2=0x55b112cd3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f21040e1a00 tx=0x7f21040ec470 comp rx=0 tx=0).stop 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f210ea2b640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 0x7f20f40b6280 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2100005fd0 tx=0x7f2100009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- 172.21.15.60:0/3404997250 shutdown_connections 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cd6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cd3e50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- 172.21.15.60:0/3404997250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112cc3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- 172.21.15.60:0/3404997250 >> 172.21.15.60:0/3404997250 conn(0x55b112cc1500 msgr2=0x55b112cc26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- 172.21.15.60:0/3404997250 shutdown_connections 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- 172.21.15.60:0/3404997250 wait complete. 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 -- start start 2024-07-24T21:51:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 Processor -- start 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cd12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.309+0000 7f211283f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 0x55b112cd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f211283f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f211283f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b112a8d700 con 0x55b112cd4390 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f211283f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b112bcc5f0 con 0x55b112cc3ff0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f211283f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b112cd7090 con 0x55b112cc36d0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f211283f740 10 client.0 _ll_register_callbacks cb 0x55b112bec960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210ea2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cd12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210ea2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47492/0 (socket says 172.21.15.60:47492) 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210ea2b640 1 -- 172.21.15.60:0/1392087945 learned_addr learned my addr 172.21.15.60:0/1392087945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 0x55b112cd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 -- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 msgr2=0x55b112cd12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cd12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 -- 172.21.15.60:0/1392087945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 msgr2=0x55b112cd1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 -- 172.21.15.60:0/1392087945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b112bbf680 con 0x55b112cc3ff0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210ea2b640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f210da29640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 0x55b112cd1800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009230 tx=0x7f20f8031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe415a6070275855 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.310+0000 7f20f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8035040 con 0x55b112cc3ff0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21040aaa60 con 0x55b112cc3ff0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20f8004240 con 0x55b112cc3ff0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20f8004440 con 0x55b112cc3ff0 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 10 client.26162 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b11298a650 con 0x55b112cc3ff0 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21040aaa60 con 0x55b112cc3ff0 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f20f8004b30 con 0x55b112cc3ff0 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 10 client.26162 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 20 client.26162 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112ce3e40 0x55b112ce6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 4 client.26162 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b112c3c900 con 0x55b112ce3e40 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.311+0000 7f210e22a640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112ce3e40 0x55b112ce6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.312+0000 7f210e22a640 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112ce3e40 0x55b112ce6230 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ac4b02ae3014a8a server_cookie=f8ed5a2269bf86f2 in_seq=0 out_seq=0 2024-07-24T21:51:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.312+0000 7f20f1ffb640 10 client.26162 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:51:50.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f20f1ffb640 1 -- 172.21.15.60:0/1392087945 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b112c3c900 con 0x55b112ce3e40 2024-07-24T21:51:50.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f20f1ffb640 10 client.26162 handle_command_reply: tid=0 2024-07-24T21:51:50.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 client.26162 shutdown 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 msgr2=0x55b112cd1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 0x55b112cd1800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009230 tx=0x7f20f8031740 comp rx=0 tx=0).stop 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 shutdown_connections 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112ce3e40 0x55b112ce6230 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cd4390 0x55b112cd1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc3ff0 0x55b112cd1800 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f20f1ffb640 0 client.26162 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 --2- 172.21.15.60:0/1392087945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cd12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 >> 172.21.15.60:0/1392087945 conn(0x55b112cc1500 msgr2=0x55b112cc7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 shutdown_connections 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.313+0000 7f211283f740 1 -- 172.21.15.60:0/1392087945 wait complete. 2024-07-24T21:51:50.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.314+0000 7f211283f740 20 client.26162 trim_cache size 0 max 16384 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 Processor -- start 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 -- start start 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112ccee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cdf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ae810 con 0x55b112cc3ff0 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ac080 con 0x55b112cc36d0 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f211283f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21081a9ed0 con 0x55b112cd4390 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47494/0 (socket says 172.21.15.60:47494) 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 -- 172.21.15.60:0/866871348 learned_addr learned my addr 172.21.15.60:0/866871348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210e22a640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112ccee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210ea2b640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cdf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 -- 172.21.15.60:0/866871348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 msgr2=0x55b112cdf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cdf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 -- 172.21.15.60:0/866871348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 msgr2=0x55b112ccee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112ccee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210da29640 1 -- 172.21.15.60:0/866871348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b112cc3e20 con 0x55b112cc3ff0 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.328+0000 7f210ea2b640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cdf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f210da29640 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f20f8031440 tx=0x7f20f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105701613eeb01 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f20f3fff640 1 -- 172.21.15.60:0/866871348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8035070 con 0x55b112cc3ff0 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f20f3fff640 1 -- 172.21.15.60:0/866871348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f8004fd0 con 0x55b112cc3ff0 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f20f3fff640 1 -- 172.21.15.60:0/866871348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8003d90 con 0x55b112cc3ff0 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 -- 172.21.15.60:0/866871348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 msgr2=0x55b112cc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f20f8031440 tx=0x7f20f8000bc0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 -- 172.21.15.60:0/866871348 shutdown_connections 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cd4390 0x55b112cdf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cc0d60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 --2- 172.21.15.60:0/866871348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cc36d0 0x55b112ccee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 -- 172.21.15.60:0/866871348 >> 172.21.15.60:0/866871348 conn(0x55b112cc1500 msgr2=0x55b112cd3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.329+0000 7f211283f740 1 -- 172.21.15.60:0/866871348 shutdown_connections 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 -- 172.21.15.60:0/866871348 wait complete. 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 -- start start 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 Processor -- start 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cdc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 0x55b112cdc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21081aa540 con 0x55b112cc3ff0 2024-07-24T21:51:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21081ab9b0 con 0x55b112cd4390 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21081aac10 con 0x55b112cc36d0 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210da29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cdc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210ea2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 0x55b112cdc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f211283f740 10 client.0 _ll_register_callbacks cb 0x7f20f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210ea2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 0x55b112cdc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59676/0 (socket says 172.21.15.60:59676) 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210e22a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cebf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47016/0 (socket says 172.21.15.60:47016) 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.330+0000 7f210ea2b640 1 -- 172.21.15.60:0/3642072414 learned_addr learned my addr 172.21.15.60:0/3642072414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 -- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 msgr2=0x55b112cebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 -- 172.21.15.60:0/3642072414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 msgr2=0x55b112cdc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 0x55b112cdc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 -- 172.21.15.60:0/3642072414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b112cdcea0 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f210da29640 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cdc320 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009900 tx=0x7f20f80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae7d4130889b5cd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f20f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8035040 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20f8004970 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f8004fd0 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.331+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20f8066050 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 10 client.16707 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b112d4ead0 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f805c030 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f20f804f480 con 0x55b112cc3ff0 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 10 client.16707 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 10 client.16707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:51:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 20 client.16707 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112d4f150 0x55b112d51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 4 client.16707 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:51:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f20f8004fd0 con 0x55b112d4f150 2024-07-24T21:51:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.332+0000 7f210e22a640 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112d4f150 0x55b112d51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.333+0000 7f20f1ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:51:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.333+0000 7f210e22a640 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112d4f150 0x55b112d51570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ee372f8f19660f7 server_cookie=a87382fe4d6e9166 in_seq=0 out_seq=0 2024-07-24T21:51:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.335+0000 7f20f1ffb640 1 -- 172.21.15.60:0/3642072414 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f20f8004fd0 con 0x55b112d4f150 2024-07-24T21:51:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.335+0000 7f20f1ffb640 10 client.16707 handle_command_reply: tid=0 2024-07-24T21:51:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.335+0000 7f211283f740 1 client.16707 shutdown 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.335+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 msgr2=0x55b112cdc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.335+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cdc320 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009900 tx=0x7f20f80314a0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 shutdown_connections 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b112d4f150 0x55b112d51570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b112cd4390 0x55b112cdc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b112cc3ff0 0x55b112cdc320 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f20f1ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 --2- 172.21.15.60:0/3642072414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b112cc36d0 0x55b112cebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 >> 172.21.15.60:0/3642072414 conn(0x55b112cc1500 msgr2=0x55b112cdf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 shutdown_connections 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 1 -- 172.21.15.60:0/3642072414 wait complete. 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.336+0000 7f211283f740 20 client.16707 trim_cache size 0 max 16384 2024-07-24T21:51:50.342 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 msgr2=0x7f20f40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 0x7f20f40b6280 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2100005fd0 tx=0x7f2100009500 comp rx=0 tx=0).stop 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 msgr2=0x7f2108198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f210400e9f0 tx=0x7f210400edf0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 shutdown_connections 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20f40b3dc0 0x7f20f40b6280 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081079e0 0x7f210819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2108107060 0x7f21081991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 --2- 172.21.15.60:0/4170568097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2108101650 0x7f2108198c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 >> 172.21.15.60:0/4170568097 conn(0x7f21080fd0c0 msgr2=0x7f21080fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 shutdown_connections 2024-07-24T21:51:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:50.337+0000 7f211048e640 1 -- 172.21.15.60:0/4170568097 wait complete. 2024-07-24T21:51:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 -- 172.21.15.60:0/1066616854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 msgr2=0x7f01801019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 --2- 172.21.15.60:0/1066616854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f01801019c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0170009900 tx=0x7f017002a410 comp rx=0 tx=0).stop 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 -- 172.21.15.60:0/1066616854 shutdown_connections 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 --2- 172.21.15.60:0/1066616854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180107910 0x7f018010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 --2- 172.21.15.60:0/1066616854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180106f90 0x7f01801073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 --2- 172.21.15.60:0/1066616854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f01801019c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 -- 172.21.15.60:0/1066616854 >> 172.21.15.60:0/1066616854 conn(0x7f01800782e0 msgr2=0x7f01800ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 -- 172.21.15.60:0/1066616854 shutdown_connections 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.994+0000 7f0185257640 1 -- 172.21.15.60:0/1066616854 wait complete. 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.995+0000 7f0185257640 1 Processor -- start 2024-07-24T21:51:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.995+0000 7f0185257640 1 -- start start 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 0x7f0180198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 0x7f018019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0180111540 con 0x7f0180106f90 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47022/0 (socket says 172.21.15.60:47022) 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017ed9d640 1 -- 172.21.15.60:0/1067917787 learned_addr learned my addr 172.21.15.60:0/1067917787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01801113c0 con 0x7f0180107910 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0177fff640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 0x7f0180198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01801116c0 con 0x7f01801015e0 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 0x7f018019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 -- 172.21.15.60:0/1067917787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 msgr2=0x7f0180197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 -- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 msgr2=0x7f0180198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 0x7f0180198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017f59e640 1 -- 172.21.15.60:0/1067917787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018019dba0 con 0x7f0180107910 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.996+0000 7f0177fff640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 0x7f0180198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f017f59e640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 0x7f018019d3d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fd0 tx=0x7f017800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f017cd99640 1 -- 172.21.15.60:0/1067917787 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017800ce20 con 0x7f0180107910 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018019de90 con 0x7f0180107910 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f017cd99640 1 -- 172.21.15.60:0/1067917787 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0178005020 con 0x7f0180107910 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f017cd99640 1 -- 172.21.15.60:0/1067917787 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178010690 con 0x7f0180107910 2024-07-24T21:51:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.997+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01801a4550 con 0x7f0180107910 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:50.999+0000 7f017cd99640 1 -- 172.21.15.60:0/1067917787 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f01780054d0 con 0x7f0180107910 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 Processor -- start 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 -- start start 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968459ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96846a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f9683cf280 con 0x55f9684598c0 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9683be780 con 0x55f96845a1e0 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0187fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f9683c27c0 con 0x55f968468380 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0177fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968459ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0177fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968459ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47530/0 (socket says 172.21.15.60:47530) 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0177fff640 1 -- 172.21.15.60:0/1547933286 learned_addr learned my addr 172.21.15.60:0/1547933286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.000+0000 7f0177fff640 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96846a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 -- 172.21.15.60:0/1547933286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 msgr2=0x55f96846a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96846a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 -- 172.21.15.60:0/1547933286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 msgr2=0x55f968459ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968459ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 -- 172.21.15.60:0/1547933286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f968122790 con 0x55f96845a1e0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467e40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0170004770 tx=0x7f0170032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f5d3e0827c823e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f01757fa640 1 -- 172.21.15.60:0/1547933286 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170035040 con 0x55f96845a1e0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f01757fa640 1 -- 172.21.15.60:0/1547933286 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01700317a0 con 0x55f96845a1e0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017cd99640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 0x7f016c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f01757fa640 1 -- 172.21.15.60:0/1547933286 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700047a0 con 0x55f96845a1e0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f0177fff640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 0x7f016c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.001+0000 7f017cd99640 1 -- 172.21.15.60:0/1067917787 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f01780e46a0 con 0x7f0180107910 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0177fff640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 0x7f016c0b6200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0168013d30 tx=0x7f0168009b70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- 172.21.15.60:0/1547933286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 msgr2=0x55f968467e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467e40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0170004770 tx=0x7f0170032ed0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- 172.21.15.60:0/1547933286 shutdown_connections 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96846a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467e40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1547933286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968459ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- 172.21.15.60:0/1547933286 >> 172.21.15.60:0/1547933286 conn(0x55f9684576f0 msgr2=0x55f968458890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- 172.21.15.60:0/1547933286 shutdown_connections 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- 172.21.15.60:0/1547933286 wait complete. 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.002+0000 7f0187fb4740 1 -- start start 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 Processor -- start 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f9684655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f9682f8ab0 con 0x55f9684598c0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9683469e0 con 0x55f96845a1e0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f96846afa0 con 0x55f968468380 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0187fb4740 10 client.0 _ll_register_callbacks cb 0x55f9680cea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47536/0 (socket says 172.21.15.60:47536) 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 -- 172.21.15.60:0/4279479593 learned_addr learned my addr 172.21.15.60:0/4279479593 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f0177fff640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017f59e640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f9684655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 -- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 msgr2=0x55f9684655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f9684655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 -- 172.21.15.60:0/4279479593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 msgr2=0x55f968465090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968465090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 -- 172.21.15.60:0/4279479593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96819b620 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f017ed9d640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f01700057c0 tx=0x7f0170009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745f3de2604ca59 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.003+0000 7f01667fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017004f020 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0170046450 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f96836dce0 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f017005a050 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 10 client.16719 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f968476750 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f96836dce0 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f017003c850 con 0x55f9684598c0 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 10 client.16719 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 20 client.16719 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968477aa0 0x55f968479e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 4 client.16719 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.004+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9683d11d0 con 0x55f968477aa0 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.005+0000 7f0177fff640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968477aa0 0x55f968479e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.005+0000 7f0177fff640 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968477aa0 0x55f968479e90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab71748a1d710af0 server_cookie=a725006dbfece0ca in_seq=0 out_seq=0 2024-07-24T21:51:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.005+0000 7f01667fc640 10 client.16719 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f01667fc640 1 -- 172.21.15.60:0/4279479593 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9683d11d0 con 0x55f968477aa0 2024-07-24T21:51:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f01667fc640 10 client.16719 handle_command_reply: tid=0 2024-07-24T21:51:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 client.16719 shutdown 2024-07-24T21:51:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 msgr2=0x55f968467bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f01700057c0 tx=0x7f0170009260 comp rx=0 tx=0).stop 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 shutdown_connections 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968477aa0 0x55f968479e90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f9684655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 --2- 172.21.15.60:0/4279479593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f968467bb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f01667fc640 0 client.16719 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.007+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 >> 172.21.15.60:0/4279479593 conn(0x55f9684576f0 msgr2=0x55f96845d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.008+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 shutdown_connections 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.008+0000 7f0187fb4740 1 -- 172.21.15.60:0/4279479593 wait complete. 2024-07-24T21:51:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.008+0000 7f0187fb4740 20 client.16719 trim_cache size 0 max 16384 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.025+0000 7f0187fb4740 1 Processor -- start 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.025+0000 7f0187fb4740 1 -- start start 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9684598c0 0x55f968474080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f01801ae700 con 0x55f968468380 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01801abf70 con 0x55f96845a1e0 2024-07-24T21:51:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0187fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01801a9dc0 con 0x55f9684598c0 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0177fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47538/0 (socket says 172.21.15.60:47538) 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9684598c0 0x55f968474080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 -- 172.21.15.60:0/2972495584 learned_addr learned my addr 172.21.15.60:0/2972495584 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 -- 172.21.15.60:0/2972495584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9684598c0 msgr2=0x55f968474080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9684598c0 0x55f968474080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 -- 172.21.15.60:0/2972495584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 msgr2=0x55f968467a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 -- 172.21.15.60:0/2972495584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96845d9e0 con 0x55f968468380 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0177fff640 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f017f59e640 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f01780e1ea0 tx=0x7f01780aba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5b36f3f37f2c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.026+0000 7f0174ff9640 1 -- 172.21.15.60:0/2972495584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01780b4360 con 0x55f968468380 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0174ff9640 1 -- 172.21.15.60:0/2972495584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01780c5070 con 0x55f968468380 2024-07-24T21:51:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0174ff9640 1 -- 172.21.15.60:0/2972495584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01780b3ed0 con 0x55f968468380 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 -- 172.21.15.60:0/2972495584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 msgr2=0x55f9684638f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f01780e1ea0 tx=0x7f01780aba70 comp rx=0 tx=0).stop 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 -- 172.21.15.60:0/2972495584 shutdown_connections 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f968468380 0x55f9684638f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f968467a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 --2- 172.21.15.60:0/2972495584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9684598c0 0x55f968474080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 -- 172.21.15.60:0/2972495584 >> 172.21.15.60:0/2972495584 conn(0x55f9684576f0 msgr2=0x55f96846a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 -- 172.21.15.60:0/2972495584 shutdown_connections 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.027+0000 7f0187fb4740 1 -- 172.21.15.60:0/2972495584 wait complete. 2024-07-24T21:51:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 -- start start 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 Processor -- start 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f96847fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96847e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f01801aa430 con 0x55f9684598c0 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01801ab8a0 con 0x55f96845a1e0 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01801aab00 con 0x55f968468380 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0187fb4740 10 client.0 _ll_register_callbacks cb 0x7f0170040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f0177fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f96847fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47548/0 (socket says 172.21.15.60:47548) 2024-07-24T21:51:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017ed9d640 1 -- 172.21.15.60:0/1671132993 learned_addr learned my addr 172.21.15.60:0/1671132993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017f59e640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96847e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017ed9d640 1 -- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 msgr2=0x55f96847e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.028+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96847e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f017ed9d640 1 -- 172.21.15.60:0/1671132993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 msgr2=0x55f96847fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f96847fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f017ed9d640 1 -- 172.21.15.60:0/1671132993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96847ee30 con 0x55f9684598c0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f0177fff640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f96847fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f017ed9d640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0170009230 tx=0x7f0170000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c6ad80343a056f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f01667fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170063060 con 0x55f9684598c0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0170004440 con 0x55f9684598c0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01780c5070 con 0x55f9684598c0 2024-07-24T21:51:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f017003ce60 con 0x55f9684598c0 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f0187fb4740 10 client.16725 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f96846fe00 con 0x55f9684598c0 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.029+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01780c5070 con 0x55f9684598c0 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0170003bc0 con 0x55f9684598c0 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 10 client.16725 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 20 client.16725 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968470610 0x55f968470a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 4 client.16725 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f968458eb0 con 0x55f968470610 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0177fff640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968470610 0x55f968470a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f0177fff640 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968470610 0x55f968470a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55467e3338f63375 server_cookie=2a17bb264fa13975 in_seq=0 out_seq=0 2024-07-24T21:51:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:51.030+0000 7f01667fc640 10 client.16725 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:51.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:51:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:51 smithi060 ceph-mon[33000]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.32k op/s 2024-07-24T21:51:51.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:51 smithi123 ceph-mon[37143]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.32k op/s 2024-07-24T21:51:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:51 smithi086 ceph-mon[37128]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 22 MiB/s wr, 1.32k op/s 2024-07-24T21:51:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.352+0000 7f01667fc640 1 -- 172.21.15.60:0/1671132993 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f968458eb0 con 0x55f968470610 2024-07-24T21:51:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.352+0000 7f01667fc640 10 client.16725 handle_command_reply: tid=0 2024-07-24T21:51:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.352+0000 7f0187fb4740 1 client.16725 shutdown 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 msgr2=0x55f96847f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0170009230 tx=0x7f0170000bc0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 shutdown_connections 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f968470610 0x55f968470a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f968468380 0x55f96847e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f96845a1e0 0x55f96847fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 --2- 172.21.15.60:0/1671132993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f9684598c0 0x55f96847f820 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 >> 172.21.15.60:0/1671132993 conn(0x55f9684576f0 msgr2=0x55f968463870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f01667fc640 0 client.16725 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 shutdown_connections 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 1 -- 172.21.15.60:0/1671132993 wait complete. 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.353+0000 7f0187fb4740 20 client.16725 trim_cache size 0 max 16384 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:51:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:51:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.354+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 msgr2=0x7f016c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.354+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 0x7f016c0b6200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0168013d30 tx=0x7f0168009b70 comp rx=0 tx=0).stop 2024-07-24T21:51:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.354+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 msgr2=0x7f018019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.354+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 0x7f018019d3d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fd0 tx=0x7f017800ef10 comp rx=0 tx=0).stop 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 shutdown_connections 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f016c0b3d40 0x7f016c0b6200 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0180107910 0x7f018019d3d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0180106f90 0x7f0180198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 --2- 172.21.15.60:0/1067917787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01801015e0 0x7f0180197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 >> 172.21.15.60:0/1067917787 conn(0x7f01800782e0 msgr2=0x7f0180104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 shutdown_connections 2024-07-24T21:51:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:51:52.355+0000 7f0185257640 1 -- 172.21.15.60:0/1067917787 wait complete. 2024-07-24T21:51:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 -- 172.21.15.60:0/3945136540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 msgr2=0x7f219810c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 --2- 172.21.15.60:0/3945136540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f219810c470 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2180009900 tx=0x7f218002a410 comp rx=0 tx=0).stop 2024-07-24T21:51:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 -- 172.21.15.60:0/3945136540 shutdown_connections 2024-07-24T21:51:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 --2- 172.21.15.60:0/3945136540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219810cac0 0x7f219810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 --2- 172.21.15.60:0/3945136540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f219810c470 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 --2- 172.21.15.60:0/3945136540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2198103660 0x7f2198103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 -- 172.21.15.60:0/3945136540 >> 172.21.15.60:0/3945136540 conn(0x7f21980fd0c0 msgr2=0x7f21980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 -- 172.21.15.60:0/3945136540 shutdown_connections 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.394+0000 7f219d1c1640 1 -- 172.21.15.60:0/3945136540 wait complete. 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 Processor -- start 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 -- start start 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f2198199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219810cac0 0x7f219819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2198112710 con 0x7f2198103f80 2024-07-24T21:51:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2198112590 con 0x7f2198103660 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219d1c1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2198112890 con 0x7f219810cac0 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f2198199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f2198199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47566/0 (socket says 172.21.15.60:47566) 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f2196d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59754/0 (socket says 172.21.15.60:59754) 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219659c640 1 -- 172.21.15.60:0/2931864966 learned_addr learned my addr 172.21.15.60:0/2931864966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.395+0000 7f219759e640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219810cac0 0x7f219819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 -- 172.21.15.60:0/2931864966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219810cac0 msgr2=0x7f219819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219810cac0 0x7f219819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 -- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 msgr2=0x7f2198199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f2198199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 -- 172.21.15.60:0/2931864966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219819ed90 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f2196d9d640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f218800e9f0 tx=0x7f218800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f218ffff640 1 -- 172.21.15.60:0/2931864966 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218800ccc0 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219819f080 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f218ffff640 1 -- 172.21.15.60:0/2931864966 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2188005020 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f218ffff640 1 -- 172.21.15.60:0/2931864966 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188004a00 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.396+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21981011f0 con 0x7f2198103660 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.399+0000 7f219f572740 1 Processor -- start 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 -- start start 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a391ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a391e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5568a38869b0 con 0x5568a390f6d0 2024-07-24T21:51:53.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568a3875eb0 con 0x5568a391bf80 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219f572740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568a3879ef0 con 0x5568a390fff0 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f218ffff640 1 -- 172.21.15.60:0/2931864966 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2188004ba0 con 0x7f2198103660 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47576/0 (socket says 172.21.15.60:47576) 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219759e640 1 -- 172.21.15.60:0/798431185 learned_addr learned my addr 172.21.15.60:0/798431185 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f219659c640 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a391e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.400+0000 7f2196d9d640 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a391ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 -- 172.21.15.60:0/798431185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 msgr2=0x5568a391ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a391ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 -- 172.21.15.60:0/798431185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 msgr2=0x5568a391e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a391e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 -- 172.21.15.60:0/798431185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a37ff320 con 0x5568a390f6d0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f219759e640 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2190009fd0 tx=0x7f21900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b736b6b8755403f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f218d7fa640 1 -- 172.21.15.60:0/798431185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190004d20 con 0x5568a390f6d0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f218d7fa640 1 -- 172.21.15.60:0/798431185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190004050 con 0x5568a390f6d0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f218d7fa640 1 -- 172.21.15.60:0/798431185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190005280 con 0x5568a390f6d0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f218ffff640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 0x7f21840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.401+0000 7f2196d9d640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 0x7f21840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f218ffff640 1 -- 172.21.15.60:0/2931864966 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f21880e3ff0 con 0x7f2198103660 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f2196d9d640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 0x7f21840b6200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f21880e17f0 tx=0x7f2188005460 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 -- 172.21.15.60:0/798431185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 msgr2=0x5568a390fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2190009fd0 tx=0x7f21900118d0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 -- 172.21.15.60:0/798431185 shutdown_connections 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a391e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a391ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 --2- 172.21.15.60:0/798431185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390f6d0 0x5568a390fab0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 -- 172.21.15.60:0/798431185 >> 172.21.15.60:0/798431185 conn(0x5568a390d500 msgr2=0x5568a390e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 -- 172.21.15.60:0/798431185 shutdown_connections 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.402+0000 7f219f572740 1 -- 172.21.15.60:0/798431185 wait complete. 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 -- start start 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 Processor -- start 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 0x5568a3918ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3919970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5568a36d9700 con 0x5568a391bf80 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568a38185f0 con 0x5568a390f6d0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568a391ec00 con 0x5568a390fff0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219f572740 10 client.0 _ll_register_callbacks cb 0x5568a3838960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 0x5568a3918ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 0x5568a3918ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59776/0 (socket says 172.21.15.60:59776) 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219759e640 1 -- 172.21.15.60:0/507738066 learned_addr learned my addr 172.21.15.60:0/507738066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47094/0 (socket says 172.21.15.60:47094) 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.403+0000 7f219759e640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3919970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 -- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 msgr2=0x5568a3918ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 0x5568a3918ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 -- 172.21.15.60:0/507738066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 msgr2=0x5568a3919970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3919970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 -- 172.21.15.60:0/507738066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a380b680 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219659c640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2180004770 tx=0x7f2180031e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e935ebf8019fe2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f217e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180035040 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2190004050 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f2180031760 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2180004040 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219f572740 10 client.26209 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.404+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5568a35d6650 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190004050 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f218003d450 con 0x5568a390fff0 2024-07-24T21:51:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 10 client.26209 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 10 client.26209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 20 client.26209 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a392ba60 0x5568a392de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 4 client.26209 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568a3888900 con 0x5568a392ba60 2024-07-24T21:51:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219759e640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a392ba60 0x5568a392de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f219759e640 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a392ba60 0x5568a392de80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53f2b5aff722f195 server_cookie=44716ec2d478d886 in_seq=0 out_seq=0 2024-07-24T21:51:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.405+0000 7f217e7fc640 10 client.26209 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:51:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f217e7fc640 1 -- 172.21.15.60:0/507738066 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568a3888900 con 0x5568a392ba60 2024-07-24T21:51:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f217e7fc640 10 client.26209 handle_command_reply: tid=0 2024-07-24T21:51:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 client.26209 shutdown 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 msgr2=0x5568a3919430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2180004770 tx=0x7f2180031e80 comp rx=0 tx=0).stop 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 shutdown_connections 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a392ba60 0x5568a392de80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3919970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390fff0 0x5568a3919430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 --2- 172.21.15.60:0/507738066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390f6d0 0x5568a3918ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f217e7fc640 0 client.26209 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 >> 172.21.15.60:0/507738066 conn(0x5568a390d500 msgr2=0x5568a39130a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 shutdown_connections 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.406+0000 7f219f572740 1 -- 172.21.15.60:0/507738066 wait complete. 2024-07-24T21:51:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.407+0000 7f219f572740 20 client.26209 trim_cache size 0 max 16384 2024-07-24T21:51:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 Processor -- start 2024-07-24T21:51:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 -- start start 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a39132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390fff0 0x5568a39337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21981b2d50 con 0x5568a391bf80 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21981b05c0 con 0x5568a390fff0 2024-07-24T21:51:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219f572740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21981ae410 con 0x5568a390f6d0 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47598/0 (socket says 172.21.15.60:47598) 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219759e640 1 -- 172.21.15.60:0/3042943770 learned_addr learned my addr 172.21.15.60:0/3042943770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219759e640 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a39132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.423+0000 7f219659c640 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390fff0 0x5568a39337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 -- 172.21.15.60:0/3042943770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 msgr2=0x5568a39132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a39132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 -- 172.21.15.60:0/3042943770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390fff0 msgr2=0x5568a39337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390fff0 0x5568a39337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 -- 172.21.15.60:0/3042943770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a391b450 con 0x5568a391bf80 2024-07-24T21:51:53.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f219759e640 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2190009320 tx=0x7f2190000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ddcb69e7361cd3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f218cff9640 1 -- 172.21.15.60:0/3042943770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190004a00 con 0x5568a391bf80 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f218cff9640 1 -- 172.21.15.60:0/3042943770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190004490 con 0x5568a391bf80 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.424+0000 7f218cff9640 1 -- 172.21.15.60:0/3042943770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190005040 con 0x5568a391bf80 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- 172.21.15.60:0/3042943770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 msgr2=0x5568a3933ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2190009320 tx=0x7f2190000f40 comp rx=0 tx=0).stop 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- 172.21.15.60:0/3042943770 shutdown_connections 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a391bf80 0x5568a3933ce0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a390fff0 0x5568a39337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 --2- 172.21.15.60:0/3042943770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a39132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- 172.21.15.60:0/3042943770 >> 172.21.15.60:0/3042943770 conn(0x5568a390d500 msgr2=0x5568a391ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- 172.21.15.60:0/3042943770 shutdown_connections 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- 172.21.15.60:0/3042943770 wait complete. 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 -- start start 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.425+0000 7f219f572740 1 Processor -- start 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390fff0 0x5568a39203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a3920910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21981aea80 con 0x5568a390fff0 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21981afef0 con 0x5568a391bf80 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21981af150 con 0x5568a390f6d0 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219f572740 10 client.0 _ll_register_callbacks cb 0x7f2188005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47120/0 (socket says 172.21.15.60:47120) 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 -- 172.21.15.60:0/1042395269 learned_addr learned my addr 172.21.15.60:0/1042395269 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a3920910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219659c640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390fff0 0x5568a39203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 -- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 msgr2=0x5568a3920910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a3920910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 -- 172.21.15.60:0/1042395269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390fff0 msgr2=0x5568a39203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390fff0 0x5568a39203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.426+0000 7f219759e640 1 -- 172.21.15.60:0/1042395269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a3995970 con 0x5568a390f6d0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f219759e640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2190002230 tx=0x7f21900096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1db3519159e1cf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f217e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190004c00 con 0x5568a390f6d0 2024-07-24T21:51:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f2190003f70 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2190004490 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2190004170 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f219f572740 10 client.26215 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5568a3996c50 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.427+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190004490 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2190034040 con 0x5568a390f6d0 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 10 client.26215 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 10 client.26215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:51:53.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 20 client.26215 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:53.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a3997130 0x5568a3999550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:53.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 4 client.26215 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:51:53.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568a38843a0 con 0x5568a3997130 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219659c640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a3997130 0x5568a3999550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f219659c640 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a3997130 0x5568a3999550 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c1526ccb0f8df30 server_cookie=cc1b43c4f395dba in_seq=0 out_seq=0 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.428+0000 7f217e7fc640 10 client.26215 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f217e7fc640 1 -- 172.21.15.60:0/1042395269 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568a38843a0 con 0x5568a3997130 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f217e7fc640 10 client.26215 handle_command_reply: tid=0 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 client.26215 shutdown 2024-07-24T21:51:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 msgr2=0x5568a391fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2190002230 tx=0x7f21900096c0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 shutdown_connections 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5568a3997130 0x5568a3999550 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568a391bf80 0x5568a3920910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5568a390fff0 0x5568a39203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 --2- 172.21.15.60:0/1042395269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568a390f6d0 0x5568a391fe90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f217e7fc640 0 client.26215 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 >> 172.21.15.60:0/1042395269 conn(0x5568a390d500 msgr2=0x5568a3917770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 shutdown_connections 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 1 -- 172.21.15.60:0/1042395269 wait complete. 2024-07-24T21:51:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.430+0000 7f219f572740 20 client.26215 trim_cache size 0 max 16384 2024-07-24T21:51:53.437 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:51:53.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 msgr2=0x7f21840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 0x7f21840b6200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f21880e17f0 tx=0x7f2188005460 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 msgr2=0x7f2198198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f218800e9f0 tx=0x7f218800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 shutdown_connections 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21840b3d40 0x7f21840b6200 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219810cac0 0x7f219819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2198103f80 0x7f2198199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 --2- 172.21.15.60:0/2931864966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2198103660 0x7f2198198ce0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 >> 172.21.15.60:0/2931864966 conn(0x7f21980fd0c0 msgr2=0x7f21980fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 shutdown_connections 2024-07-24T21:51:53.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:53.431+0000 7f219d1c1640 1 -- 172.21.15.60:0/2931864966 wait complete. 2024-07-24T21:51:53.442 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:53 smithi060 ceph-mon[33000]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:53.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:53 smithi123 ceph-mon[37143]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:53 smithi086 ceph-mon[37128]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-07-24T21:51:54.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:51:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:54.712 INFO:tasks.workunit.client.0.smithi060.stdout: 385 1240560 1410 877281 1386 335747 1627 2024-07-24T21:51:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:55 smithi123 ceph-mon[37143]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-07-24T21:51:55.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:55 smithi060 ceph-mon[33000]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-07-24T21:51:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:55 smithi086 ceph-mon[37128]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-07-24T21:51:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 -- 172.21.15.60:0/2031424997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c1025e0 msgr2=0x7f505c1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 --2- 172.21.15.60:0/2031424997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c1025e0 0x7f505c1029c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5050009900 tx=0x7f505002a410 comp rx=0 tx=0).stop 2024-07-24T21:51:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 -- 172.21.15.60:0/2031424997 shutdown_connections 2024-07-24T21:51:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 --2- 172.21.15.60:0/2031424997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c108970 0x7f505c10f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 --2- 172.21.15.60:0/2031424997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 --2- 172.21.15.60:0/2031424997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c1025e0 0x7f505c1029c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 -- 172.21.15.60:0/2031424997 >> 172.21.15.60:0/2031424997 conn(0x7f505c0fe010 msgr2=0x7f505c100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 -- 172.21.15.60:0/2031424997 shutdown_connections 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.366+0000 7f5061ba1640 1 -- 172.21.15.60:0/2031424997 wait complete. 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 Processor -- start 2024-07-24T21:51:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 -- start start 2024-07-24T21:51:56.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 0x7f505c19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f505c110470 con 0x7f505c107ff0 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1102f0 con 0x7f505c1025e0 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5061ba1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1105f0 con 0x7f505c108970 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f505bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f505bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49338/0 (socket says 172.21.15.60:49338) 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:43602/0 (socket says 172.21.15.60:43602) 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f50613a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 0x7f505c19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.367+0000 7f505bfff640 1 -- 172.21.15.60:0/3998654855 learned_addr learned my addr 172.21.15.60:0/3998654855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 -- 172.21.15.60:0/3998654855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 msgr2=0x7f505c196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 -- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 msgr2=0x7f505c196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 -- 172.21.15.60:0/3998654855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c19c8d0 con 0x7f505c108970 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f5060b9f640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f50613a0640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 0x7f505c19c100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5054012f90 tx=0x7f5054010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f5059ffb640 1 -- 172.21.15.60:0/3998654855 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054010df0 con 0x7f505c108970 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f5059ffb640 1 -- 172.21.15.60:0/3998654855 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f505400d0d0 con 0x7f505c108970 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505c19cb60 con 0x7f505c108970 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.368+0000 7f5059ffb640 1 -- 172.21.15.60:0/3998654855 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505400bd20 con 0x7f505c108970 2024-07-24T21:51:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.369+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505c1a3280 con 0x7f505c108970 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 Processor -- start 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f5059ffb640 1 -- 172.21.15.60:0/3998654855 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f505401f020 con 0x7f505c108970 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 -- start start 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da4a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55995d9b29b0 con 0x55995da3c180 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55995d9a1eb0 con 0x55995da48110 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.371+0000 7f50651b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55995d9a5ef0 con 0x55995da3b860 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54384/0 (socket says 172.21.15.60:54384) 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 -- 172.21.15.60:0/1309945735 learned_addr learned my addr 172.21.15.60:0/1309945735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49348/0 (socket says 172.21.15.60:49348) 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da4a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 -- 172.21.15.60:0/1309945735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 msgr2=0x55995da3bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 -- 172.21.15.60:0/1309945735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 msgr2=0x55995da4a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da4a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 -- 172.21.15.60:0/1309945735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995d92b320 con 0x55995da3c180 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5059ffb640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 0x7f504c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f505002aa00 tx=0x7f50500043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c76cabcc4382e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.372+0000 7f505bfff640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 0x7f504c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f5046ffd640 1 -- 172.21.15.60:0/1309945735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050035040 con 0x55995da3c180 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f5046ffd640 1 -- 172.21.15.60:0/1309945735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5050004770 con 0x55995da3c180 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f5046ffd640 1 -- 172.21.15.60:0/1309945735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050003920 con 0x55995da3c180 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f5059ffb640 1 -- 172.21.15.60:0/3998654855 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f50540e7c90 con 0x7f505c108970 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 -- 172.21.15.60:0/1309945735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 msgr2=0x55995da47bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f505bfff640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 0x7f504c0b6320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f50480095f0 tx=0x7f5048005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f505002aa00 tx=0x7f50500043f0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 -- 172.21.15.60:0/1309945735 shutdown_connections 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da4a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da47bd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 --2- 172.21.15.60:0/1309945735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da3bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 -- 172.21.15.60:0/1309945735 >> 172.21.15.60:0/1309945735 conn(0x55995da39690 msgr2=0x55995da3a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.373+0000 7f50651b4740 1 -- 172.21.15.60:0/1309945735 shutdown_connections 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.374+0000 7f50651b4740 1 -- 172.21.15.60:0/1309945735 wait complete. 2024-07-24T21:51:56.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.374+0000 7f50651b4740 1 -- start start 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.374+0000 7f50651b4740 1 Processor -- start 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da45b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55995d805700 con 0x55995da3c180 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55995d9445f0 con 0x55995da48110 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55995da4ad30 con 0x55995da3b860 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50651b4740 10 client.0 _ll_register_callbacks cb 0x55995d964960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f505bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54396/0 (socket says 172.21.15.60:54396) 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 -- 172.21.15.60:0/1329405523 learned_addr learned my addr 172.21.15.60:0/1329405523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50613a0640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da45b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 -- 172.21.15.60:0/1329405523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 msgr2=0x55995da45b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da45b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 -- 172.21.15.60:0/1329405523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 msgr2=0x55995da455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f5060b9f640 1 -- 172.21.15.60:0/1329405523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995d937680 con 0x55995da3b860 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f50613a0640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da45b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.375+0000 7f505bfff640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f5060b9f640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5050005ab0 tx=0x7f5050005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742ed3e80bfab87b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050035040 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5050004770 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f5050004b20 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f505005a050 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f50651b4740 10 client.26227 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55995d702650 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5050004770 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5050055030 con 0x55995da3b860 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f50651b4740 10 client.26227 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.376+0000 7f50651b4740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f50651b4740 20 client.26227 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995da57ba0 0x55995da5a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f50651b4740 4 client.26227 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:51:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55995d9b4900 con 0x55995da57ba0 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f505bfff640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995da57ba0 0x55995da5a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f505bfff640 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995da57ba0 0x55995da5a060 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a992efadb521cbc8 server_cookie=e88b0b29deeb78cf in_seq=0 out_seq=0 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.377+0000 7f503bfff640 10 client.26227 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f503bfff640 1 -- 172.21.15.60:0/1329405523 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55995d9b4900 con 0x55995da57ba0 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f503bfff640 10 client.26227 handle_command_reply: tid=0 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 client.26227 shutdown 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 msgr2=0x55995da45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5050005ab0 tx=0x7f5050005900 comp rx=0 tx=0).stop 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 shutdown_connections 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995da57ba0 0x55995da5a060 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995da45b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3c180 0x55995da455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 --2- 172.21.15.60:0/1329405523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da45080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 >> 172.21.15.60:0/1329405523 conn(0x55995da39690 msgr2=0x55995da3f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.378+0000 7f503bfff640 0 client.26227 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.379+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 shutdown_connections 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.379+0000 7f50651b4740 1 -- 172.21.15.60:0/1329405523 wait complete. 2024-07-24T21:51:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.379+0000 7f50651b4740 20 client.26227 trim_cache size 0 max 16384 2024-07-24T21:51:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 Processor -- start 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 -- start start 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 0x55995da46dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1ae5f0 con 0x55995da48110 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1abe60 con 0x55995da3c180 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50651b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1a9cb0 con 0x55995da3b860 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50613a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54404/0 (socket says 172.21.15.60:54404) 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f50613a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49374/0 (socket says 172.21.15.60:49374) 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 -- 172.21.15.60:0/4200042388 learned_addr learned my addr 172.21.15.60:0/4200042388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f505bfff640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 0x55995da46dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 -- 172.21.15.60:0/4200042388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 msgr2=0x55995da46dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 0x55995da46dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 -- 172.21.15.60:0/4200042388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 msgr2=0x55995da5f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.396+0000 7f5060b9f640 1 -- 172.21.15.60:0/4200042388 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995da43fe0 con 0x55995da3b860 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f50613a0640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f505bfff640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 0x55995da46dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f5060b9f640 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5050001eb0 tx=0x7f5050005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfab753e24a5f3a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f50467fc640 1 -- 172.21.15.60:0/4200042388 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050063070 con 0x55995da3b860 2024-07-24T21:51:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f50467fc640 1 -- 172.21.15.60:0/4200042388 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5050003ca0 con 0x55995da3b860 2024-07-24T21:51:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.397+0000 7f50467fc640 1 -- 172.21.15.60:0/4200042388 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050035040 con 0x55995da3b860 2024-07-24T21:51:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- 172.21.15.60:0/4200042388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 msgr2=0x55995da34890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5050001eb0 tx=0x7f5050005a20 comp rx=0 tx=0).stop 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- 172.21.15.60:0/4200042388 shutdown_connections 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da48110 0x55995da5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da3c180 0x55995da46dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 --2- 172.21.15.60:0/4200042388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3b860 0x55995da34890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- 172.21.15.60:0/4200042388 >> 172.21.15.60:0/4200042388 conn(0x55995da39690 msgr2=0x55995da466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- 172.21.15.60:0/4200042388 shutdown_connections 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- 172.21.15.60:0/4200042388 wait complete. 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 -- start start 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.398+0000 7f50651b4740 1 Processor -- start 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3c180 0x55995da4bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995dac0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1aa320 con 0x55995da3b860 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1ab790 con 0x55995da48110 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1aa9f0 con 0x55995da3c180 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50651b4740 10 client.0 _ll_register_callbacks cb 0x7f5048011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49390/0 (socket says 172.21.15.60:49390) 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 -- 172.21.15.60:0/3157806946 learned_addr learned my addr 172.21.15.60:0/3157806946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f505bfff640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3c180 0x55995da4bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f50613a0640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995dac0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 -- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3c180 msgr2=0x55995da4bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3c180 0x55995da4bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 -- 172.21.15.60:0/3157806946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 msgr2=0x55995dac0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:51:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995dac0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.399+0000 7f5060b9f640 1 -- 172.21.15.60:0/3157806946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995dac1070 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f50613a0640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995dac0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f5060b9f640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f50500057c0 tx=0x7f5050005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64263c3f03e3562f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f503bfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050035070 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f50500048e0 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5050003ca0 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5050063040 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f50651b4740 10 client.16761 fetch_fsmap learned FSMap version 31 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.400+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55995dac2370 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5050003ca0 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5050066030 con 0x55995da3b860 2024-07-24T21:51:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 10 client.16761 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 20 client.16761 populate_metadata read hostname 'smithi060' 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995dac2a60 0x55995dac4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 4 client.16761 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55995da448e0 con 0x55995dac2a60 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.401+0000 7f505bfff640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995dac2a60 0x55995dac4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f505bfff640 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995dac2a60 0x55995dac4e80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330589cc151df09a server_cookie=880ade72e2fb7d55 in_seq=0 out_seq=0 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f503bfff640 10 client.16761 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f503bfff640 1 -- 172.21.15.60:0/3157806946 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55995da448e0 con 0x55995dac2a60 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f503bfff640 10 client.16761 handle_command_reply: tid=0 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 client.16761 shutdown 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 msgr2=0x55995da4b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f50500057c0 tx=0x7f5050005d90 comp rx=0 tx=0).stop 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 shutdown_connections 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55995dac2a60 0x55995dac4e80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995da48110 0x55995dac0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55995da3c180 0x55995da4bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 --2- 172.21.15.60:0/3157806946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55995da3b860 0x55995da4b750 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 >> 172.21.15.60:0/3157806946 conn(0x55995da39690 msgr2=0x55995da466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f503bfff640 0 client.16761 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:51:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 shutdown_connections 2024-07-24T21:51:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 1 -- 172.21.15.60:0/3157806946 wait complete. 2024-07-24T21:51:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.402+0000 7f50651b4740 20 client.16761 trim_cache size 0 max 16384 2024-07-24T21:51:56.409 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 msgr2=0x7f504c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 0x7f504c0b6320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f50480095f0 tx=0x7f5048005f90 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 msgr2=0x7f505c19c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 0x7f505c19c100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5054012f90 tx=0x7f5054010490 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 shutdown_connections 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f504c0b3e60 0x7f504c0b6320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.403+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f505c108970 0x7f505c19c100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.404+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c107ff0 0x7f505c196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.404+0000 7f5061ba1640 1 --2- 172.21.15.60:0/3998654855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505c1025e0 0x7f505c196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:51:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.404+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 >> 172.21.15.60:0/3998654855 conn(0x7f505c0fe010 msgr2=0x7f505c100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:51:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.404+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 shutdown_connections 2024-07-24T21:51:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:51:56.404+0000 7f5061ba1640 1 -- 172.21.15.60:0/3998654855 wait complete. 2024-07-24T21:51:57.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:57 smithi123 ceph-mon[37143]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-07-24T21:51:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:57 smithi086 ceph-mon[37128]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-07-24T21:51:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:57 smithi060 ceph-mon[33000]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-07-24T21:51:58.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:51:58.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:51:58.516 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:51:59.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:51:59 smithi060 ceph-mon[33000]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 1.01k op/s 2024-07-24T21:51:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:51:59 smithi123 ceph-mon[37143]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 1.01k op/s 2024-07-24T21:51:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:51:59 smithi086 ceph-mon[37128]: pgmap v536: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 1.01k op/s 2024-07-24T21:52:01.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:01 smithi060 ceph-mon[33000]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 786 op/s 2024-07-24T21:52:01.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:01 smithi123 ceph-mon[37143]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 786 op/s 2024-07-24T21:52:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:01 smithi086 ceph-mon[37128]: pgmap v537: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 786 op/s 2024-07-24T21:52:02.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.087+0000 7fd881655640 1 -- 172.21.15.60:0/1508624121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c109b40 msgr2=0x7fd87c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.087+0000 7fd881655640 1 --2- 172.21.15.60:0/1508624121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c109b40 0x7fd87c110ba0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd874009be0 tx=0x7fd87402a6f0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.088+0000 7fd881655640 1 -- 172.21.15.60:0/1508624121 shutdown_connections 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.088+0000 7fd881655640 1 --2- 172.21.15.60:0/1508624121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c11ba90 0x7fd87c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.088+0000 7fd881655640 1 --2- 172.21.15.60:0/1508624121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd87c109f20 0x7fd87c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.088+0000 7fd881655640 1 --2- 172.21.15.60:0/1508624121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c109b40 0x7fd87c110ba0 secure :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd874009be0 tx=0x7fd87402a6f0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.088+0000 7fd881655640 1 -- 172.21.15.60:0/1508624121 >> 172.21.15.60:0/1508624121 conn(0x7fd87c06c4f0 msgr2=0x7fd87c06c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.090+0000 7fd881655640 1 -- 172.21.15.60:0/1508624121 shutdown_connections 2024-07-24T21:52:02.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.090+0000 7fd881655640 1 -- 172.21.15.60:0/1508624121 wait complete. 2024-07-24T21:52:02.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.091+0000 7fd881655640 1 Processor -- start 2024-07-24T21:52:02.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.095+0000 7fd881655640 1 -- start start 2024-07-24T21:52:02.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd881655640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd87c109f20 0x7fd87c1a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd881655640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49408/0 (socket says 172.21.15.60:49408) 2024-07-24T21:52:02.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd87b7fe640 1 -- 172.21.15.60:0/3715504353 learned_addr learned my addr 172.21.15.60:0/3715504353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:02.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.096+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c1ad3c0 0x7fd87c1ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.097+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c121440 con 0x7fd87c11ba90 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.097+0000 7fd880e54640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c1ad3c0 0x7fd87c1ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.097+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1212c0 con 0x7fd87c1ad3c0 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.097+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1215c0 con 0x7fd87c109f20 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 -- 172.21.15.60:0/3715504353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd87c109f20 msgr2=0x7fd87c1a7ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd87c109f20 0x7fd87c1a7ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 -- 172.21.15.60:0/3715504353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c1ad3c0 msgr2=0x7fd87c1ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c1ad3c0 0x7fd87c1ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1adf50 con 0x7fd87c11ba90 2024-07-24T21:52:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.098+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9f0 tx=0x7fd87000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.099+0000 7fd8797fa640 1 -- 172.21.15.60:0/3715504353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000cc90 con 0x7fd87c11ba90 2024-07-24T21:52:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.099+0000 7fd8797fa640 1 -- 172.21.15.60:0/3715504353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd87000ce30 con 0x7fd87c11ba90 2024-07-24T21:52:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.099+0000 7fd8797fa640 1 -- 172.21.15.60:0/3715504353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870010430 con 0x7fd87c11ba90 2024-07-24T21:52:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.101+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1b4540 con 0x7fd87c11ba90 2024-07-24T21:52:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.106+0000 7fd8797fa640 1 -- 172.21.15.60:0/3715504353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd8700105d0 con 0x7fd87c11ba90 2024-07-24T21:52:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.107+0000 7fd8797fa640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 0x7fd8640b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.108+0000 7fd87bfff640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 0x7fd8640b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.108+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1b48e0 con 0x7fd87c11ba90 2024-07-24T21:52:02.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.112+0000 7fd8797fa640 1 -- 172.21.15.60:0/3715504353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd8700a7ca0 con 0x7fd87c11ba90 2024-07-24T21:52:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.113+0000 7fd87bfff640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 0x7fd8640b6450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd874009b30 tx=0x7fd874005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.117+0000 7fd884c68740 1 Processor -- start 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.124+0000 7fd884c68740 1 -- start start 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.124+0000 7fd884c68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.124+0000 7fd884c68740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa647470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.124+0000 7fd884c68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa649da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.124+0000 7fd884c68740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562ffa5b29f0 con 0x562ffa63ba20 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd884c68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ffa5a1ef0 con 0x562ffa6479b0 2024-07-24T21:52:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd884c68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ffa5a5f30 con 0x562ffa63b100 2024-07-24T21:52:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54458/0 (socket says 172.21.15.60:54458) 2024-07-24T21:52:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd87bfff640 1 -- 172.21.15.60:0/282842240 learned_addr learned my addr 172.21.15.60:0/282842240 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa647470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.125+0000 7fd880e54640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa649da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.129+0000 7fd87bfff640 1 -- 172.21.15.60:0/282842240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 msgr2=0x562ffa649da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.129+0000 7fd87bfff640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa649da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.129+0000 7fd87bfff640 1 -- 172.21.15.60:0/282842240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 msgr2=0x562ffa647470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.129+0000 7fd87bfff640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa647470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.129+0000 7fd87bfff640 1 -- 172.21.15.60:0/282842240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffa302930 con 0x562ffa63b100 2024-07-24T21:52:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.133+0000 7fd87bfff640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd874033000 tx=0x7fd87403a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66899de70f968b32 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.133+0000 7fd880e54640 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa649da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:52:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.133+0000 7fd8627fc640 1 -- 172.21.15.60:0/282842240 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874036040 con 0x562ffa63b100 2024-07-24T21:52:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.133+0000 7fd8627fc640 1 -- 172.21.15.60:0/282842240 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874041070 con 0x562ffa63b100 2024-07-24T21:52:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.134+0000 7fd884c68740 1 -- 172.21.15.60:0/282842240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 msgr2=0x562ffa63b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.134+0000 7fd884c68740 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd874033000 tx=0x7fd87403a000 comp rx=0 tx=0).stop 2024-07-24T21:52:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.136+0000 7fd884c68740 1 -- 172.21.15.60:0/282842240 shutdown_connections 2024-07-24T21:52:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.136+0000 7fd884c68740 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa649da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.136+0000 7fd884c68740 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa647470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.136+0000 7fd884c68740 1 --2- 172.21.15.60:0/282842240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63b100 0x562ffa63b4e0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd874033000 tx=0x7fd87403a000 comp rx=0 tx=0).stop 2024-07-24T21:52:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.136+0000 7fd884c68740 1 -- 172.21.15.60:0/282842240 >> 172.21.15.60:0/282842240 conn(0x562ffa638f30 msgr2=0x562ffa63a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.139+0000 7fd884c68740 1 -- 172.21.15.60:0/282842240 shutdown_connections 2024-07-24T21:52:02.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.144+0000 7fd884c68740 1 -- 172.21.15.60:0/282842240 wait complete. 2024-07-24T21:52:02.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.145+0000 7fd884c68740 1 -- start start 2024-07-24T21:52:02.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.145+0000 7fd884c68740 1 Processor -- start 2024-07-24T21:52:02.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.149+0000 7fd884c68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63ba20 0x562ffa644a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.149+0000 7fd884c68740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa644f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd884c68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63ba20 0x562ffa644a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd884c68740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562ffa405890 con 0x562ffa6479b0 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd884c68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ffa544390 con 0x562ffa6454c0 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd880e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd880e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:43688/0 (socket says 172.21.15.60:43688) 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd880e54640 1 -- 172.21.15.60:0/279086110 learned_addr learned my addr 172.21.15.60:0/279086110 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa644f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ffa64a670 con 0x562ffa63ba20 2024-07-24T21:52:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.150+0000 7fd884c68740 10 client.0 _ll_register_callbacks cb 0x562ffa54e500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:02.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.153+0000 7fd880e54640 1 -- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63ba20 msgr2=0x562ffa644a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.153+0000 7fd880e54640 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63ba20 0x562ffa644a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.153+0000 7fd880e54640 1 -- 172.21.15.60:0/279086110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 msgr2=0x562ffa644f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.153+0000 7fd880e54640 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa644f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.153+0000 7fd880e54640 1 -- 172.21.15.60:0/279086110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffa53ed60 con 0x562ffa6454c0 2024-07-24T21:52:02.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.154+0000 7fd880e54640 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd86c011730 tx=0x7fd86c011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57155c4f1aa67d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.156+0000 7fd85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.161+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c01e060 con 0x562ffa6454c0 2024-07-24T21:52:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.161+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd86c00fdb0 con 0x562ffa6454c0 2024-07-24T21:52:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.162+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd874041070 con 0x562ffa6454c0 2024-07-24T21:52:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.168+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd86c0193f0 con 0x562ffa6454c0 2024-07-24T21:52:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.171+0000 7fd884c68740 10 client.26231 fetch_fsmap learned FSMap version 31 2024-07-24T21:52:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.171+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562ffa656b20 con 0x562ffa6454c0 2024-07-24T21:52:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd874041070 con 0x562ffa6454c0 2024-07-24T21:52:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd86c018030 con 0x562ffa6454c0 2024-07-24T21:52:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 10 client.26231 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:52:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:02.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 20 client.26231 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:02.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa657f80 0x562ffa65a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 4 client.26231 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:02.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.174+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffa5b4940 con 0x562ffa657f80 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.179+0000 7fd87bfff640 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa657f80 0x562ffa65a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.184+0000 7fd87bfff640 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa657f80 0x562ffa65a3c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a024968e83a12226 server_cookie=764859a436cad8a0 in_seq=0 out_seq=0 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.184+0000 7fd85b7fe640 10 client.26231 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.184+0000 7fd85b7fe640 1 -- 172.21.15.60:0/279086110 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ffa5b4940 con 0x562ffa657f80 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.184+0000 7fd85b7fe640 10 client.26231 handle_command_reply: tid=0 2024-07-24T21:52:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.185+0000 7fd884c68740 1 client.26231 shutdown 2024-07-24T21:52:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.185+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 msgr2=0x562ffa6539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.185+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd86c011730 tx=0x7fd86c011bf0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 shutdown_connections 2024-07-24T21:52:02.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa657f80 0x562ffa65a3c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6454c0 0x562ffa6539f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa644f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 --2- 172.21.15.60:0/279086110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa63ba20 0x562ffa644a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 >> 172.21.15.60:0/279086110 conn(0x562ffa638f30 msgr2=0x562ffa63ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.187+0000 7fd85b7fe640 0 client.26231 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:02.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.191+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 shutdown_connections 2024-07-24T21:52:02.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.191+0000 7fd884c68740 1 -- 172.21.15.60:0/279086110 wait complete. 2024-07-24T21:52:02.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.191+0000 7fd884c68740 20 client.26231 trim_cache size 0 max 16384 2024-07-24T21:52:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.251+0000 7fd884c68740 1 Processor -- start 2024-07-24T21:52:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.253+0000 7fd884c68740 1 -- start start 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.253+0000 7fd884c68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa63ba20 0x562ffa6537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.253+0000 7fd884c68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa63e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.253+0000 7fd884c68740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd87b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa63e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49434/0 (socket says 172.21.15.60:49434) 2024-07-24T21:52:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 -- 172.21.15.60:0/1096623828 learned_addr learned my addr 172.21.15.60:0/1096623828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bfd80 con 0x562ffa6479b0 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bd5f0 con 0x562ffa63ba20 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bb440 con 0x562ffa6454c0 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 -- 172.21.15.60:0/1096623828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 msgr2=0x562ffa63e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa63e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 -- 172.21.15.60:0/1096623828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa63ba20 msgr2=0x562ffa6537e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:52:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa63ba20 0x562ffa6537e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.254+0000 7fd880e54640 1 -- 172.21.15.60:0/1096623828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffa633ec0 con 0x562ffa6479b0 2024-07-24T21:52:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.255+0000 7fd880e54640 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00f790 tx=0x7fd86c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b12f597cae1f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.258+0000 7fd861ffb640 1 -- 172.21.15.60:0/1096623828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c028080 con 0x562ffa6479b0 2024-07-24T21:52:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.258+0000 7fd861ffb640 1 -- 172.21.15.60:0/1096623828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c0054e0 con 0x562ffa6479b0 2024-07-24T21:52:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.258+0000 7fd861ffb640 1 -- 172.21.15.60:0/1096623828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00fbb0 con 0x562ffa6479b0 2024-07-24T21:52:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.259+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 msgr2=0x562ffa65fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00f790 tx=0x7fd86c00cbd0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 shutdown_connections 2024-07-24T21:52:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa6479b0 0x562ffa65fcd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa63e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 --2- 172.21.15.60:0/1096623828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa63ba20 0x562ffa6537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.260+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 >> 172.21.15.60:0/1096623828 conn(0x562ffa638f30 msgr2=0x562ffa63e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.267+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 shutdown_connections 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.271+0000 7fd884c68740 1 -- 172.21.15.60:0/1096623828 wait complete. 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.271+0000 7fd884c68740 1 -- start start 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.271+0000 7fd884c68740 1 Processor -- start 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa64c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa64c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa6bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bbab0 con 0x562ffa63ba20 2024-07-24T21:52:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bcf20 con 0x562ffa6479b0 2024-07-24T21:52:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1bc180 con 0x562ffa6454c0 2024-07-24T21:52:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.272+0000 7fd884c68740 10 client.0 _ll_register_callbacks cb 0x7fd86c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd880e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa6bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd880e54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa6bbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:43710/0 (socket says 172.21.15.60:43710) 2024-07-24T21:52:02.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd880e54640 1 -- 172.21.15.60:0/797747347 learned_addr learned my addr 172.21.15.60:0/797747347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:02.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa64c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 -- 172.21.15.60:0/797747347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 msgr2=0x562ffa6bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa6bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 -- 172.21.15.60:0/797747347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 msgr2=0x562ffa64c0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa64c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 -- 172.21.15.60:0/797747347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffa6bc380 con 0x562ffa6454c0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.273+0000 7fd87b7fe640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa64c670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8700e21f0 tx=0x7fd8700094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826d07dd7e8cefdb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.276+0000 7fd85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.276+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700e2680 con 0x562ffa6454c0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.276+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fd8700aa910 con 0x562ffa6454c0 2024-07-24T21:52:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.276+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c0054e0 con 0x562ffa6454c0 2024-07-24T21:52:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.282+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd8700ab6b0 con 0x562ffa6454c0 2024-07-24T21:52:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.283+0000 7fd884c68740 10 client.26242 fetch_fsmap learned FSMap version 31 2024-07-24T21:52:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.283+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562ffa6c3780 con 0x562ffa6454c0 2024-07-24T21:52:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.289+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c0054e0 con 0x562ffa6454c0 2024-07-24T21:52:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.289+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8700b0e40 con 0x562ffa6454c0 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 10 client.26242 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 20 client.26242 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa6c3ec0 0x562ffa6c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 4 client.26242 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:52:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.293+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ffa5b03e0 con 0x562ffa6c3ec0 2024-07-24T21:52:02.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.295+0000 7fd87bfff640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa6c3ec0 0x562ffa6c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.304+0000 7fd87bfff640 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa6c3ec0 0x562ffa6c62e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe8cda65a8b1bcb server_cookie=ef5130efb9867b58 in_seq=0 out_seq=0 2024-07-24T21:52:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.304+0000 7fd85b7fe640 10 client.26242 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:02.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.308+0000 7fd85b7fe640 1 -- 172.21.15.60:0/797747347 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ffa5b03e0 con 0x562ffa6c3ec0 2024-07-24T21:52:02.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.308+0000 7fd85b7fe640 10 client.26242 handle_command_reply: tid=0 2024-07-24T21:52:02.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.308+0000 7fd884c68740 1 client.26242 shutdown 2024-07-24T21:52:02.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.313+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 msgr2=0x562ffa64c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.313+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa64c670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8700e21f0 tx=0x7fd8700094a0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 shutdown_connections 2024-07-24T21:52:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562ffa6c3ec0 0x562ffa6c62e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ffa6479b0 0x562ffa6bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ffa6454c0 0x562ffa64c670 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 --2- 172.21.15.60:0/797747347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562ffa63ba20 0x562ffa64c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd85b7fe640 0 client.26242 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.316+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 >> 172.21.15.60:0/797747347 conn(0x562ffa638f30 msgr2=0x562ffa646890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.318+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 shutdown_connections 2024-07-24T21:52:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.322+0000 7fd884c68740 1 -- 172.21.15.60:0/797747347 wait complete. 2024-07-24T21:52:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.322+0000 7fd884c68740 20 client.26242 trim_cache size 0 max 16384 2024-07-24T21:52:02.325 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:52:02.325 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:52:02.325 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "69137c0b-3e1b-4039-932f-7243537f927e", 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.323+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 msgr2=0x7fd8640b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.323+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 0x7fd8640b6450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd874009b30 tx=0x7fd874005fd0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.323+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 msgr2=0x7fd87c1a8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.323+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9f0 tx=0x7fd87000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 shutdown_connections 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd8640b3f90 0x7fd8640b6450 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd874009b30 tx=0x7fd874005fd0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c1ad3c0 0x7fd87c1ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd87c11ba90 0x7fd87c1a8020 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 --2- 172.21.15.60:0/3715504353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd87c109f20 0x7fd87c1a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:02.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.325+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 >> 172.21.15.60:0/3715504353 conn(0x7fd87c06c4f0 msgr2=0x7fd87c11bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:02.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.329+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 shutdown_connections 2024-07-24T21:52:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:02.332+0000 7fd881655640 1 -- 172.21.15.60:0/3715504353 wait complete. 2024-07-24T21:52:03.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 762 op/s 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:03.574 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 762 op/s 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 762 op/s 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:03.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:03.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:04.712 INFO:tasks.workunit.client.0.smithi060.stdout: 398 1443121 787 1022989 643 409312 1613 2024-07-24T21:52:05.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:05 smithi123 ceph-mon[37143]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 710 op/s 2024-07-24T21:52:05.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:05 smithi086 ceph-mon[37128]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 710 op/s 2024-07-24T21:52:05.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:05 smithi060 ceph-mon[33000]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 710 op/s 2024-07-24T21:52:07.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:07 smithi123 ceph-mon[37143]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 704 op/s 2024-07-24T21:52:07.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:07 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:07 smithi086 ceph-mon[37128]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 704 op/s 2024-07-24T21:52:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:07 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:07 smithi060 ceph-mon[33000]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 704 op/s 2024-07-24T21:52:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:07 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 -- 172.21.15.60:0/3786183879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1618109f20 msgr2=0x7f161810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 --2- 172.21.15.60:0/3786183879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1618109f20 0x7f161810a360 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f16140098b0 tx=0x7f161402a3d0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 -- 172.21.15.60:0/3786183879 shutdown_connections 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 --2- 172.21.15.60:0/3786183879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f161811ba90 0x7f161811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 --2- 172.21.15.60:0/3786183879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1618109f20 0x7f161810a360 secure :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f16140098b0 tx=0x7f161402a3d0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 --2- 172.21.15.60:0/3786183879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 0x7f1618110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.153+0000 7f161e0fe640 1 -- 172.21.15.60:0/3786183879 >> 172.21.15.60:0/3786183879 conn(0x7f161806d570 msgr2=0x7f161806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.156+0000 7f161e0fe640 1 -- 172.21.15.60:0/3786183879 shutdown_connections 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.156+0000 7f161e0fe640 1 -- 172.21.15.60:0/3786183879 wait complete. 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.157+0000 7f161e0fe640 1 Processor -- start 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.157+0000 7f161e0fe640 1 -- start start 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.157+0000 7f161e0fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 0x7f16181a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161e0fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 0x7f16181a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161e0fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161e0fe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f16181212e0 con 0x7f161811ba90 2024-07-24T21:52:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161e0fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1618121160 con 0x7f1618109b40 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161e0fe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1618121460 con 0x7f16181ad360 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161d8fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161d8fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40914/0 (socket says 172.21.15.60:40914) 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161d8fd640 1 -- 172.21.15.60:0/1629971043 learned_addr learned my addr 172.21.15.60:0/1629971043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.158+0000 7f161d0fc640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 0x7f16181a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.159+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 0x7f16181a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.159+0000 7f161d8fd640 1 -- 172.21.15.60:0/1629971043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 msgr2=0x7f16181a7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.159+0000 7f161d8fd640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 0x7f16181a7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.159+0000 7f161d8fd640 1 -- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 msgr2=0x7f16181a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.160+0000 7f161d8fd640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 0x7f16181a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.160+0000 7f161d8fd640 1 -- 172.21.15.60:0/1629971043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16181adf90 con 0x7f16181ad360 2024-07-24T21:52:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.160+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 0x7f16181a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:52:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.160+0000 7f161d8fd640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1610009fa0 tx=0x7f161000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.202+0000 7f160e7fc640 1 -- 172.21.15.60:0/1629971043 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610004b20 con 0x7f16181ad360 2024-07-24T21:52:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.203+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16181b44e0 con 0x7f16181ad360 2024-07-24T21:52:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.203+0000 7f160e7fc640 1 -- 172.21.15.60:0/1629971043 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1610003e20 con 0x7f16181ad360 2024-07-24T21:52:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.203+0000 7f160e7fc640 1 -- 172.21.15.60:0/1629971043 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161000b570 con 0x7f16181ad360 2024-07-24T21:52:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.203+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181b4940 con 0x7f16181ad360 2024-07-24T21:52:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.214+0000 7f162218c740 1 Processor -- start 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 -- start start 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d048451e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d048460740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d0483c7280 con 0x55d048451aa0 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0483b6780 con 0x55d04845e350 2024-07-24T21:52:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.215+0000 7f162218c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0483ba7c0 con 0x55d0484523c0 2024-07-24T21:52:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.216+0000 7f161c8fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d048451e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.216+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.216+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40922/0 (socket says 172.21.15.60:40922) 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.216+0000 7f161d0fc640 1 -- 172.21.15.60:0/481228612 learned_addr learned my addr 172.21.15.60:0/481228612 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 -- 172.21.15.60:0/481228612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 msgr2=0x55d048460740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d048460740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 -- 172.21.15.60:0/481228612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 msgr2=0x55d048451e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d048451e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 -- 172.21.15.60:0/481228612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04811a790 con 0x55d0484523c0 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161c8fb640 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d048451e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.217+0000 7f161d0fc640 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f160800bd10 tx=0x7f16080107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5d04c82056beba server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.218+0000 7f160e7fc640 1 -- 172.21.15.60:0/1629971043 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f161001d020 con 0x7f16181ad360 2024-07-24T21:52:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.219+0000 7f160e7fc640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 0x7f16040b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.220+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 0x7f16040b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.221+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 0x7f16040b6410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f161402a940 tx=0x7f1614000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.236+0000 7f160e7fc640 1 -- 172.21.15.60:0/1629971043 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f16100e4370 con 0x7f16181ad360 2024-07-24T21:52:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.258+0000 7f16037fe640 1 -- 172.21.15.60:0/481228612 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608004030 con 0x55d0484523c0 2024-07-24T21:52:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.258+0000 7f16037fe640 1 -- 172.21.15.60:0/481228612 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16080035f0 con 0x55d0484523c0 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.259+0000 7f162218c740 1 -- 172.21.15.60:0/481228612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 msgr2=0x55d04845de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.259+0000 7f162218c740 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f160800bd10 tx=0x7f16080107a0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.260+0000 7f162218c740 1 -- 172.21.15.60:0/481228612 shutdown_connections 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.260+0000 7f162218c740 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d048460740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.260+0000 7f162218c740 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d04845de10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.260+0000 7f162218c740 1 --2- 172.21.15.60:0/481228612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d048451e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.260+0000 7f162218c740 1 -- 172.21.15.60:0/481228612 >> 172.21.15.60:0/481228612 conn(0x55d04844f8d0 msgr2=0x55d048450a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.269+0000 7f162218c740 1 -- 172.21.15.60:0/481228612 shutdown_connections 2024-07-24T21:52:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.269+0000 7f162218c740 1 -- 172.21.15.60:0/481228612 wait complete. 2024-07-24T21:52:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.270+0000 7f162218c740 1 -- start start 2024-07-24T21:52:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.270+0000 7f162218c740 1 Processor -- start 2024-07-24T21:52:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d04845da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d0482f0ab0 con 0x55d04845e350 2024-07-24T21:52:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d04833e9e0 con 0x55d0484523c0 2024-07-24T21:52:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d048472090 con 0x55d048451aa0 2024-07-24T21:52:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f162218c740 10 client.0 _ll_register_callbacks cb 0x55d0480c6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f161c8fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.271+0000 7f161c8fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:51816/0 (socket says 172.21.15.60:51816) 2024-07-24T21:52:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 -- 172.21.15.60:0/1806579481 learned_addr learned my addr 172.21.15.60:0/1806579481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 -- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 msgr2=0x55d04845da20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:52:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d04845da20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 -- 172.21.15.60:0/1806579481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 msgr2=0x55d04845b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 -- 172.21.15.60:0/1806579481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d048193620 con 0x55d0484523c0 2024-07-24T21:52:08.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.272+0000 7f161c8fb640 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1614034000 tx=0x7f1614005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3920a763b9194cce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.310+0000 7f1600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.310+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614004b90 con 0x55d0484523c0 2024-07-24T21:52:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.310+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d048365ce0 con 0x55d0484523c0 2024-07-24T21:52:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.310+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1614004d30 con 0x55d0484523c0 2024-07-24T21:52:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.310+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1614051470 con 0x55d0484523c0 2024-07-24T21:52:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.311+0000 7f162218c740 10 client.26243 fetch_fsmap learned FSMap version 31 2024-07-24T21:52:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.312+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d04847d870 con 0x55d0484523c0 2024-07-24T21:52:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.314+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d048365ce0 con 0x55d0484523c0 2024-07-24T21:52:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.317+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1614045030 con 0x55d0484523c0 2024-07-24T21:52:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 10 client.26243 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:52:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 20 client.26243 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04847f1d0 0x55d048481690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 4 client.26243 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0483c91d0 con 0x55d04847f1d0 2024-07-24T21:52:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.318+0000 7f161d0fc640 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04847f1d0 0x55d048481690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.319+0000 7f161d0fc640 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04847f1d0 0x55d048481690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73a31925e33670c server_cookie=7cae69a14a79f552 in_seq=0 out_seq=0 2024-07-24T21:52:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.320+0000 7f1600ff9640 10 client.26243 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.327+0000 7f1600ff9640 1 -- 172.21.15.60:0/1806579481 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0483c91d0 con 0x55d04847f1d0 2024-07-24T21:52:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.327+0000 7f1600ff9640 10 client.26243 handle_command_reply: tid=0 2024-07-24T21:52:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.327+0000 7f162218c740 1 client.26243 shutdown 2024-07-24T21:52:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 msgr2=0x55d04845af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1614034000 tx=0x7f1614005f90 comp rx=0 tx=0).stop 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 shutdown_connections 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04847f1d0 0x55d048481690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845af00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 --2- 172.21.15.60:0/1806579481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d04845da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.331+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 >> 172.21.15.60:0/1806579481 conn(0x55d04844f8d0 msgr2=0x55d048455470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.334+0000 7f1600ff9640 0 client.26243 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.334+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 shutdown_connections 2024-07-24T21:52:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.334+0000 7f162218c740 1 -- 172.21.15.60:0/1806579481 wait complete. 2024-07-24T21:52:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.334+0000 7f162218c740 20 client.26243 trim_cache size 0 max 16384 2024-07-24T21:52:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.644+0000 7f162218c740 1 Processor -- start 2024-07-24T21:52:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.644+0000 7f162218c740 1 -- start start 2024-07-24T21:52:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f162218c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f162218c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f162218c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40946/0 (socket says 172.21.15.60:40946) 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.645+0000 7f161d0fc640 1 -- 172.21.15.60:0/2696998280 learned_addr learned my addr 172.21.15.60:0/2696998280 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f16181b73e0 con 0x55d04845e350 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16181b4c80 con 0x55d0484523c0 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16181179d0 con 0x55d048451aa0 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161c8fb640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d8fd640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 -- 172.21.15.60:0/2696998280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 msgr2=0x55d04845aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 -- 172.21.15.60:0/2696998280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 msgr2=0x55d04845b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 -- 172.21.15.60:0/2696998280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0484527a0 con 0x55d048451aa0 2024-07-24T21:52:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d8fd640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:52:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.646+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1608003ef0 tx=0x7f1608010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ce19c7fac83696 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.695+0000 7f1602ffd640 1 -- 172.21.15.60:0/2696998280 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608029070 con 0x55d048451aa0 2024-07-24T21:52:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.695+0000 7f1602ffd640 1 -- 172.21.15.60:0/2696998280 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160800b900 con 0x55d048451aa0 2024-07-24T21:52:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.695+0000 7f1602ffd640 1 -- 172.21.15.60:0/2696998280 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608003850 con 0x55d048451aa0 2024-07-24T21:52:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.696+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 msgr2=0x55d0484811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.696+0000 7f162218c740 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1608003ef0 tx=0x7f1608010fd0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.707+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 shutdown_connections 2024-07-24T21:52:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.707+0000 7f162218c740 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d04845e350 0x55d04845b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.707+0000 7f162218c740 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0484523c0 0x55d04845aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.707+0000 7f162218c740 1 --2- 172.21.15.60:0/2696998280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d048451aa0 0x55d0484811e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.707+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 >> 172.21.15.60:0/2696998280 conn(0x55d04844f8d0 msgr2=0x55d048459760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.722+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 shutdown_connections 2024-07-24T21:52:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.722+0000 7f162218c740 1 -- 172.21.15.60:0/2696998280 wait complete. 2024-07-24T21:52:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.723+0000 7f162218c740 1 -- start start 2024-07-24T21:52:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.723+0000 7f162218c740 1 Processor -- start 2024-07-24T21:52:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.724+0000 7f162218c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d048477c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d0484765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f161d0fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38882/0 (socket says 172.21.15.60:38882) 2024-07-24T21:52:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f161d0fc640 1 -- 172.21.15.60:0/2162928128 learned_addr learned my addr 172.21.15.60:0/2162928128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1618117f60 con 0x55d048451aa0 2024-07-24T21:52:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16181193d0 con 0x55d04845e350 2024-07-24T21:52:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1618118630 con 0x55d0484523c0 2024-07-24T21:52:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f162218c740 10 client.0 _ll_register_callbacks cb 0x7f1610009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f161c8fb640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d048477c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.725+0000 7f161d8fd640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d0484765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 -- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 msgr2=0x55d048477c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d048477c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 -- 172.21.15.60:0/2162928128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 msgr2=0x55d0484765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d0484765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d048476e20 con 0x55d048451aa0 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161c8fb640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d048477c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:52:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.726+0000 7f161d0fc640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1608005cd0 tx=0x7f1608011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc923c4c984dc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.757+0000 7f1600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:52:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.757+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608029050 con 0x55d048451aa0 2024-07-24T21:52:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.757+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1608005240 con 0x55d048451aa0 2024-07-24T21:52:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.761+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f160800b900 con 0x55d048451aa0 2024-07-24T21:52:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.764+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f160803c050 con 0x55d048451aa0 2024-07-24T21:52:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.764+0000 7f162218c740 10 client.16791 fetch_fsmap learned FSMap version 31 2024-07-24T21:52:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.764+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d04845f420 con 0x55d048451aa0 2024-07-24T21:52:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.765+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160800b900 con 0x55d048451aa0 2024-07-24T21:52:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.767+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f160801b420 con 0x55d048451aa0 2024-07-24T21:52:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 10 client.16791 fetch_fsmap finished waiting for FSMap version 31 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 20 client.16791 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04845fab0 0x55d04845fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 4 client.16791 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0483c4c70 con 0x55d04845fab0 2024-07-24T21:52:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.768+0000 7f161c8fb640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04845fab0 0x55d04845fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.769+0000 7f161c8fb640 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04845fab0 0x55d04845fec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bab8963c6e64e21 server_cookie=14374f8625afaa2 in_seq=0 out_seq=0 2024-07-24T21:52:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:08.769+0000 7f1600ff9640 10 client.16791 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:09.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:09 smithi123 ceph-mon[37143]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 956 op/s 2024-07-24T21:52:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:09 smithi086 ceph-mon[37128]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 956 op/s 2024-07-24T21:52:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:09 smithi060 ceph-mon[33000]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 956 op/s 2024-07-24T21:52:11.297 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.320+0000 7f1600ff9640 1 -- 172.21.15.60:0/2162928128 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0483c4c70 con 0x55d04845fab0 2024-07-24T21:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.320+0000 7f1600ff9640 10 client.16791 handle_command_reply: tid=0 2024-07-24T21:52:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.320+0000 7f162218c740 1 client.16791 shutdown 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 msgr2=0x55d0484776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1608005cd0 tx=0x7f1608011b00 comp rx=0 tx=0).stop 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 shutdown_connections 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d04845fab0 0x55d04845fec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d04845e350 0x55d0484765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0484523c0 0x55d048477c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 --2- 172.21.15.60:0/2162928128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d048451aa0 0x55d0484776a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 >> 172.21.15.60:0/2162928128 conn(0x55d04844f8d0 msgr2=0x55d048455d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f1600ff9640 0 client.16791 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 shutdown_connections 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.321+0000 7f162218c740 1 -- 172.21.15.60:0/2162928128 wait complete. 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.322+0000 7f162218c740 20 client.16791 trim_cache size 0 max 16384 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:52:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:52:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.352+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 msgr2=0x7f16040b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.352+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 0x7f16040b6410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f161402a940 tx=0x7f1614000bc0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.355+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 msgr2=0x7f16181ad810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.355+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1610009fa0 tx=0x7f161000c800 comp rx=0 tx=0).stop 2024-07-24T21:52:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.356+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 shutdown_connections 2024-07-24T21:52:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.356+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16040b3f50 0x7f16040b6410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.356+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16181ad360 0x7f16181ad810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.356+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f161811ba90 0x7f16181a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.357+0000 7f161e0fe640 1 --2- 172.21.15.60:0/1629971043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1618109b40 0x7f16181a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.357+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 >> 172.21.15.60:0/1629971043 conn(0x7f161806d570 msgr2=0x7f161811c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.357+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 shutdown_connections 2024-07-24T21:52:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:11.357+0000 7f161e0fe640 1 -- 172.21.15.60:0/1629971043 wait complete. 2024-07-24T21:52:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:11 smithi123 ceph-mon[37143]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-24T21:52:11.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:11 smithi060 ceph-mon[33000]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-24T21:52:11.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:11 smithi086 ceph-mon[37128]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 1.09k op/s 2024-07-24T21:52:13.408 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:13 smithi060 ceph-mon[33000]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 1.22k op/s 2024-07-24T21:52:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:13 smithi123 ceph-mon[37143]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 1.22k op/s 2024-07-24T21:52:13.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[68866]: *** Caught signal (Segmentation fault) ** 2024-07-24T21:52:13.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[68866]: in thread 7f693d561640 thread_name:MR_Finisher 2024-07-24T21:52:13.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:13 smithi123 podman[72769]: 2024-07-24 21:52:13.616943845 +0000 UTC m=+0.053442387 container died 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=main-86385b1, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-24T21:52:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:13 smithi086 ceph-mon[37128]: pgmap v543: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 1.22k op/s 2024-07-24T21:52:14.721 INFO:tasks.workunit.client.0.smithi060.stdout: 411 1882484 762 1338592 952 581364 1980 2024-07-24T21:52:15.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:15 smithi060 ceph-mon[33000]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 1.71k op/s 2024-07-24T21:52:15.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:15 smithi123 ceph-mon[37143]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 1.71k op/s 2024-07-24T21:52:15.738 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:15 smithi123 podman[72769]: 2024-07-24 21:52:15.397003107 +0000 UTC m=+1.833501633 container remove 4648084d62fc99762cd46e9c777d625d02a0630e0b00103ca782584e2e5524ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=main-86385b1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-24T21:52:15.739 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:15 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-07-24T21:52:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:15 smithi086 ceph-mon[37128]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 1.71k op/s 2024-07-24T21:52:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.942+0000 7fafacf77640 1 -- 172.21.15.60:0/3926910707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa8105f40 msgr2=0x7fafa8106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.942+0000 7fafacf77640 1 --2- 172.21.15.60:0/3926910707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa8105f40 0x7fafa8106380 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009860 tx=0x7faf9c02a360 comp rx=0 tx=0).stop 2024-07-24T21:52:15.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.947+0000 7fafacf77640 1 -- 172.21.15.60:0/3926910707 shutdown_connections 2024-07-24T21:52:15.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.947+0000 7fafacf77640 1 --2- 172.21.15.60:0/3926910707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa81068c0 0x7fafa81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.947+0000 7fafacf77640 1 --2- 172.21.15.60:0/3926910707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa8105f40 0x7fafa8106380 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.947+0000 7fafacf77640 1 --2- 172.21.15.60:0/3926910707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.947+0000 7fafacf77640 1 -- 172.21.15.60:0/3926910707 >> 172.21.15.60:0/3926910707 conn(0x7fafa80fc3f0 msgr2=0x7fafa80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.951+0000 7fafacf77640 1 -- 172.21.15.60:0/3926910707 shutdown_connections 2024-07-24T21:52:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.952+0000 7fafacf77640 1 -- 172.21.15.60:0/3926910707 wait complete. 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.953+0000 7fafacf77640 1 Processor -- start 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.954+0000 7fafacf77640 1 -- start start 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.954+0000 7fafacf77640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa80741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.954+0000 7fafacf77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 0x7fafa8074730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.954+0000 7fafacf77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33098/0 (socket says 172.21.15.60:33098) 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa80741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa80741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44860/0 (socket says 172.21.15.60:44860) 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 0x7fafa8074730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 0x7fafa8074730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39740/0 (socket says 172.21.15.60:39740) 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafa6d9d640 1 -- 172.21.15.60:0/1439931571 learned_addr learned my addr 172.21.15.60:0/1439931571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafacf77640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8118d40 con 0x7fafa81068c0 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafacf77640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8118bc0 con 0x7fafa8105f40 2024-07-24T21:52:15.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.955+0000 7fafacf77640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8118ec0 con 0x7fafa8100700 2024-07-24T21:52:15.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.956+0000 7fafa6d9d640 1 -- 172.21.15.60:0/1439931571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 msgr2=0x7fafa80741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:15.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.956+0000 7fafa6d9d640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa80741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.956+0000 7fafa6d9d640 1 -- 172.21.15.60:0/1439931571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 msgr2=0x7fafa8074730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:15.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.956+0000 7fafa6d9d640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 0x7fafa8074730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.956+0000 7fafa6d9d640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa806ded0 con 0x7fafa81068c0 2024-07-24T21:52:15.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.957+0000 7fafa6d9d640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fafa000eda0 tx=0x7fafa000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:15.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.961+0000 7faf977fe640 1 -- 172.21.15.60:0/1439931571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0003b70 con 0x7fafa81068c0 2024-07-24T21:52:15.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.961+0000 7fafacf77640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa806e1c0 con 0x7fafa81068c0 2024-07-24T21:52:15.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.961+0000 7fafacf77640 1 -- 172.21.15.60:0/1439931571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa8073080 con 0x7fafa81068c0 2024-07-24T21:52:15.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 Processor -- start 2024-07-24T21:52:15.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 -- start start 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4d4250 0x557d3d4d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d3d42d990 con 0x557d3d4b6990 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d3d41ce90 con 0x557d3d4d4250 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7fafaf328740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d3d420ed0 con 0x557d3d4b72b0 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7faf977fe640 1 -- 172.21.15.60:0/1439931571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa0003d10 con 0x7fafa81068c0 2024-07-24T21:52:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.968+0000 7faf977fe640 1 -- 172.21.15.60:0/1439931571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00178e0 con 0x7fafa81068c0 2024-07-24T21:52:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.971+0000 7fafa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.971+0000 7fafa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44872/0 (socket says 172.21.15.60:44872) 2024-07-24T21:52:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.971+0000 7fafa659c640 1 -- 172.21.15.60:0/3395467005 learned_addr learned my addr 172.21.15.60:0/3395467005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.971+0000 7fafa659c640 1 -- 172.21.15.60:0/3395467005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4d4250 msgr2=0x557d3d4d6640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.971+0000 7fafa659c640 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4d4250 0x557d3d4d6640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.972+0000 7fafa659c640 1 -- 172.21.15.60:0/3395467005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 msgr2=0x557d3d4b6d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.972+0000 7fafa659c640 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4b6d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.972+0000 7fafa659c640 1 -- 172.21.15.60:0/3395467005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3d3a6300 con 0x557d3d4b72b0 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.972+0000 7fafa659c640 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf9800ac50 tx=0x7faf9800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c8c743540490d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.973+0000 7faf8ffff640 1 -- 172.21.15.60:0/3395467005 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9801d070 con 0x557d3d4b72b0 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.973+0000 7faf8ffff640 1 -- 172.21.15.60:0/3395467005 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf98018af0 con 0x557d3d4b72b0 2024-07-24T21:52:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 -- 172.21.15.60:0/3395467005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 msgr2=0x557d3d4d3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf9800ac50 tx=0x7faf9800f790 comp rx=0 tx=0).stop 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 -- 172.21.15.60:0/3395467005 shutdown_connections 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4d4250 0x557d3d4d6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b72b0 0x557d3d4d3d10 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf9800ac50 tx=0x7faf9800f790 comp rx=0 tx=0).stop 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 --2- 172.21.15.60:0/3395467005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.974+0000 7fafaf328740 1 -- 172.21.15.60:0/3395467005 >> 172.21.15.60:0/3395467005 conn(0x557d3d4b47c0 msgr2=0x557d3d4b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.975+0000 7fafaf328740 1 -- 172.21.15.60:0/3395467005 shutdown_connections 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.975+0000 7fafaf328740 1 -- 172.21.15.60:0/3395467005 wait complete. 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.975+0000 7fafaf328740 1 -- start start 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.975+0000 7faf977fe640 1 -- 172.21.15.60:0/1439931571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fafa0017a80 con 0x7fafa81068c0 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 Processor -- start 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 0x557d3d4c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d3d280700 con 0x557d3d4d4250 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d3d3bf5d0 con 0x557d3d4c60b0 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d3d4bcec0 con 0x557d3d4b6990 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.976+0000 7fafaf328740 10 client.0 _ll_register_callbacks cb 0x557d3d3df940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.977+0000 7fafa659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.977+0000 7fafa659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33108/0 (socket says 172.21.15.60:33108) 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.977+0000 7fafa659c640 1 -- 172.21.15.60:0/104826271 learned_addr learned my addr 172.21.15.60:0/104826271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa659c640 1 -- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 msgr2=0x557d3d4c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.977+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 0x557d3d4c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa659c640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 0x557d3d4c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa659c640 1 -- 172.21.15.60:0/104826271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 msgr2=0x557d3d4c6530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa659c640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4c6530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa659c640 1 -- 172.21.15.60:0/104826271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3d3b2660 con 0x557d3d4d4250 2024-07-24T21:52:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.978+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 0x557d3d4c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:52:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.980+0000 7faf977fe640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 0x7faf900d8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.980+0000 7faf977fe640 1 -- 172.21.15.60:0/1439931571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fafa00e4020 con 0x7fafa81068c0 2024-07-24T21:52:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.982+0000 7fafa659c640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faf980058c0 tx=0x7faf98005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf49d9bcb5c7cb1a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.982+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 0x7faf900d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.982+0000 7faf8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:52:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.983+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9801d040 con 0x557d3d4d4250 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.983+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d3d42f8e0 con 0x557d3d4d4250 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.986+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7faf98005360 con 0x557d3d4d4250 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.986+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf9802e050 con 0x557d3d4d4250 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.986+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 0x7faf900d8260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000c00 tx=0x7faf9c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.986+0000 7fafaf328740 10 client.16800 fetch_fsmap learned FSMap version 32 2024-07-24T21:52:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.986+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d3d17d650 con 0x557d3d4d4250 2024-07-24T21:52:15.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.987+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d3d42f8e0 con 0x557d3d4d4250 2024-07-24T21:52:15.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.987+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faf98020070 con 0x557d3d4d4250 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 10 client.16800 fetch_fsmap finished waiting for FSMap version 32 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 20 client.16800 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d4d0be0 0x557d3d4d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 4 client.16800 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3d42b380 con 0x557d3d4d0be0 2024-07-24T21:52:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.988+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d4d0be0 0x557d3d4d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.990+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d4d0be0 0x557d3d4d3000 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bebb09b2d830aee server_cookie=335d04837233507f in_seq=0 out_seq=0 2024-07-24T21:52:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:15.990+0000 7faf8dffb640 10 client.16800 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.016+0000 7faf8dffb640 1 -- 172.21.15.60:0/104826271 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d3d42b380 con 0x557d3d4d0be0 2024-07-24T21:52:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.016+0000 7faf8dffb640 10 client.16800 handle_command_reply: tid=0 2024-07-24T21:52:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.016+0000 7fafaf328740 1 client.16800 shutdown 2024-07-24T21:52:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.016+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 msgr2=0x557d3d4c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.016+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faf980058c0 tx=0x7faf98005270 comp rx=0 tx=0).stop 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 shutdown_connections 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d4d0be0 0x557d3d4d3000 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4d4250 0x557d3d4c0f70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7faf8dffb640 0 client.16800 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 --2- 172.21.15.60:0/104826271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4b6990 0x557d3d4c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.017+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 >> 172.21.15.60:0/104826271 conn(0x557d3d4b47c0 msgr2=0x557d3d4b9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:16.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.018+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 shutdown_connections 2024-07-24T21:52:16.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.018+0000 7fafaf328740 1 -- 172.21.15.60:0/104826271 wait complete. 2024-07-24T21:52:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.020+0000 7fafaf328740 20 client.16800 trim_cache size 0 max 16384 2024-07-24T21:52:16.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 Processor -- start 2024-07-24T21:52:16.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 -- start start 2024-07-24T21:52:16.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4be1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faf88011360 con 0x557d3d4b6990 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf8800eba0 con 0x557d3d4c60b0 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafaf328740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf8800c9f0 con 0x557d3d4d4250 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.064+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4be1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39752/0 (socket says 172.21.15.60:39752) 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa5d9b640 1 -- 172.21.15.60:0/4212934125 learned_addr learned my addr 172.21.15.60:0/4212934125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 -- 172.21.15.60:0/4212934125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 msgr2=0x557d3d4d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 -- 172.21.15.60:0/4212934125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 msgr2=0x557d3d4d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:16.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 -- 172.21.15.60:0/4212934125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3d47b210 con 0x557d3d4b6990 2024-07-24T21:52:16.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.065+0000 7fafa659c640 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4be1d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faf98004bf0 tx=0x7faf9800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470d2790450d6f1a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.066+0000 7faf8f7fe640 1 -- 172.21.15.60:0/4212934125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98037070 con 0x557d3d4b6990 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.066+0000 7faf8f7fe640 1 -- 172.21.15.60:0/4212934125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9800aaa0 con 0x557d3d4b6990 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 -- 172.21.15.60:0/4212934125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 msgr2=0x557d3d4be1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4be1d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faf98004bf0 tx=0x7faf9800f790 comp rx=0 tx=0).stop 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 -- 172.21.15.60:0/4212934125 shutdown_connections 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 --2- 172.21.15.60:0/4212934125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4b6990 0x557d3d4be1d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 -- 172.21.15.60:0/4212934125 >> 172.21.15.60:0/4212934125 conn(0x557d3d4b47c0 msgr2=0x557d3d4b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 -- 172.21.15.60:0/4212934125 shutdown_connections 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.067+0000 7fafaf328740 1 -- 172.21.15.60:0/4212934125 wait complete. 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 -- start start 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 Processor -- start 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4cd200 0x557d3d4db800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faf8800d060 con 0x557d3d4cd200 2024-07-24T21:52:16.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf8800e4d0 con 0x557d3d4c60b0 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.068+0000 7fafaf328740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf8800d730 con 0x557d3d4d4250 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafaf328740 10 client.0 _ll_register_callbacks cb 0x7faf9801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44912/0 (socket says 172.21.15.60:44912) 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 -- 172.21.15.60:0/3426131586 learned_addr learned my addr 172.21.15.60:0/3426131586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 -- 172.21.15.60:0/3426131586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 msgr2=0x557d3d4dace0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4dace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 -- 172.21.15.60:0/3426131586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4cd200 msgr2=0x557d3d4db800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4cd200 0x557d3d4db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.069+0000 7fafa5d9b640 1 -- 172.21.15.60:0/3426131586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3d4dc030 con 0x557d3d4d4250 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.070+0000 7fafa5d9b640 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009830 tx=0x7faf9c02a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef40049aeaf0954 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.070+0000 7faf8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.070+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c057060 con 0x557d3d4d4250 2024-07-24T21:52:16.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.070+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7faf9c0307d0 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.073+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d3d4c0070 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.074+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf9c037040 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.074+0000 7fafaf328740 10 client.26269 fetch_fsmap learned FSMap version 32 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.074+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d3d53d5e0 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.074+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d3d4c0070 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.074+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faf9c004aa0 con 0x557d3d4d4250 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 10 client.26269 fetch_fsmap finished waiting for FSMap version 32 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 20 client.26269 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d53dda0 0x557d3d5401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:16.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 4 client.26269 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:52:16.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.075+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d3d428ac0 con 0x557d3d53dda0 2024-07-24T21:52:16.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.076+0000 7fafa659c640 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d53dda0 0x557d3d5401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:16.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.090+0000 7fafa659c640 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d53dda0 0x557d3d5401e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fca752c2a7dc1c1 server_cookie=1a68d20bc5c93836 in_seq=0 out_seq=0 2024-07-24T21:52:16.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.090+0000 7faf8d7fa640 10 client.26269 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:16.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.097+0000 7faf8d7fa640 1 -- 172.21.15.60:0/3426131586 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x557d3d428ac0 con 0x557d3d53dda0 2024-07-24T21:52:16.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.097+0000 7faf8d7fa640 10 client.26269 handle_command_reply: tid=0 2024-07-24T21:52:16.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.097+0000 7fafaf328740 1 client.26269 shutdown 2024-07-24T21:52:16.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.097+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 msgr2=0x557d3d4db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:16.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.097+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009830 tx=0x7faf9c02a360 comp rx=0 tx=0).stop 2024-07-24T21:52:16.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 shutdown_connections 2024-07-24T21:52:16.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d3d53dda0 0x557d3d5401e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d3d4cd200 0x557d3d4db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d3d4d4250 0x557d3d4db220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 --2- 172.21.15.60:0/3426131586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3d4c60b0 0x557d3d4dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 >> 172.21.15.60:0/3426131586 conn(0x557d3d4b47c0 msgr2=0x557d3d4d3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7faf8d7fa640 0 client.26269 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.101+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 shutdown_connections 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.102+0000 7fafaf328740 1 -- 172.21.15.60:0/3426131586 wait complete. 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.102+0000 7fafaf328740 20 client.26269 trim_cache size 0 max 16384 2024-07-24T21:52:16.104 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (6075 inodes in the stack)", 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "69137c0b-3e1b-4039-932f-7243537f927e": { 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "69137c0b-3e1b-4039-932f-7243537f927e", 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:52:16.105 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:52:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 msgr2=0x7faf900d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 0x7faf900d8260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000c00 tx=0x7faf9c009500 comp rx=0 tx=0).stop 2024-07-24T21:52:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 msgr2=0x7fafa806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fafa000eda0 tx=0x7fafa000c6e0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 shutdown_connections 2024-07-24T21:52:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faf900d5da0 0x7faf900d8260 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000c00 tx=0x7faf9c009500 comp rx=0 tx=0).stop 2024-07-24T21:52:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafa81068c0 0x7fafa806d610 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa8105f40 0x7fafa8074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 --2- 172.21.15.60:0/1439931571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa8100700 0x7fafa80741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.104+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 >> 172.21.15.60:0/1439931571 conn(0x7fafa80fc3f0 msgr2=0x7fafa8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.106+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 shutdown_connections 2024-07-24T21:52:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:16.106+0000 7faf957fa640 1 -- 172.21.15.60:0/1439931571 wait complete. 2024-07-24T21:52:16.243 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:15 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-07-24T21:52:16.243 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:15 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Consumed 10.338s CPU time. 2024-07-24T21:52:16.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:16 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:16.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:16 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:16.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:16 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:17.290 INFO:tasks.cephfs.filesystem:scrub status for tag:69137c0b-3e1b-4039-932f-7243537f927e - {'path': '/', 'tag': '69137c0b-3e1b-4039-932f-7243537f927e', 'options': 'recursive,force'} 2024-07-24T21:52:17.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:17 smithi060 ceph-mon[33000]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 2.05k op/s 2024-07-24T21:52:17.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:17 smithi123 ceph-mon[37143]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 2.05k op/s 2024-07-24T21:52:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:17 smithi086 ceph-mon[37128]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 2.05k op/s 2024-07-24T21:52:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:19 smithi123 ceph-mon[37143]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 2.19k op/s 2024-07-24T21:52:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:19 smithi086 ceph-mon[37128]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 2.19k op/s 2024-07-24T21:52:19.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:19 smithi060 ceph-mon[33000]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 17 MiB/s wr, 2.19k op/s 2024-07-24T21:52:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:20.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:20.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:20 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:20.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:20 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:20.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:20 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:21.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:21 smithi123 ceph-mon[37143]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.94k op/s 2024-07-24T21:52:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:21 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:21 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:21 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:21 smithi086 ceph-mon[37128]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.94k op/s 2024-07-24T21:52:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:21 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:21 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:21 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:21.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:21 smithi060 ceph-mon[33000]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.94k op/s 2024-07-24T21:52:21.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:21 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:21.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:21 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:21.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:21 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:22 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:52:22.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:22 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:22 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:22 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:52:22.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:22 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:22 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:22 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:52:22.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:22 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:22.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:22 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:23 smithi123 ceph-mon[37143]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-07-24T21:52:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:23 smithi086 ceph-mon[37128]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-07-24T21:52:23.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:23 smithi060 ceph-mon[33000]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-07-24T21:52:24.713 INFO:tasks.workunit.client.0.smithi060.stdout: 427 1707920 959 1215507 840 560885 1739 2024-07-24T21:52:25.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:25 smithi123 ceph-mon[37143]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.02k op/s 2024-07-24T21:52:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:25 smithi086 ceph-mon[37128]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.02k op/s 2024-07-24T21:52:25.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:25 smithi060 ceph-mon[33000]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 21 MiB/s wr, 2.02k op/s 2024-07-24T21:52:26.266 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:25 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-07-24T21:52:26.266 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:25 smithi123 systemd[1]: Stopped Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:52:26.266 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:25 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Consumed 10.338s CPU time. 2024-07-24T21:52:26.266 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:26 smithi123 systemd[1]: Starting Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:52:26.742 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:26 smithi123 podman[72861]: 2024-07-24 21:52:26.16897239 +0000 UTC m=+0.027673138 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:52:27.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:27 smithi060 ceph-mon[33000]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-07-24T21:52:27.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:27 smithi123 podman[72861]: 2024-07-24 21:52:27.35669983 +0000 UTC m=+1.215400559 container create 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, RELEASE=main-86385b1, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T21:52:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:27 smithi123 ceph-mon[37143]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-07-24T21:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.766+0000 7fe350759640 1 -- 172.21.15.60:0/3399792667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109f20 msgr2=0x7fe34810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.766+0000 7fe350759640 1 --2- 172.21.15.60:0/3399792667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109f20 0x7fe34810a360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe344009950 tx=0x7fe34402a480 comp rx=0 tx=0).stop 2024-07-24T21:52:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.770+0000 7fe350759640 1 -- 172.21.15.60:0/3399792667 shutdown_connections 2024-07-24T21:52:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.771+0000 7fe350759640 1 --2- 172.21.15.60:0/3399792667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe34811ba90 0x7fe34811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.771+0000 7fe350759640 1 --2- 172.21.15.60:0/3399792667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109f20 0x7fe34810a360 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.771+0000 7fe350759640 1 --2- 172.21.15.60:0/3399792667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348109b40 0x7fe348110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.771+0000 7fe350759640 1 -- 172.21.15.60:0/3399792667 >> 172.21.15.60:0/3399792667 conn(0x7fe34806d5a0 msgr2=0x7fe34806d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.775+0000 7fe350759640 1 -- 172.21.15.60:0/3399792667 shutdown_connections 2024-07-24T21:52:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.775+0000 7fe350759640 1 -- 172.21.15.60:0/3399792667 wait complete. 2024-07-24T21:52:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.776+0000 7fe350759640 1 Processor -- start 2024-07-24T21:52:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 -- start start 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348109f20 0x7fe3481a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe34811ba90 0x7fe3481ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481213a0 con 0x7fe348109b40 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe348121220 con 0x7fe348109f20 2024-07-24T21:52:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.777+0000 7fe350759640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe348121520 con 0x7fe34811ba90 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.778+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.778+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45312/0 (socket says 172.21.15.60:45312) 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.778+0000 7fe34e4f5640 1 -- 172.21.15.60:0/520486281 learned_addr learned my addr 172.21.15.60:0/520486281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 -- 172.21.15.60:0/520486281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe34811ba90 msgr2=0x7fe3481ad1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe34811ba90 0x7fe3481ad1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 -- 172.21.15.60:0/520486281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348109f20 msgr2=0x7fe3481a7e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348109f20 0x7fe3481a7e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 -- 172.21.15.60:0/520486281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3481ad980 con 0x7fe348109b40 2024-07-24T21:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.779+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b6d0 tx=0x7fe33800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:27 smithi086 ceph-mon[37128]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-07-24T21:52:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.811+0000 7fe33f7fe640 1 -- 172.21.15.60:0/520486281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338019070 con 0x7fe348109b40 2024-07-24T21:52:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.812+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3481adc70 con 0x7fe348109b40 2024-07-24T21:52:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.812+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481b4330 con 0x7fe348109b40 2024-07-24T21:52:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.816+0000 7fe353584740 1 Processor -- start 2024-07-24T21:52:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.820+0000 7fe353584740 1 -- start start 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a79fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e60a715280 con 0x55e60a7ac370 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e60a704780 con 0x55e60a79fac0 2024-07-24T21:52:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.821+0000 7fe353584740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e60a7087c0 con 0x55e60a7a03e0 2024-07-24T21:52:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45320/0 (socket says 172.21.15.60:45320) 2024-07-24T21:52:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 -- 172.21.15.60:0/4126817886 learned_addr learned my addr 172.21.15.60:0/4126817886 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 -- 172.21.15.60:0/4126817886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 msgr2=0x55e60a7abe30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7abe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 -- 172.21.15.60:0/4126817886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 msgr2=0x55e60a79fea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:52:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a79fea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.822+0000 7fe34dcf4640 1 -- 172.21.15.60:0/4126817886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60a468790 con 0x55e60a7ac370 2024-07-24T21:52:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.825+0000 7fe34dcf4640 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe344033040 tx=0x7fe344009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881b8432ec9ca6e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.827+0000 7fe33f7fe640 1 -- 172.21.15.60:0/520486281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33800cbd0 con 0x7fe348109b40 2024-07-24T21:52:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.827+0000 7fe33f7fe640 1 -- 172.21.15.60:0/520486281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338012340 con 0x7fe348109b40 2024-07-24T21:52:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.829+0000 7fe33f7fe640 1 -- 172.21.15.60:0/520486281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe33801c070 con 0x7fe348109b40 2024-07-24T21:52:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.848+0000 7fe33f7fe640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 0x7fe3340b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.848+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 0x7fe3340b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.850+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 0x7fe3340b6440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe340005fd0 tx=0x7fe340005f00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.851+0000 7fe33f7fe640 1 -- 172.21.15.60:0/520486281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe3380e3ef0 con 0x7fe348109b40 2024-07-24T21:52:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.898+0000 7fe33cff9640 1 -- 172.21.15.60:0/4126817886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344041070 con 0x55e60a7ac370 2024-07-24T21:52:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.898+0000 7fe33cff9640 1 -- 172.21.15.60:0/4126817886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3440048c0 con 0x55e60a7ac370 2024-07-24T21:52:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.898+0000 7fe33cff9640 1 -- 172.21.15.60:0/4126817886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34403d740 con 0x55e60a7ac370 2024-07-24T21:52:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.899+0000 7fe353584740 1 -- 172.21.15.60:0/4126817886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 msgr2=0x55e60a7ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.899+0000 7fe353584740 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe344033040 tx=0x7fe344009620 comp rx=0 tx=0).stop 2024-07-24T21:52:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 -- 172.21.15.60:0/4126817886 shutdown_connections 2024-07-24T21:52:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7ae760 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 --2- 172.21.15.60:0/4126817886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a79fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 -- 172.21.15.60:0/4126817886 >> 172.21.15.60:0/4126817886 conn(0x55e60a79d8f0 msgr2=0x55e60a79ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 -- 172.21.15.60:0/4126817886 shutdown_connections 2024-07-24T21:52:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.900+0000 7fe353584740 1 -- 172.21.15.60:0/4126817886 wait complete. 2024-07-24T21:52:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 -- start start 2024-07-24T21:52:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 Processor -- start 2024-07-24T21:52:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7a03e0 0x55e60a7a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e60a63eab0 con 0x55e60a7a03e0 2024-07-24T21:52:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e60a68c9e0 con 0x55e60a7ac370 2024-07-24T21:52:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e60a7aeff0 con 0x55e60a79fac0 2024-07-24T21:52:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.901+0000 7fe353584740 10 client.0 _ll_register_callbacks cb 0x55e60a414a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.902+0000 7fe34ecf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.902+0000 7fe34ecf6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49208/0 (socket says 172.21.15.60:49208) 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.902+0000 7fe34ecf6640 1 -- 172.21.15.60:0/2795101909 learned_addr learned my addr 172.21.15.60:0/2795101909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.902+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 -- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 msgr2=0x55e60a7abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 -- 172.21.15.60:0/2795101909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7a03e0 msgr2=0x55e60a7a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7a03e0 0x55e60a7a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 -- 172.21.15.60:0/2795101909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60a4e1620 con 0x55e60a7ac370 2024-07-24T21:52:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.903+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe34000f980 tx=0x7fe340004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48681586b11b5d3f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.937+0000 7fe331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.937+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340020030 con 0x55e60a7ac370 2024-07-24T21:52:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.937+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e60a6b3ce0 con 0x55e60a7ac370 2024-07-24T21:52:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.937+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe340004b20 con 0x55e60a7ac370 2024-07-24T21:52:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.938+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe34002a050 con 0x55e60a7ac370 2024-07-24T21:52:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.938+0000 7fe353584740 10 client.26261 fetch_fsmap learned FSMap version 33 2024-07-24T21:52:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.938+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e60a7ba8e0 con 0x55e60a7ac370 2024-07-24T21:52:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e60a6b3ce0 con 0x55e60a7ac370 2024-07-24T21:52:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fe34001c990 con 0x55e60a7ac370 2024-07-24T21:52:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 10 client.26261 fetch_fsmap finished waiting for FSMap version 33 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 20 client.26261 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a7bc2f0 0x55e60a7be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 4 client.26261 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60a7171d0 con 0x55e60a7bc2f0 2024-07-24T21:52:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.941+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a7bc2f0 0x55e60a7be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.942+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a7bc2f0 0x55e60a7be770 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53449b90ad714547 server_cookie=abd80b67ba4bffe0 in_seq=0 out_seq=0 2024-07-24T21:52:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.943+0000 7fe331ffb640 10 client.26261 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.947+0000 7fe331ffb640 1 -- 172.21.15.60:0/2795101909 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e60a7171d0 con 0x55e60a7bc2f0 2024-07-24T21:52:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.947+0000 7fe331ffb640 10 client.26261 handle_command_reply: tid=0 2024-07-24T21:52:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.947+0000 7fe353584740 1 client.26261 shutdown 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.947+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 msgr2=0x55e60a7a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.947+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe34000f980 tx=0x7fe340004ab0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 shutdown_connections 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a7bc2f0 0x55e60a7be770 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7ac370 0x55e60a7a9520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7a03e0 0x55e60a7a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 --2- 172.21.15.60:0/2795101909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 >> 172.21.15.60:0/2795101909 conn(0x55e60a79d8f0 msgr2=0x55e60a7a3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe331ffb640 0 client.26261 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 shutdown_connections 2024-07-24T21:52:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 1 -- 172.21.15.60:0/2795101909 wait complete. 2024-07-24T21:52:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:27.948+0000 7fe353584740 20 client.26261 trim_cache size 0 max 16384 2024-07-24T21:52:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.276+0000 7fe353584740 1 Processor -- start 2024-07-24T21:52:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.277+0000 7fe353584740 1 -- start start 2024-07-24T21:52:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe353584740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe353584740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 0x55e60a7a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:57792/0 (socket says 172.21.15.60:57792) 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 0x55e60a7a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.300+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 0x55e60a7a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49210/0 (socket says 172.21.15.60:49210) 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.301+0000 7fe34e4f5640 1 -- 172.21.15.60:0/4164879045 learned_addr learned my addr 172.21.15.60:0/4164879045 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.301+0000 7fe353584740 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.301+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0112d0 con 0x55e60a7ac370 2024-07-24T21:52:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c00eae0 con 0x55e60a7a03e0 2024-07-24T21:52:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c00c930 con 0x55e60a79fac0 2024-07-24T21:52:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe34e4f5640 1 -- 172.21.15.60:0/4164879045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 msgr2=0x55e60a7a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 0x55e60a7a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe34e4f5640 1 -- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 msgr2=0x55e60a7a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.302+0000 7fe34e4f5640 1 -- 172.21.15.60:0/4164879045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60a7a0210 con 0x55e60a79fac0 2024-07-24T21:52:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.303+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:52:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.303+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3380e17d0 tx=0x7fe338000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7bec2febaf0aaf9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.305+0000 7fe333fff640 1 -- 172.21.15.60:0/4164879045 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33800cd80 con 0x55e60a79fac0 2024-07-24T21:52:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.305+0000 7fe333fff640 1 -- 172.21.15.60:0/4164879045 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3380a6ec0 con 0x55e60a79fac0 2024-07-24T21:52:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.305+0000 7fe333fff640 1 -- 172.21.15.60:0/4164879045 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380a9e90 con 0x55e60a79fac0 2024-07-24T21:52:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.306+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 msgr2=0x55e60a7b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.307+0000 7fe353584740 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3380e17d0 tx=0x7fe338000bc0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.324+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 shutdown_connections 2024-07-24T21:52:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.324+0000 7fe353584740 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.324+0000 7fe353584740 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a7a03e0 0x55e60a7a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.324+0000 7fe353584740 1 --2- 172.21.15.60:0/4164879045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a79fac0 0x55e60a7b81a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.324+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 >> 172.21.15.60:0/4164879045 conn(0x55e60a79d8f0 msgr2=0x55e60a79e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.337+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 shutdown_connections 2024-07-24T21:52:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.338+0000 7fe353584740 1 -- 172.21.15.60:0/4164879045 wait complete. 2024-07-24T21:52:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.338+0000 7fe353584740 1 -- start start 2024-07-24T21:52:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.339+0000 7fe353584740 1 Processor -- start 2024-07-24T21:52:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.339+0000 7fe353584740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49222/0 (socket says 172.21.15.60:49222) 2024-07-24T21:52:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34dcf4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7a51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:57798/0 (socket says 172.21.15.60:57798) 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 -- 172.21.15.60:0/1360008759 learned_addr learned my addr 172.21.15.60:0/1360008759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c00cfa0 con 0x55e60a7ac370 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c00e410 con 0x55e60a79fac0 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c00d670 con 0x55e60a7a03e0 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34ecf6640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe353584740 10 client.0 _ll_register_callbacks cb 0x7fe340007320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 -- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 msgr2=0x55e60a7a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 -- 172.21.15.60:0/1360008759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 msgr2=0x55e60a7a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.340+0000 7fe34e4f5640 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60a7a5c90 con 0x55e60a79fac0 2024-07-24T21:52:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.341+0000 7fe34e4f5640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3380c4040 tx=0x7fe3380b3a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6256a09c413bee35 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.366+0000 7fe331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.366+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380b8060 con 0x55e60a79fac0 2024-07-24T21:52:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.366+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3380a6ec0 con 0x55e60a79fac0 2024-07-24T21:52:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.366+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe3380ab0a0 con 0x55e60a79fac0 2024-07-24T21:52:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.373+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe3380aaa00 con 0x55e60a79fac0 2024-07-24T21:52:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.373+0000 7fe353584740 10 client.26267 fetch_fsmap learned FSMap version 34 2024-07-24T21:52:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.373+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e60a7a6f40 con 0x55e60a79fac0 2024-07-24T21:52:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.375+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3380a9e90 con 0x55e60a79fac0 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.376+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) ==== 7765+0+0 (secure 0 0 0) 0x7fe3380a9900 con 0x55e60a79fac0 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.376+0000 7fe353584740 10 client.26267 fetch_fsmap finished waiting for FSMap version 34 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.376+0000 7fe353584740 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.376+0000 7fe353584740 20 client.26267 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.377+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a826420 0x55e60a828840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.377+0000 7fe353584740 4 client.26267 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:52:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.377+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe3380a6ec0 con 0x55e60a826420 2024-07-24T21:52:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.377+0000 7fe34dcf4640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a826420 0x55e60a828840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.378+0000 7fe34dcf4640 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a826420 0x55e60a828840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7aa2f106d4a79fa server_cookie=4d7f079c679e9ed4 in_seq=0 out_seq=0 2024-07-24T21:52:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:28.381+0000 7fe331ffb640 10 client.26267 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:28 smithi123 ceph-mon[37143]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-07-24T21:52:28.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:28 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:52:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:28 smithi086 ceph-mon[37128]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-07-24T21:52:28.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:28 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:52:28.853 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:28 smithi060 ceph-mon[33000]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-07-24T21:52:28.853 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:28 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-07-24T21:52:29.490 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:29 smithi123 podman[72861]: 2024-07-24 21:52:29.194516355 +0000 UTC m=+3.053217105 container init 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-07-24T21:52:29.490 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:29 smithi123 podman[72861]: 2024-07-24 21:52:29.199715432 +0000 UTC m=+3.058416162 container start 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-24T21:52:29.490 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: starting mds.l at 2024-07-24T21:52:29.490 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:29 smithi123 bash[72861]: 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d 2024-07-24T21:52:29.490 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 21:52:29 smithi123 systemd[1]: Started Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:52:29.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:29 smithi123 ceph-mon[37143]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-07-24T21:52:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:29 smithi086 ceph-mon[37128]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-07-24T21:52:29.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:29 smithi060 ceph-mon[33000]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-07-24T21:52:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.982+0000 7fe331ffb640 1 -- 172.21.15.60:0/1360008759 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe3380a6ec0 con 0x55e60a826420 2024-07-24T21:52:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.982+0000 7fe331ffb640 10 client.26267 handle_command_reply: tid=0 2024-07-24T21:52:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.982+0000 7fe353584740 1 client.26267 shutdown 2024-07-24T21:52:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.982+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 msgr2=0x55e60a7b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.982+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3380c4040 tx=0x7fe3380b3a80 comp rx=0 tx=0).stop 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 shutdown_connections 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e60a826420 0x55e60a828840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e60a7ac370 0x55e60a7a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e60a7a03e0 0x55e60a7a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 --2- 172.21.15.60:0/1360008759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60a79fac0 0x55e60a7b46d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 >> 172.21.15.60:0/1360008759 conn(0x55e60a79d8f0 msgr2=0x55e60a79e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe331ffb640 0 client.26267 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 shutdown_connections 2024-07-24T21:52:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 1 -- 172.21.15.60:0/1360008759 wait complete. 2024-07-24T21:52:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:29.983+0000 7fe353584740 20 client.26267 trim_cache size 0 max 16384 2024-07-24T21:52:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:52:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:52:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:52:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:52:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.001+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 msgr2=0x7fe3340b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.001+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 0x7fe3340b6440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe340005fd0 tx=0x7fe340005f00 comp rx=0 tx=0).stop 2024-07-24T21:52:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.002+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 msgr2=0x7fe3481a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.002+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b6d0 tx=0x7fe33800bb90 comp rx=0 tx=0).stop 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 shutdown_connections 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3f80 0x7fe3340b6440 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe34811ba90 0x7fe3481ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348109f20 0x7fe3481a7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 --2- 172.21.15.60:0/520486281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348109b40 0x7fe3481a78e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 >> 172.21.15.60:0/520486281 conn(0x7fe34806d5a0 msgr2=0x7fe34811bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 shutdown_connections 2024-07-24T21:52:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:30.003+0000 7fe350759640 1 -- 172.21.15.60:0/520486281 wait complete. 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:52:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ignoring --setuser ceph since I am not root 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:52:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ignoring --setgroup ceph since I am not root 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:52:30 smithi123 ceph-mds[72523]: -- 172.21.15.123:0/2584356463 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ba1abf860 con 0x563ba1a9d400 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:52:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: starting mds.f at 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:31 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6832/1697293905,v1:172.21.15.123:6833/1697293905] up:boot 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:31 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:31 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:52:31.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:31 smithi123 ceph-mon[37143]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 1.26k op/s 2024-07-24T21:52:31.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:31 smithi123 ceph-mon[37143]: daemon mds.f restarted 2024-07-24T21:52:31.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:31 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6832/1697293905,v1:172.21.15.123:6833/1697293905] up:boot 2024-07-24T21:52:31.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:31 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:31.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:31 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:52:31.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:31 smithi060 ceph-mon[33000]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 1.26k op/s 2024-07-24T21:52:31.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:31 smithi060 ceph-mon[33000]: daemon mds.f restarted 2024-07-24T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:31 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6832/1697293905,v1:172.21.15.123:6833/1697293905] up:boot 2024-07-24T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:31 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:31 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:31 smithi086 ceph-mon[37128]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 1.26k op/s 2024-07-24T21:52:31.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:31 smithi086 ceph-mon[37128]: daemon mds.f restarted 2024-07-24T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:32 smithi060 ceph-mon[33000]: osdmap e88: 12 total, 12 up, 12 in 2024-07-24T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:32 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6834/2868776697,v1:172.21.15.123:6835/2868776697] up:boot 2024-07-24T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:32 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:32 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:32 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:32.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:32 smithi123 ceph-mon[37143]: osdmap e88: 12 total, 12 up, 12 in 2024-07-24T21:52:32.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:32 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6834/2868776697,v1:172.21.15.123:6835/2868776697] up:boot 2024-07-24T21:52:32.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:32 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:52:32.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:32 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:52:32.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:32 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:32.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:32 smithi086 ceph-mon[37128]: osdmap e88: 12 total, 12 up, 12 in 2024-07-24T21:52:32.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:32 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6834/2868776697,v1:172.21.15.123:6835/2868776697] up:boot 2024-07-24T21:52:32.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:32 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:52:32.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:32 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:52:32.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:32 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:52:33.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:33 smithi060 ceph-mon[33000]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.2 MiB/s wr, 1.16k op/s 2024-07-24T21:52:33.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:33 smithi123 ceph-mon[37143]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.2 MiB/s wr, 1.16k op/s 2024-07-24T21:52:33.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:33 smithi086 ceph-mon[37128]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.2 MiB/s wr, 1.16k op/s 2024-07-24T21:52:34.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:34 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:52:34.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:34 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:52:34.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:34 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:52:34.713 INFO:tasks.workunit.client.0.smithi060.stdout: 451 2124420 1590 1496283 1383 701111 2211 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.4 MiB/s wr, 1.09k op/s 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T21:52:35.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:35 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.4 MiB/s wr, 1.09k op/s 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T21:52:35.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:35 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.4 MiB/s wr, 1.09k op/s 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T21:52:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:35 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:36.449 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.450 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.450 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.450 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.450 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T21:52:36.450 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:36 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:36.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T21:52:36.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:36 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:36 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T21:52:37.446 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:37 smithi086 ceph-mon[37128]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 939 op/s 2024-07-24T21:52:37.492 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:37 smithi123 ceph-mon[37143]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 939 op/s 2024-07-24T21:52:37.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:37 smithi060 ceph-mon[33000]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 939 op/s 2024-07-24T21:52:39.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:39 smithi060 ceph-mon[33000]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 865 op/s 2024-07-24T21:52:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:39 smithi123 ceph-mon[37143]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 865 op/s 2024-07-24T21:52:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:39 smithi086 ceph-mon[37128]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 865 op/s 2024-07-24T21:52:41.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:41 smithi060 ceph-mon[33000]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.3 MiB/s wr, 818 op/s 2024-07-24T21:52:41.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:41 smithi123 ceph-mon[37143]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.3 MiB/s wr, 818 op/s 2024-07-24T21:52:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:41 smithi086 ceph-mon[37128]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.3 MiB/s wr, 818 op/s 2024-07-24T21:52:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:43 smithi060 ceph-mon[33000]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.0 MiB/s wr, 834 op/s 2024-07-24T21:52:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:43 smithi123 ceph-mon[37143]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.0 MiB/s wr, 834 op/s 2024-07-24T21:52:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:43 smithi086 ceph-mon[37128]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 9.0 MiB/s wr, 834 op/s 2024-07-24T21:52:44.713 INFO:tasks.workunit.client.0.smithi060.stdout: 468 1837179 975 1302973 1010 634445 1726 2024-07-24T21:52:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:45 smithi123 ceph-mon[37143]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-07-24T21:52:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:45 smithi086 ceph-mon[37128]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-07-24T21:52:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:45 smithi060 ceph-mon[33000]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 766 op/s 2024-07-24T21:52:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.423+0000 7f20f8a8d640 1 -- 172.21.15.60:0/392981896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 msgr2=0x7f20f4110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.423+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/392981896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f4110ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009ca0 tx=0x7f20ec02a8f0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 -- 172.21.15.60:0/392981896 shutdown_connections 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/392981896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f411de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/392981896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 0x7f20f410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/392981896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f4110ba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 -- 172.21.15.60:0/392981896 >> 172.21.15.60:0/392981896 conn(0x7f20f406d5a0 msgr2=0x7f20f406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 -- 172.21.15.60:0/392981896 shutdown_connections 2024-07-24T21:52:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.425+0000 7f20f8a8d640 1 -- 172.21.15.60:0/392981896 wait complete. 2024-07-24T21:52:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.430+0000 7f20f8a8d640 1 Processor -- start 2024-07-24T21:52:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.430+0000 7f20f8a8d640 1 -- start start 2024-07-24T21:52:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.430+0000 7f20f8a8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f41a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.430+0000 7f20f8a8d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 0x7f20f41a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f8a8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f41ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f8a8d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4121440 con 0x7f20f4109f20 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f8a8d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41212c0 con 0x7f20f411ba90 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f8a8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41215c0 con 0x7f20f4109b40 2024-07-24T21:52:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f41ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.431+0000 7f20f2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f41ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:32826/0 (socket says 172.21.15.60:32826) 2024-07-24T21:52:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.432+0000 7f20f2d9d640 1 -- 172.21.15.60:0/2189106007 learned_addr learned my addr 172.21.15.60:0/2189106007 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.432+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 0x7f20f41a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.433+0000 7f20f259c640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f41a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.435+0000 7f20f259c640 1 -- 172.21.15.60:0/2189106007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 msgr2=0x7f20f41ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.435+0000 7f20f259c640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f41ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.435+0000 7f20f259c640 1 -- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 msgr2=0x7f20f41a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.435+0000 7f20f259c640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 0x7f20f41a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.435+0000 7f20f259c640 1 -- 172.21.15.60:0/2189106007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f41ad9e0 con 0x7f20f4109b40 2024-07-24T21:52:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.436+0000 7f20f259c640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f41a7940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000f80 tx=0x7f20ec02fbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.470+0000 7f20e37fe640 1 -- 172.21.15.60:0/2189106007 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0307f0 con 0x7f20f4109b40 2024-07-24T21:52:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.470+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f41adc70 con 0x7f20f4109b40 2024-07-24T21:52:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.470+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f41112f0 con 0x7f20f4109b40 2024-07-24T21:52:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.472+0000 7f20e37fe640 1 -- 172.21.15.60:0/2189106007 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20ec02d070 con 0x7f20f4109b40 2024-07-24T21:52:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.472+0000 7f20e37fe640 1 -- 172.21.15.60:0/2189106007 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec03bc30 con 0x7f20f4109b40 2024-07-24T21:52:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.482+0000 7f20e37fe640 1 -- 172.21.15.60:0/2189106007 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f20ec045430 con 0x7f20f4109b40 2024-07-24T21:52:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 Processor -- start 2024-07-24T21:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 -- start start 2024-07-24T21:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b291462280 con 0x55b2914f9370 2024-07-24T21:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b291451780 con 0x55b2914ecac0 2024-07-24T21:52:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.499+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2914557c0 con 0x55b2914ed3e0 2024-07-24T21:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.501+0000 7f20f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.501+0000 7f20f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49132/0 (socket says 172.21.15.60:49132) 2024-07-24T21:52:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.501+0000 7f20f1d9b640 1 -- 172.21.15.60:0/3221370505 learned_addr learned my addr 172.21.15.60:0/3221370505 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.504+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.504+0000 7f20f259c640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.504+0000 7f20f2d9d640 1 -- 172.21.15.60:0/3221370505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 msgr2=0x55b2914f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.506+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.506+0000 7f20f2d9d640 1 -- 172.21.15.60:0/3221370505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 msgr2=0x55b2914fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.506+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.506+0000 7f20f2d9d640 1 -- 172.21.15.60:0/3221370505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2911b5790 con 0x55b2914ecac0 2024-07-24T21:52:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.507+0000 7f20f259c640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:52:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.507+0000 7f20e37fe640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 0x7f20dc0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.511+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914ecea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009990 tx=0x7f20e8009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b923e2eae181276 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.511+0000 7f20e37fe640 1 -- 172.21.15.60:0/2189106007 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f20ec105d70 con 0x7f20f4109b40 2024-07-24T21:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.514+0000 7f20e0ff9640 1 -- 172.21.15.60:0/3221370505 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8017070 con 0x55b2914ecac0 2024-07-24T21:52:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.520+0000 7f20e0ff9640 1 -- 172.21.15.60:0/3221370505 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e8013470 con 0x55b2914ecac0 2024-07-24T21:52:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.521+0000 7f20fb7e8740 1 -- 172.21.15.60:0/3221370505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 msgr2=0x55b2914ecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.521+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914ecea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009990 tx=0x7f20e8009e50 comp rx=0 tx=0).stop 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 0x7f20dc0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20fb7e8740 1 -- 172.21.15.60:0/3221370505 shutdown_connections 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/3221370505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914ecea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.557+0000 7f20fb7e8740 1 -- 172.21.15.60:0/3221370505 >> 172.21.15.60:0/3221370505 conn(0x55b2914ea8f0 msgr2=0x55b2914eba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.559+0000 7f20fb7e8740 1 -- 172.21.15.60:0/3221370505 shutdown_connections 2024-07-24T21:52:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.559+0000 7f20fb7e8740 1 -- 172.21.15.60:0/3221370505 wait complete. 2024-07-24T21:52:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.560+0000 7f20fb7e8740 1 -- start start 2024-07-24T21:52:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 Processor -- start 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b29138bab0 con 0x55b2914f9370 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2913d99e0 con 0x55b2914ecac0 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2914fc0a0 con 0x55b2914ed3e0 2024-07-24T21:52:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.561+0000 7f20fb7e8740 10 client.0 _ll_register_callbacks cb 0x55b291161a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47258/0 (socket says 172.21.15.60:47258) 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2160415938 learned_addr learned my addr 172.21.15.60:0/2160415938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f259c640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2160415938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 msgr2=0x55b2914f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2160415938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 msgr2=0x55b2914f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.562+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2160415938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29122e620 con 0x55b2914ed3e0 2024-07-24T21:52:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.563+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20e40115b0 tx=0x7f20e4011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca6144494612782e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.569+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 0x7f20dc0b6410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009960 tx=0x7f20e8004750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.590+0000 7f20d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:52:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.591+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e400f6f0 con 0x55b2914ed3e0 2024-07-24T21:52:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.591+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f20e400f890 con 0x55b2914ed3e0 2024-07-24T21:52:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.591+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b291400ce0 con 0x55b2914ed3e0 2024-07-24T21:52:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.596+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20e400fa90 con 0x55b2914ed3e0 2024-07-24T21:52:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.596+0000 7f20fb7e8740 10 client.26299 fetch_fsmap learned FSMap version 37 2024-07-24T21:52:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.596+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b2915077a0 con 0x55b2914ed3e0 2024-07-24T21:52:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.600+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b291400ce0 con 0x55b2914ed3e0 2024-07-24T21:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.601+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f20e401d400 con 0x55b2914ed3e0 2024-07-24T21:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 10 client.26299 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 20 client.26299 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2915090b0 0x55b29150b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 4 client.26299 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2914641d0 con 0x55b2915090b0 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20f259c640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2915090b0 0x55b29150b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.602+0000 7f20f259c640 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2915090b0 0x55b29150b510 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcbd1bb0941171c server_cookie=66a7ed519afcff9a in_seq=0 out_seq=0 2024-07-24T21:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.603+0000 7f20d9ffb640 10 client.26299 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20d9ffb640 1 -- 172.21.15.60:0/2160415938 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2914641d0 con 0x55b2915090b0 2024-07-24T21:52:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20d9ffb640 10 client.26299 handle_command_reply: tid=0 2024-07-24T21:52:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 client.26299 shutdown 2024-07-24T21:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 msgr2=0x55b2914f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20e40115b0 tx=0x7f20e4011a70 comp rx=0 tx=0).stop 2024-07-24T21:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 shutdown_connections 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b2915090b0 0x55b29150b510 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f5e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2160415938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.609+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 >> 172.21.15.60:0/2160415938 conn(0x55b2914ea8f0 msgr2=0x55b2914f0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.610+0000 7f20d9ffb640 0 client.26299 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.610+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 shutdown_connections 2024-07-24T21:52:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.611+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2160415938 wait complete. 2024-07-24T21:52:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:46.611+0000 7f20fb7e8740 20 client.26299 trim_cache size 0 max 16384 2024-07-24T21:52:46.997 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-24T21:52:46.997 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-24T21:52:46.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:52:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.024+0000 7f20fb7e8740 1 Processor -- start 2024-07-24T21:52:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.024+0000 7f20fb7e8740 1 -- start start 2024-07-24T21:52:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.030+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 0x55b2914eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.031+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 0x55b2914f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.031+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914f9370 0x55b2914f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 0x55b2914eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 0x55b2914eb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49138/0 (socket says 172.21.15.60:49138) 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20f259c640 1 -- 172.21.15.60:0/2902800271 learned_addr learned my addr 172.21.15.60:0/2902800271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.031+0000 7f20f1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 0x55b2914f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41c7d70 con 0x55b2914ecac0 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41c5580 con 0x55b2914ed3e0 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40740d0 con 0x55b2914f9370 2024-07-24T21:52:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.038+0000 7f20f2d9d640 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914f9370 0x55b2914f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2902800271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914f9370 msgr2=0x55b2914f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914f9370 0x55b2914f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2902800271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 msgr2=0x55b2914eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 0x55b2914eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 -- 172.21.15.60:0/2902800271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2914ed7c0 con 0x55b2914ed3e0 2024-07-24T21:52:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.039+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 0x55b2914f46c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f20e401d7a0 tx=0x7f20e400f1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e4be658d8d0e3ed server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.060+0000 7f20dbfff640 1 -- 172.21.15.60:0/2902800271 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e402a070 con 0x55b2914ed3e0 2024-07-24T21:52:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.060+0000 7f20dbfff640 1 -- 172.21.15.60:0/2902800271 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e400f6f0 con 0x55b2914ed3e0 2024-07-24T21:52:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.060+0000 7f20dbfff640 1 -- 172.21.15.60:0/2902800271 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e40148f0 con 0x55b2914ed3e0 2024-07-24T21:52:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.062+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 msgr2=0x55b2914f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.062+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 0x55b2914f46c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f20e401d7a0 tx=0x7f20e400f1d0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 shutdown_connections 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914f9370 0x55b2914f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ed3e0 0x55b2914f46c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/2902800271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914ecac0 0x55b2914eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 >> 172.21.15.60:0/2902800271 conn(0x55b2914ea8f0 msgr2=0x55b2914f5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 shutdown_connections 2024-07-24T21:52:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.080+0000 7f20fb7e8740 1 -- 172.21.15.60:0/2902800271 wait complete. 2024-07-24T21:52:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.081+0000 7f20fb7e8740 1 -- start start 2024-07-24T21:52:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.081+0000 7f20fb7e8740 1 Processor -- start 2024-07-24T21:52:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:32850/0 (socket says 172.21.15.60:32850) 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41181f0 con 0x55b2914f9370 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20f41195f0 con 0x55b2914ecac0 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4118850 con 0x55b2914ed3e0 2024-07-24T21:52:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20f259c640 1 -- 172.21.15.60:0/1557196715 learned_addr learned my addr 172.21.15.60:0/1557196715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.082+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20fb7e8740 10 client.0 _ll_register_callbacks cb 0x7f20e4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 -- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 msgr2=0x55b2914f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 -- 172.21.15.60:0/1557196715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 msgr2=0x55b2914f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 -- 172.21.15.60:0/1557196715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2914fa200 con 0x55b2914ecac0 2024-07-24T21:52:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.083+0000 7f20f259c640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20ec109220 tx=0x7f20ec03b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483ad3a07b4d9e95 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.109+0000 7f20d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.109+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0e4070 con 0x55b2914ecac0 2024-07-24T21:52:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.110+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f20ec0c9020 con 0x55b2914ecac0 2024-07-24T21:52:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.110+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e400f6f0 con 0x55b2914ecac0 2024-07-24T21:52:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.128+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20ec0da050 con 0x55b2914ecac0 2024-07-24T21:52:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.128+0000 7f20fb7e8740 10 client.26288 fetch_fsmap learned FSMap version 37 2024-07-24T21:52:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.128+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b2915011d0 con 0x55b2914ecac0 2024-07-24T21:52:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.129+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20ec0e6070 con 0x55b2914ecac0 2024-07-24T21:52:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.129+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f20ec04c020 con 0x55b2914ecac0 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 10 client.26288 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 20 client.26288 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b291501910 0x55b291575dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 4 client.26288 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:52:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.130+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f20e400f6f0 con 0x55b291501910 2024-07-24T21:52:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.133+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b291501910 0x55b291575dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.134+0000 7f20f1d9b640 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b291501910 0x55b291575dc0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19348abc2c2eeb7f server_cookie=8c69dce442a4e17a in_seq=0 out_seq=0 2024-07-24T21:52:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:47.135+0000 7f20d9ffb640 10 client.26288 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:47.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:52:47.687 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:47 smithi060 ceph-mon[33000]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 861 op/s 2024-07-24T21:52:47.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:52:47.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:52:47.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:47 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:52:47.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:47 smithi123 ceph-mon[37143]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 861 op/s 2024-07-24T21:52:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:52:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:52:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:47 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:52:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:47 smithi086 ceph-mon[37128]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 861 op/s 2024-07-24T21:52:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:52:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:52:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:47 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:52:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:49 smithi123 ceph-mon[37143]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-07-24T21:52:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:49 smithi086 ceph-mon[37128]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-07-24T21:52:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:49 smithi060 ceph-mon[33000]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-07-24T21:52:51.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:51 smithi123 ceph-mon[37143]: pgmap v563: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.48k op/s 2024-07-24T21:52:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:51 smithi086 ceph-mon[37128]: pgmap v563: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.48k op/s 2024-07-24T21:52:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:51 smithi060 ceph-mon[33000]: pgmap v563: 97 pgs: 97 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.48k op/s 2024-07-24T21:52:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.193+0000 7f20d9ffb640 1 -- 172.21.15.60:0/1557196715 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f20e400f6f0 con 0x55b291501910 2024-07-24T21:52:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.193+0000 7f20d9ffb640 10 client.26288 handle_command_reply: tid=0 2024-07-24T21:52:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 client.26288 shutdown 2024-07-24T21:52:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 msgr2=0x55b2914fac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20ec109220 tx=0x7f20ec03b730 comp rx=0 tx=0).stop 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 shutdown_connections 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b291501910 0x55b291575dc0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b2914f9370 0x55b2914f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2914ed3e0 0x55b2914f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 --2- 172.21.15.60:0/1557196715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2914ecac0 0x55b2914fac80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 >> 172.21.15.60:0/1557196715 conn(0x55b2914ea8f0 msgr2=0x55b2914efc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20d9ffb640 0 client.26288 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.194+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 shutdown_connections 2024-07-24T21:52:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.195+0000 7f20fb7e8740 1 -- 172.21.15.60:0/1557196715 wait complete. 2024-07-24T21:52:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.195+0000 7f20fb7e8740 20 client.26288 trim_cache size 0 max 16384 2024-07-24T21:52:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:52:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:52:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:52:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:52:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.201+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 msgr2=0x7f20dc0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.201+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 0x7f20dc0b6410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009960 tx=0x7f20e8004750 comp rx=0 tx=0).stop 2024-07-24T21:52:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.203+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 msgr2=0x7f20f41a7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.203+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f41a7940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000f80 tx=0x7f20ec02fbc0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 shutdown_connections 2024-07-24T21:52:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f20dc0b3f50 0x7f20dc0b6410 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f411ba90 0x7f20f41ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f4109f20 0x7f20f41a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 --2- 172.21.15.60:0/2189106007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20f4109b40 0x7f20f41a7940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.207+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 >> 172.21.15.60:0/2189106007 conn(0x7f20f406d5a0 msgr2=0x7f20f411bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.217+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 shutdown_connections 2024-07-24T21:52:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:52:52.217+0000 7f20f8a8d640 1 -- 172.21.15.60:0/2189106007 wait complete. 2024-07-24T21:52:53.007 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:52:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:52:53.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:53 smithi123 ceph-mon[37143]: pgmap v564: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2024-07-24T21:52:53.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:53 smithi060 ceph-mon[33000]: pgmap v564: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2024-07-24T21:52:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:53 smithi086 ceph-mon[37128]: pgmap v564: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2024-07-24T21:52:54.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:54.713 INFO:tasks.workunit.client.0.smithi060.stdout: 482 1620248 541 1157628 1016 579719 1670 2024-07-24T21:52:54.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:52:55.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:55 smithi060 ceph-mon[33000]: pgmap v565: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2024-07-24T21:52:55.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:55 smithi123 ceph-mon[37143]: pgmap v565: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2024-07-24T21:52:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:55 smithi086 ceph-mon[37128]: pgmap v565: 97 pgs: 97 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 19 MiB/s wr, 1.56k op/s 2024-07-24T21:52:57.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.444+0000 7fa5772a8640 1 -- 172.21.15.60:0/4141019564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109f20 msgr2=0x7fa57010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:57.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.444+0000 7fa5772a8640 1 --2- 172.21.15.60:0/4141019564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109f20 0x7fa57010a360 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa568009ca0 tx=0x7fa56802a960 comp rx=0 tx=0).stop 2024-07-24T21:52:57.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.452+0000 7fa5772a8640 1 -- 172.21.15.60:0/4141019564 shutdown_connections 2024-07-24T21:52:57.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.452+0000 7fa5772a8640 1 --2- 172.21.15.60:0/4141019564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57011ba90 0x7fa57011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:57.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.452+0000 7fa5772a8640 1 --2- 172.21.15.60:0/4141019564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109f20 0x7fa57010a360 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:57.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.452+0000 7fa5772a8640 1 --2- 172.21.15.60:0/4141019564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109b40 0x7fa570110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.452+0000 7fa5772a8640 1 -- 172.21.15.60:0/4141019564 >> 172.21.15.60:0/4141019564 conn(0x7fa57006d980 msgr2=0x7fa57006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:57.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.459+0000 7fa5772a8640 1 -- 172.21.15.60:0/4141019564 shutdown_connections 2024-07-24T21:52:57.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.460+0000 7fa5772a8640 1 -- 172.21.15.60:0/4141019564 wait complete. 2024-07-24T21:52:57.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.461+0000 7fa5772a8640 1 Processor -- start 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 -- start start 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109b40 0x7fa5701a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57011ba90 0x7fa5701ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa570121550 con 0x7fa570109b40 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701213d0 con 0x7fa57011ba90 2024-07-24T21:52:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.463+0000 7fa5772a8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701216d0 con 0x7fa570109f20 2024-07-24T21:52:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:33116/0 (socket says 172.21.15.60:33116) 2024-07-24T21:52:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 -- 172.21.15.60:0/1155509818 learned_addr learned my addr 172.21.15.60:0/1155509818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa575044640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109b40 0x7fa5701a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 -- 172.21.15.60:0/1155509818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57011ba90 msgr2=0x7fa5701ad320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:52:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57011ba90 0x7fa5701ad320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 -- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109b40 msgr2=0x7fa5701a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109b40 0x7fa5701a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.465+0000 7fa574843640 1 -- 172.21.15.60:0/1155509818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5701adaf0 con 0x7fa570109f20 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.466+0000 7fa574843640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa56802aed0 tx=0x7fa56802f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.467+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568034030 con 0x7fa570109f20 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.467+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa568030b10 con 0x7fa570109f20 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.467+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568004680 con 0x7fa570109f20 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.467+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701add80 con 0x7fa570109f20 2024-07-24T21:52:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.467+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa570111270 con 0x7fa570109f20 2024-07-24T21:52:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.469+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554005290 con 0x7fa570109f20 2024-07-24T21:52:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.479+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa568037020 con 0x7fa570109f20 2024-07-24T21:52:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.481+0000 7fa5667fc640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 0x7fa55c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:57.484 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:57 smithi060 ceph-mon[33000]: pgmap v566: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-07-24T21:52:57.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.484+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7fa568104b00 con 0x7fa570109f20 2024-07-24T21:52:57.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.484+0000 7fa575044640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 0x7fa55c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:57.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.484+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa5681345e0 con 0x7fa570109f20 2024-07-24T21:52:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:57.485+0000 7fa575044640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 0x7fa55c0b6410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa56000bbe0 tx=0x7fa56000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:57 smithi123 ceph-mon[37143]: pgmap v566: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-07-24T21:52:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:57 smithi086 ceph-mon[37128]: pgmap v566: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.65k op/s 2024-07-24T21:52:58.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.048+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa554005580 con 0x7fa570109f20 2024-07-24T21:52:58.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.058+0000 7fa5667fc640 1 -- 172.21.15.60:0/1155509818 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11922 (secure 0 0 0) 0x7fa568004820 con 0x7fa570109f20 2024-07-24T21:52:58.063 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:52:58.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"btime":"2024-07-24T21:52:31:056096+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:52:31.055984+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:52:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 37 2024-07-24T21:52:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 msgr2=0x7fa55c0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 0x7fa55c0b6410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa56000bbe0 tx=0x7fa56000b390 comp rx=0 tx=0).stop 2024-07-24T21:52:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 msgr2=0x7fa5701a7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa56802aed0 tx=0x7fa56802f8f0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 shutdown_connections 2024-07-24T21:52:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa55c0b3f50 0x7fa55c0b6410 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57011ba90 0x7fa5701ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa570109f20 0x7fa5701a7f80 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa56802aed0 tx=0x7fa56802f8f0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 --2- 172.21.15.60:0/1155509818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa570109b40 0x7fa5701a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.066+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 >> 172.21.15.60:0/1155509818 conn(0x7fa57006d980 msgr2=0x7fa57011bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.069+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 shutdown_connections 2024-07-24T21:52:58.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.069+0000 7fa5772a8640 1 -- 172.21.15.60:0/1155509818 wait complete. 2024-07-24T21:52:58.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:58 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1155509818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:52:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.574+0000 7f73effff640 1 -- 172.21.15.60:0/1095468852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f0109da0 msgr2=0x7f73f010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.575+0000 7f73effff640 1 --2- 172.21.15.60:0/1095468852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f0109da0 0x7f73f010a180 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ca0 tx=0x7f73e802a8f0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.576+0000 7f73effff640 1 -- 172.21.15.60:0/1095468852 shutdown_connections 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.576+0000 7f73effff640 1 --2- 172.21.15.60:0/1095468852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f010c5a0 0x7f73f0115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.576+0000 7f73effff640 1 --2- 172.21.15.60:0/1095468852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f010bc00 0x7f73f010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.576+0000 7f73effff640 1 --2- 172.21.15.60:0/1095468852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f0109da0 0x7f73f010a180 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.576+0000 7f73effff640 1 -- 172.21.15.60:0/1095468852 >> 172.21.15.60:0/1095468852 conn(0x7f73f006c8e0 msgr2=0x7f73f006ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.577+0000 7f73effff640 1 -- 172.21.15.60:0/1095468852 shutdown_connections 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.580+0000 7f73effff640 1 -- 172.21.15.60:0/1095468852 wait complete. 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.582+0000 7f73effff640 1 Processor -- start 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.582+0000 7f73effff640 1 -- start start 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.583+0000 7f73effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 0x7f73f019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.583+0000 7f73effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f010bc00 0x7f73f019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 0x7f73f019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 0x7f73f019f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38790/0 (socket says 172.21.15.60:38790) 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73eeffd640 1 -- 172.21.15.60:0/1247357105 learned_addr learned my addr 172.21.15.60:0/1247357105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 0x7f73f01a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0119490 con 0x7f73f010c5a0 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0119310 con 0x7f73f0109da0 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.584+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0119610 con 0x7f73f010bc00 2024-07-24T21:52:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.585+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 0x7f73f01a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 -- 172.21.15.60:0/1247357105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f010bc00 msgr2=0x7f73f019fe20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f010bc00 0x7f73f019fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 -- 172.21.15.60:0/1247357105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 msgr2=0x7f73f019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 0x7f73f019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f01a5990 con 0x7f73f010c5a0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.586+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 0x7f73f01a51c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004640 tx=0x7f73e4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.587+0000 7f73f496f640 1 -- 172.21.15.60:0/1247357105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4014030 con 0x7f73f010c5a0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.587+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f01a5c80 con 0x7f73f010c5a0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.587+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f0073fe0 con 0x7f73f010c5a0 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.597+0000 7f73f7d22740 1 Processor -- start 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 -- start start 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 0x557afb165be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 0x557afb171b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557afb0dc960 con 0x557afb166120 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557afb0cbe60 con 0x557afb165800 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f7d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557afb0cfea0 con 0x557afb1720b0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f496f640 1 -- 172.21.15.60:0/1247357105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e40141d0 con 0x7f73f010c5a0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f496f640 1 -- 172.21.15.60:0/1247357105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004b20 con 0x7f73f010c5a0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.599+0000 7f73f496f640 1 -- 172.21.15.60:0/1247357105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f73e4023030 con 0x7f73f010c5a0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.600+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 0x557afb171b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.600+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 0x557afb171b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59960/0 (socket says 172.21.15.60:59960) 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.600+0000 7f73eeffd640 1 -- 172.21.15.60:0/3261475741 learned_addr learned my addr 172.21.15.60:0/3261475741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 0x557afb165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 -- 172.21.15.60:0/3261475741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 msgr2=0x557afb1744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 -- 172.21.15.60:0/3261475741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 msgr2=0x557afb171b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 0x557afb171b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.601+0000 7f73ee7fc640 1 -- 172.21.15.60:0/3261475741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afb0552d0 con 0x557afb165800 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.603+0000 7f73f496f640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 0x7f73d80b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.603+0000 7f73f496f640 1 -- 172.21.15.60:0/1247357105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f73e40ec810 con 0x7f73f010c5a0 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.604+0000 7f73eeffd640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 0x7f73d80b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.605+0000 7f73eeffd640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 0x7f73d80b64f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fc0 tx=0x7f73e8005eb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.606+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 0x557afb165be0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73f010e710 tx=0x7f73e000b060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7582625a30067e71 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.609+0000 7f73d5ffb640 1 -- 172.21.15.60:0/3261475741 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0011040 con 0x557afb165800 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.609+0000 7f73d5ffb640 1 -- 172.21.15.60:0/3261475741 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e00065a0 con 0x557afb165800 2024-07-24T21:52:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.609+0000 7f73d5ffb640 1 -- 172.21.15.60:0/3261475741 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0018430 con 0x557afb165800 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 -- 172.21.15.60:0/3261475741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 msgr2=0x557afb165be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 0x557afb165be0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73f010e710 tx=0x7f73e000b060 comp rx=0 tx=0).stop 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 -- 172.21.15.60:0/3261475741 shutdown_connections 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb166120 0x557afb171b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3261475741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb165800 0x557afb165be0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73f010e710 tx=0x7f73e000b060 comp rx=0 tx=0).stop 2024-07-24T21:52:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.612+0000 7f73f7d22740 1 -- 172.21.15.60:0/3261475741 >> 172.21.15.60:0/3261475741 conn(0x557afb163630 msgr2=0x557afb1647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.614+0000 7f73f7d22740 1 -- 172.21.15.60:0/3261475741 shutdown_connections 2024-07-24T21:52:58.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.618+0000 7f73f7d22740 1 -- 172.21.15.60:0/3261475741 wait complete. 2024-07-24T21:52:58.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.618+0000 7f73f7d22740 1 -- start start 2024-07-24T21:52:58.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 Processor -- start 2024-07-24T21:52:58.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 0x557afb16f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb1720b0 0x557afb16f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 0x557afb17e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557afaf2f700 con 0x557afb1720b0 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557afb06e5a0 con 0x557afb16fd10 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557afb174cd0 con 0x557afb166120 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.619+0000 7f73f7d22740 10 client.0 _ll_register_callbacks cb 0x557afb08e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.620+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 0x557afb16f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.620+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 0x557afb16f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:33136/0 (socket says 172.21.15.60:33136) 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.620+0000 7f73eeffd640 1 -- 172.21.15.60:0/90894129 learned_addr learned my addr 172.21.15.60:0/90894129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:58.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.621+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 0x557afb17e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.621+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb1720b0 0x557afb16f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.622+0000 7f73ef7fe640 1 -- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 msgr2=0x557afb16f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.622+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 0x557afb16f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.622+0000 7f73ef7fe640 1 -- 172.21.15.60:0/90894129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb1720b0 msgr2=0x557afb16f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.622+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb1720b0 0x557afb16f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.622+0000 7f73ef7fe640 1 -- 172.21.15.60:0/90894129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afb061630 con 0x557afb16fd10 2024-07-24T21:52:58.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.624+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 0x557afb17e090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73e40e9f80 tx=0x7f73e40b3bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92e87ed370552d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.624+0000 7f73ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:58.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.624+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40bc4f0 con 0x557afb16fd10 2024-07-24T21:52:58.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.624+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f73e40bc270 con 0x557afb16fd10 2024-07-24T21:52:58.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.625+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e00065a0 con 0x557afb16fd10 2024-07-24T21:52:58.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.627+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73e40b2730 con 0x557afb16fd10 2024-07-24T21:52:58.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.627+0000 7f73f7d22740 10 client.26300 fetch_fsmap learned FSMap version 37 2024-07-24T21:52:58.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.627+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557afae2c650 con 0x557afb16fd10 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e00065a0 con 0x557afb16fd10 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f73e401cce0 con 0x557afb16fd10 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 10 client.26300 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 20 client.26300 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb193390 0x557afb1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 4 client.26300 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:52:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.628+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557afb0de8b0 con 0x557afb193390 2024-07-24T21:52:58.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.632+0000 7f73eeffd640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb193390 0x557afb1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.644+0000 7f73eeffd640 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb193390 0x557afb1957b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774880fa94f73a82 server_cookie=89bd2cdb9ea5a2c7 in_seq=0 out_seq=0 2024-07-24T21:52:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.644+0000 7f73ceffd640 10 client.26300 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:58.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.650+0000 7f73ceffd640 1 -- 172.21.15.60:0/90894129 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557afb0de8b0 con 0x557afb193390 2024-07-24T21:52:58.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.650+0000 7f73ceffd640 10 client.26300 handle_command_reply: tid=0 2024-07-24T21:52:58.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.651+0000 7f73f7d22740 1 client.26300 shutdown 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.652+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 msgr2=0x557afb17e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.652+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 0x557afb17e090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73e40e9f80 tx=0x7f73e40b3bd0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 shutdown_connections 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb193390 0x557afb1957b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb16fd10 0x557afb17e090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb1720b0 0x557afb16f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 --2- 172.21.15.60:0/90894129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb166120 0x557afb16f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 >> 172.21.15.60:0/90894129 conn(0x557afb163630 msgr2=0x557afb1691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.653+0000 7f73ceffd640 0 client.26300 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.656+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 shutdown_connections 2024-07-24T21:52:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.659+0000 7f73f7d22740 1 -- 172.21.15.60:0/90894129 wait complete. 2024-07-24T21:52:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.660+0000 7f73f7d22740 20 client.26300 trim_cache size 0 max 16384 2024-07-24T21:52:58.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.724+0000 7f73f7d22740 1 Processor -- start 2024-07-24T21:52:58.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.724+0000 7f73f7d22740 1 -- start start 2024-07-24T21:52:58.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb1643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb17de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc011290 con 0x557afb16fd10 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00eaa0 con 0x557afb166120 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73f7d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00c8f0 con 0x557afb1720b0 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:33152/0 (socket says 172.21.15.60:33152) 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73ef7fe640 1 -- 172.21.15.60:0/3974252009 learned_addr learned my addr 172.21.15.60:0/3974252009 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:58.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.725+0000 7f73eeffd640 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb1643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 -- 172.21.15.60:0/3974252009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 msgr2=0x557afb1643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb1643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 -- 172.21.15.60:0/3974252009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 msgr2=0x557afb17de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb17de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 -- 172.21.15.60:0/3974252009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afb16e1f0 con 0x557afb1720b0 2024-07-24T21:52:58.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.726+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b3d80 tx=0x7f73e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f02b6bdf24c413f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.730+0000 7f73d57fa640 1 -- 172.21.15.60:0/3974252009 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40b32a0 con 0x557afb1720b0 2024-07-24T21:52:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.730+0000 7f73d57fa640 1 -- 172.21.15.60:0/3974252009 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e40bc4f0 con 0x557afb1720b0 2024-07-24T21:52:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.730+0000 7f73d57fa640 1 -- 172.21.15.60:0/3974252009 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40c8d40 con 0x557afb1720b0 2024-07-24T21:52:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.732+0000 7f73f7d22740 1 -- 172.21.15.60:0/3974252009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 msgr2=0x557afb169050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.732+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b3d80 tx=0x7f73e40046c0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.732+0000 7f73f7d22740 1 -- 172.21.15.60:0/3974252009 shutdown_connections 2024-07-24T21:52:58.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.732+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb169050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.733+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb17de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.733+0000 7f73f7d22740 1 --2- 172.21.15.60:0/3974252009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb1643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.733+0000 7f73f7d22740 1 -- 172.21.15.60:0/3974252009 >> 172.21.15.60:0/3974252009 conn(0x557afb163630 msgr2=0x557afb168940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.737+0000 7f73f7d22740 1 -- 172.21.15.60:0/3974252009 shutdown_connections 2024-07-24T21:52:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.737+0000 7f73f7d22740 1 -- 172.21.15.60:0/3974252009 wait complete. 2024-07-24T21:52:58.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.738+0000 7f73f7d22740 1 -- start start 2024-07-24T21:52:58.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.738+0000 7f73f7d22740 1 Processor -- start 2024-07-24T21:52:58.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb176750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73f7d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38824/0 (socket says 172.21.15.60:38824) 2024-07-24T21:52:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73eeffd640 1 -- 172.21.15.60:0/2116484042 learned_addr learned my addr 172.21.15.60:0/2116484042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:52:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.740+0000 7f73ef7fe640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.740+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb176750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:58 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1155509818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:52:58.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.739+0000 7f73f7d22740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00cf60 con 0x557afb16fd10 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00e3d0 con 0x557afb166120 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc00d630 con 0x557afb1720b0 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73f7d22740 10 client.0 _ll_register_callbacks cb 0x7f73e40af410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73eeffd640 1 -- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 msgr2=0x557afb1fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73eeffd640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73eeffd640 1 -- 172.21.15.60:0/2116484042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 msgr2=0x557afb176750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:52:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73eeffd640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb176750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.741+0000 7f73eeffd640 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afb1fd590 con 0x557afb166120 2024-07-24T21:52:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.742+0000 7f73eeffd640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d68b8f436fd398 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:52:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.744+0000 7f73ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:52:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.745+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8003e00 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.745+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e40bc4f0 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.745+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8521+0+0 (secure 0 0 0) 0x7f73e8004040 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.745+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73e8004240 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.746+0000 7f73f7d22740 10 client.26306 fetch_fsmap learned FSMap version 37 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.746+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557afb1fe790 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.746+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e40bc4f0 con 0x557afb166120 2024-07-24T21:52:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.746+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f73e80305f0 con 0x557afb166120 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 10 client.26306 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 20 client.26306 populate_metadata read hostname 'smithi060' 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb1fef20 0x557afb201340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 4 client.26306 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:52:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.747+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557afb0da350 con 0x557afb1fef20 2024-07-24T21:52:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.748+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb1fef20 0x557afb201340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:52:58.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.751+0000 7f73ee7fc640 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb1fef20 0x557afb201340 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e988db1b89b22a22 server_cookie=36fd652390012277 in_seq=0 out_seq=0 2024-07-24T21:52:58.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.751+0000 7f73ceffd640 10 client.26306 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:58.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.754+0000 7f73ceffd640 1 -- 172.21.15.60:0/2116484042 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557afb0da350 con 0x557afb1fef20 2024-07-24T21:52:58.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.754+0000 7f73ceffd640 10 client.26306 handle_command_reply: tid=0 2024-07-24T21:52:58.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.754+0000 7f73f7d22740 1 client.26306 shutdown 2024-07-24T21:52:58.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 msgr2=0x557afb176210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80095c0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 shutdown_connections 2024-07-24T21:52:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557afb1fef20 0x557afb201340 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557afb1720b0 0x557afb1fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557afb16fd10 0x557afb176750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 --2- 172.21.15.60:0/2116484042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557afb166120 0x557afb176210 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80095c0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 >> 172.21.15.60:0/2116484042 conn(0x557afb163630 msgr2=0x557afb1710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73ceffd640 0 client.26306 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:52:58.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 shutdown_connections 2024-07-24T21:52:58.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.762+0000 7f73f7d22740 1 -- 172.21.15.60:0/2116484042 wait complete. 2024-07-24T21:52:58.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.763+0000 7f73f7d22740 20 client.26306 trim_cache size 0 max 16384 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "69137c0b-3e1b-4039-932f-7243537f927e": { 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "69137c0b-3e1b-4039-932f-7243537f927e", 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:52:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:52:58.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.776+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 msgr2=0x7f73d80b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.776+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 0x7f73d80b64f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fc0 tx=0x7f73e8005eb0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.776+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 msgr2=0x7f73f01a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:52:58.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.776+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 0x7f73f01a51c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004640 tx=0x7f73e4009c40 comp rx=0 tx=0).stop 2024-07-24T21:52:58.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 shutdown_connections 2024-07-24T21:52:58.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f73d80b4030 0x7f73d80b64f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73f010c5a0 0x7f73f01a51c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f010bc00 0x7f73f019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 --2- 172.21.15.60:0/1247357105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f0109da0 0x7f73f019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:52:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.779+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 >> 172.21.15.60:0/1247357105 conn(0x7f73f006c8e0 msgr2=0x7f73f010cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:52:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.780+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 shutdown_connections 2024-07-24T21:52:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:52:58.780+0000 7f73effff640 1 -- 172.21.15.60:0/1247357105 wait complete. 2024-07-24T21:52:58.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:58 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1155509818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:52:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:52:59 smithi123 ceph-mon[37143]: pgmap v567: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-07-24T21:52:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:52:59 smithi086 ceph-mon[37128]: pgmap v567: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-07-24T21:52:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:52:59 smithi060 ceph-mon[33000]: pgmap v567: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-07-24T21:53:00.489 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-24T21:53:00.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-24T21:53:00.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:00 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:53:00.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:00 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:53:00.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:00 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:53:01.034 INFO:tasks.cephfs.filesystem:scrub status for tag:69137c0b-3e1b-4039-932f-7243537f927e - {'path': '/', 'tag': '69137c0b-3e1b-4039-932f-7243537f927e', 'options': 'recursive,force'} 2024-07-24T21:53:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:53:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:53:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:01 smithi123 ceph-mon[37143]: pgmap v568: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-07-24T21:53:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:53:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:53:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:01 smithi086 ceph-mon[37128]: pgmap v568: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-07-24T21:53:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:53:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:53:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:01 smithi060 ceph-mon[33000]: pgmap v568: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 1.19k op/s 2024-07-24T21:53:03.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:03 smithi123 ceph-mon[37143]: pgmap v569: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 844 op/s 2024-07-24T21:53:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:03 smithi086 ceph-mon[37128]: pgmap v569: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 844 op/s 2024-07-24T21:53:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:03 smithi060 ceph-mon[33000]: pgmap v569: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 844 op/s 2024-07-24T21:53:04.713 INFO:tasks.workunit.client.0.smithi060.stdout: 508 2073443 1312 1477860 1459 761826 2722 2024-07-24T21:53:05.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:05 smithi123 ceph-mon[37143]: pgmap v570: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 704 op/s 2024-07-24T21:53:05.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:05 smithi086 ceph-mon[37128]: pgmap v570: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 704 op/s 2024-07-24T21:53:05.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:05 smithi060 ceph-mon[33000]: pgmap v570: 97 pgs: 97 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.4 MiB/s wr, 704 op/s 2024-07-24T21:53:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.945+0000 7fe0125cb640 1 -- 172.21.15.60:0/4052010902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109f20 msgr2=0x7fe00c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.945+0000 7fe0125cb640 1 --2- 172.21.15.60:0/4052010902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109f20 0x7fe00c10a360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe004009be0 tx=0x7fe00402a6f0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 -- 172.21.15.60:0/4052010902 shutdown_connections 2024-07-24T21:53:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 --2- 172.21.15.60:0/4052010902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 --2- 172.21.15.60:0/4052010902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109f20 0x7fe00c10a360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 --2- 172.21.15.60:0/4052010902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109b40 0x7fe00c110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 -- 172.21.15.60:0/4052010902 >> 172.21.15.60:0/4052010902 conn(0x7fe00c06d5a0 msgr2=0x7fe00c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 -- 172.21.15.60:0/4052010902 shutdown_connections 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.981+0000 7fe0125cb640 1 -- 172.21.15.60:0/4052010902 wait complete. 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.982+0000 7fe0125cb640 1 Processor -- start 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.982+0000 7fe0125cb640 1 -- start start 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109b40 0x7fe00c1a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 0x7fe00c1a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c121310 con 0x7fe00c109f20 2024-07-24T21:53:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c121190 con 0x7fe00c11ba90 2024-07-24T21:53:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe0125cb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c121490 con 0x7fe00c109b40 2024-07-24T21:53:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe010b68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe010b68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54780/0 (socket says 172.21.15.60:54780) 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe010b68640 1 -- 172.21.15.60:0/2407492211 learned_addr learned my addr 172.21.15.60:0/2407492211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 0x7fe00c1a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.983+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109b40 0x7fe00c1a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.984+0000 7fe010b68640 1 -- 172.21.15.60:0/2407492211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109b40 msgr2=0x7fe00c1a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.984+0000 7fe010b68640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109b40 0x7fe00c1a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.984+0000 7fe010b68640 1 -- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 msgr2=0x7fe00c1a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.989+0000 7fe010b68640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 0x7fe00c1a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.989+0000 7fe010b68640 1 -- 172.21.15.60:0/2407492211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1ad970 con 0x7fe00c11ba90 2024-07-24T21:53:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.989+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 0x7fe00c1a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:05.990+0000 7fe010b68640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c970 tx=0x7fdffc00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.007+0000 7fe0097fa640 1 -- 172.21.15.60:0/2407492211 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc004230 con 0x7fe00c11ba90 2024-07-24T21:53:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.008+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1adc60 con 0x7fe00c11ba90 2024-07-24T21:53:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.008+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c07ada0 con 0x7fe00c11ba90 2024-07-24T21:53:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.008+0000 7fe0097fa640 1 -- 172.21.15.60:0/2407492211 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc00ae50 con 0x7fe00c11ba90 2024-07-24T21:53:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.008+0000 7fe0097fa640 1 -- 172.21.15.60:0/2407492211 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc003d90 con 0x7fe00c11ba90 2024-07-24T21:53:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.013+0000 7fe0097fa640 1 -- 172.21.15.60:0/2407492211 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fdffc00f470 con 0x7fe00c11ba90 2024-07-24T21:53:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.015+0000 7fe0097fa640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 0x7fdff40b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.016+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 0x7fdff40b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.017+0000 7fe0153fe740 1 Processor -- start 2024-07-24T21:53:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 -- start start 2024-07-24T21:53:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2b5f280 con 0x55d9c2bea3c0 2024-07-24T21:53:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2b4e780 con 0x55d9c2be9aa0 2024-07-24T21:53:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe0153fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2b527c0 con 0x55d9c2bf6350 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.018+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47624/0 (socket says 172.21.15.60:47624) 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54794/0 (socket says 172.21.15.60:54794) 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00b7fe640 1 -- 172.21.15.60:0/2109424920 learned_addr learned my addr 172.21.15.60:0/2109424920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe010b68640 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 -- 172.21.15.60:0/2109424920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 msgr2=0x55d9c2bf8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 -- 172.21.15.60:0/2109424920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 msgr2=0x55d9c2bf5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.019+0000 7fe00bfff640 1 -- 172.21.15.60:0/2109424920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c28b2790 con 0x55d9c2be9aa0 2024-07-24T21:53:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.020+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe000004280 tx=0x7fe00001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0ef1f06021cb98 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.052+0000 7fe00bfff640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 0x7fdff40b6410 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe0000118d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.052+0000 7fe0097fa640 1 -- 172.21.15.60:0/2407492211 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7fdffc0e4c10 con 0x7fe00c11ba90 2024-07-24T21:53:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.060+0000 7fdffa7fc640 1 -- 172.21.15.60:0/2109424920 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000025070 con 0x55d9c2be9aa0 2024-07-24T21:53:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.061+0000 7fdffa7fc640 1 -- 172.21.15.60:0/2109424920 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe000018070 con 0x55d9c2be9aa0 2024-07-24T21:53:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.062+0000 7fe0153fe740 1 -- 172.21.15.60:0/2109424920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 msgr2=0x55d9c2be9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.062+0000 7fe0153fe740 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe000004280 tx=0x7fe00001b000 comp rx=0 tx=0).stop 2024-07-24T21:53:06.096 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 -- 172.21.15.60:0/2109424920 shutdown_connections 2024-07-24T21:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 --2- 172.21.15.60:0/2109424920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2be9e80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 -- 172.21.15.60:0/2109424920 >> 172.21.15.60:0/2109424920 conn(0x55d9c2be78d0 msgr2=0x55d9c2be8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 -- 172.21.15.60:0/2109424920 shutdown_connections 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 -- 172.21.15.60:0/2109424920 wait complete. 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.116+0000 7fe0153fe740 1 -- start start 2024-07-24T21:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 Processor -- start 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2a88ab0 con 0x55d9c2bea3c0 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2ad69e0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9c2c0a030 con 0x55d9c2bf6350 2024-07-24T21:53:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.117+0000 7fe0153fe740 10 client.0 _ll_register_callbacks cb 0x55d9c285ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54804/0 (socket says 172.21.15.60:54804) 2024-07-24T21:53:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 -- 172.21.15.60:0/1499039674 learned_addr learned my addr 172.21.15.60:0/1499039674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe010b68640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 -- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 msgr2=0x55d9c2bf3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 -- 172.21.15.60:0/1499039674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 msgr2=0x55d9c2bf2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe00bfff640 1 -- 172.21.15.60:0/1499039674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c292b620 con 0x55d9c2be9aa0 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.118+0000 7fe010b68640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.119+0000 7fe00bfff640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098a0 tx=0x7fe000005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593a06a1fc2c32f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.161+0000 7fdfef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:53:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.161+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000025070 con 0x55d9c2be9aa0 2024-07-24T21:53:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.162+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7fe00000f760 con 0x55d9c2be9aa0 2024-07-24T21:53:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.162+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9c2afdce0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.164+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe000037050 con 0x55d9c2be9aa0 2024-07-24T21:53:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.164+0000 7fe0153fe740 10 client.26315 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.165+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d9c2c15840 con 0x55d9c2be9aa0 2024-07-24T21:53:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.170+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d9c2afdce0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe00001d070 con 0x55d9c2be9aa0 2024-07-24T21:53:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 10 client.26315 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 10 client.26315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 20 client.26315 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c17160 0x55d9c2c19620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 4 client.26315 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.171+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c2b611d0 con 0x55d9c2c17160 2024-07-24T21:53:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.177+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c17160 0x55d9c2c19620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.182+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c17160 0x55d9c2c19620 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472aebd809190409 server_cookie=363b654011ff569 in_seq=0 out_seq=0 2024-07-24T21:53:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.183+0000 7fdfef7fe640 10 client.26315 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.187+0000 7fdfef7fe640 1 -- 172.21.15.60:0/1499039674 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9c2b611d0 con 0x55d9c2c17160 2024-07-24T21:53:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.187+0000 7fdfef7fe640 10 client.26315 handle_command_reply: tid=0 2024-07-24T21:53:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.188+0000 7fe0153fe740 1 client.26315 shutdown 2024-07-24T21:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 msgr2=0x55d9c2bf5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098a0 tx=0x7fe000005590 comp rx=0 tx=0).stop 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 shutdown_connections 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c17160 0x55d9c2c19620 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2bf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2bf2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1499039674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf5a40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.193+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 >> 172.21.15.60:0/1499039674 conn(0x55d9c2be78d0 msgr2=0x55d9c2bed470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.194+0000 7fdfef7fe640 0 client.26315 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.194+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 shutdown_connections 2024-07-24T21:53:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.195+0000 7fe0153fe740 1 -- 172.21.15.60:0/1499039674 wait complete. 2024-07-24T21:53:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.195+0000 7fe0153fe740 20 client.26315 trim_cache size 0 max 16384 2024-07-24T21:53:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.577+0000 7fe0153fe740 1 Processor -- start 2024-07-24T21:53:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.577+0000 7fe0153fe740 1 -- start start 2024-07-24T21:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.637+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.637+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.637+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2be8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe0153fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00112a0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54808/0 (socket says 172.21.15.60:54808) 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 -- 172.21.15.60:0/1216915248 learned_addr learned my addr 172.21.15.60:0/1216915248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdff000eab0 con 0x55d9c2bea3c0 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdff000c9b0 con 0x55d9c2bf6350 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe010b68640 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2be8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 -- 172.21.15.60:0/1216915248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 msgr2=0x55d9c2be8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2be8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 -- 172.21.15.60:0/1216915248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2be9aa0 msgr2=0x55d9c2bf3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 -- 172.21.15.60:0/1216915248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c2bf2d00 con 0x55d9c2bea3c0 2024-07-24T21:53:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.638+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe004009b30 tx=0x7fe004009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1511198e5705a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.648+0000 7fdff9ffb640 1 -- 172.21.15.60:0/1216915248 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004036040 con 0x55d9c2bea3c0 2024-07-24T21:53:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.648+0000 7fdff9ffb640 1 -- 172.21.15.60:0/1216915248 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004049020 con 0x55d9c2bea3c0 2024-07-24T21:53:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.648+0000 7fdff9ffb640 1 -- 172.21.15.60:0/1216915248 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004004040 con 0x55d9c2bea3c0 2024-07-24T21:53:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.717+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 msgr2=0x55d9c2be8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.717+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe004009b30 tx=0x7fe004009930 comp rx=0 tx=0).stop 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 shutdown_connections 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2bf6350 0x55d9c2be8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bea3c0 0x55d9c2be8920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 --2- 172.21.15.60:0/1216915248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2be9aa0 0x55d9c2bf3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 >> 172.21.15.60:0/1216915248 conn(0x55d9c2be78d0 msgr2=0x55d9c2bf2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 shutdown_connections 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.720+0000 7fe0153fe740 1 -- 172.21.15.60:0/1216915248 wait complete. 2024-07-24T21:53:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.721+0000 7fe0153fe740 1 -- start start 2024-07-24T21:53:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.722+0000 7fe0153fe740 1 Processor -- start 2024-07-24T21:53:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.722+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.723+0000 7fe0153fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2c0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.723+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.723+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48038/0 (socket says 172.21.15.60:48038) 2024-07-24T21:53:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.723+0000 7fe00bfff640 1 -- 172.21.15.60:0/3779252344 learned_addr learned my addr 172.21.15.60:0/3779252344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.723+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bf6350 0x55d9c2c0a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdff000cf70 con 0x55d9c2bea3c0 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdff000e3e0 con 0x55d9c2bf6350 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdff000d640 con 0x55d9c2be9aa0 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe010b68640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bf6350 0x55d9c2c0a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe0153fe740 10 client.0 _ll_register_callbacks cb 0x7fe000023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe00bfff640 1 -- 172.21.15.60:0/3779252344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bf6350 msgr2=0x55d9c2c0a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe00bfff640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bf6350 0x55d9c2c0a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe00bfff640 1 -- 172.21.15.60:0/3779252344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 msgr2=0x55d9c2c0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe00bfff640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2c0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.724+0000 7fe00bfff640 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c2bea1f0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.725+0000 7fe00bfff640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe000029420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f04d6d45157352c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:06.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:06 smithi123 ceph-mon[37143]: osdmap e89: 12 total, 12 up, 12 in 2024-07-24T21:53:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.750+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004049020 con 0x55d9c2be9aa0 2024-07-24T21:53:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.767+0000 7fdfef7fe640 10 client.26344 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.767+0000 7fdfef7fe640 1 -- 172.21.15.60:0/3779252344 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000040060 con 0x55d9c2be9aa0 2024-07-24T21:53:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.771+0000 7fdfef7fe640 1 -- 172.21.15.60:0/3779252344 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7fe000005350 con 0x55d9c2be9aa0 2024-07-24T21:53:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.771+0000 7fdfef7fe640 1 -- 172.21.15.60:0/3779252344 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe000025070 con 0x55d9c2be9aa0 2024-07-24T21:53:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.771+0000 7fe0153fe740 10 client.26344 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.771+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55d9c2c806a0 con 0x55d9c2be9aa0 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fdfef7fe640 1 -- 172.21.15.60:0/3779252344 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe00001d040 con 0x55d9c2be9aa0 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 10 client.26344 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 20 client.26344 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c80c60 0x55d9c2c83080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 4 client.26344 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:53:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.773+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe004049020 con 0x55d9c2c80c60 2024-07-24T21:53:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.776+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c80c60 0x55d9c2c83080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.779+0000 7fe00b7fe640 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c80c60 0x55d9c2c83080 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcecc46d04fb157 server_cookie=19f3c84cbff7a988 in_seq=0 out_seq=0 2024-07-24T21:53:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:06.783+0000 7fdfef7fe640 10 client.26344 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:06.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:06 smithi086 ceph-mon[37128]: osdmap e89: 12 total, 12 up, 12 in 2024-07-24T21:53:06.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:06 smithi060 ceph-mon[33000]: osdmap e89: 12 total, 12 up, 12 in 2024-07-24T21:53:07.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:07 smithi060 ceph-mon[33000]: pgmap v572: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-07-24T21:53:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:07 smithi086 ceph-mon[37128]: pgmap v572: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-07-24T21:53:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:07 smithi123 ceph-mon[37143]: pgmap v572: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-07-24T21:53:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:09 smithi086 ceph-mon[37128]: pgmap v573: 97 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 71 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-07-24T21:53:09.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:09 smithi060 ceph-mon[33000]: pgmap v573: 97 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 71 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-07-24T21:53:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:09 smithi123 ceph-mon[37143]: pgmap v573: 97 pgs: 16 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 71 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-07-24T21:53:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.402+0000 7f5441594640 1 -- 172.21.15.60:0/2698257961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c1079f0 msgr2=0x7f543c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.402+0000 7f5441594640 1 --2- 172.21.15.60:0/2698257961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c1079f0 0x7f543c10e1f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f542c009860 tx=0x7f542c02c570 comp rx=0 tx=0).stop 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.403+0000 7f5441594640 1 -- 172.21.15.60:0/2698257961 shutdown_connections 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.403+0000 7f5441594640 1 --2- 172.21.15.60:0/2698257961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c1079f0 0x7f543c10e1f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.403+0000 7f5441594640 1 --2- 172.21.15.60:0/2698257961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.403+0000 7f5441594640 1 --2- 172.21.15.60:0/2698257961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c101660 0x7f543c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.403+0000 7f5441594640 1 -- 172.21.15.60:0/2698257961 >> 172.21.15.60:0/2698257961 conn(0x7f543c0fd0f0 msgr2=0x7f543c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.404+0000 7f5441594640 1 -- 172.21.15.60:0/2698257961 shutdown_connections 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.404+0000 7f5441594640 1 -- 172.21.15.60:0/2698257961 wait complete. 2024-07-24T21:53:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.405+0000 7f5441594640 1 Processor -- start 2024-07-24T21:53:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 -- start start 2024-07-24T21:53:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 0x7f543c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c1079f0 0x7f543c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1105b0 con 0x7f543c107070 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f543c110430 con 0x7f543c101660 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f5441594640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f543c110730 con 0x7f543c1079f0 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47666/0 (socket says 172.21.15.60:47666) 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.406+0000 7f543a7fc640 1 -- 172.21.15.60:0/4121997238 learned_addr learned my addr 172.21.15.60:0/4121997238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.407+0000 7f543b7fe640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c1079f0 0x7f543c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543affd640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 0x7f543c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543a7fc640 1 -- 172.21.15.60:0/4121997238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c1079f0 msgr2=0x7f543c06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543a7fc640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c1079f0 0x7f543c06d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543a7fc640 1 -- 172.21.15.60:0/4121997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 msgr2=0x7f543c074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543a7fc640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 0x7f543c074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.408+0000 7f543a7fc640 1 -- 172.21.15.60:0/4121997238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c06dcd0 con 0x7f543c107070 2024-07-24T21:53:10.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.411+0000 7f543affd640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 0x7f543c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:53:10.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.411+0000 7f543a7fc640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f543000d950 tx=0x7f543000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.416+0000 7f5423fff640 1 -- 172.21.15.60:0/4121997238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430017070 con 0x7f543c107070 2024-07-24T21:53:10.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.416+0000 7f5423fff640 1 -- 172.21.15.60:0/4121997238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5430009c30 con 0x7f543c107070 2024-07-24T21:53:10.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.416+0000 7f5423fff640 1 -- 172.21.15.60:0/4121997238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543000f620 con 0x7f543c107070 2024-07-24T21:53:10.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.418+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c06dfc0 con 0x7f543c107070 2024-07-24T21:53:10.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.418+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c072f50 con 0x7f543c107070 2024-07-24T21:53:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.426+0000 7f5423fff640 1 -- 172.21.15.60:0/4121997238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f543000f7c0 con 0x7f543c107070 2024-07-24T21:53:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.429+0000 7f5423fff640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 0x7f54240b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.429+0000 7f5423fff640 1 -- 172.21.15.60:0/4121997238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7f54300e4ff0 con 0x7f543c107070 2024-07-24T21:53:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.433+0000 7f5443945740 1 Processor -- start 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.435+0000 7f5443945740 1 -- start start 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5599cd6166a0 con 0x5599cd6ac5d0 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599cd605ba0 con 0x5599cd6a07a0 2024-07-24T21:53:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.436+0000 7f5443945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599cd609be0 con 0x5599cd6a1d00 2024-07-24T21:53:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.438+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.438+0000 7f543affd640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 0x7f54240b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.438+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47682/0 (socket says 172.21.15.60:47682) 2024-07-24T21:53:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.438+0000 7f543b7fe640 1 -- 172.21.15.60:0/3008224148 learned_addr learned my addr 172.21.15.60:0/3008224148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:10.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.438+0000 7f543affd640 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.439+0000 7f543affd640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 0x7f54240b6410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f5434009bb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.440+0000 7f543affd640 1 -- 172.21.15.60:0/3008224148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 msgr2=0x5599cd6a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:10.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.440+0000 7f543affd640 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.440+0000 7f543affd640 1 -- 172.21.15.60:0/3008224148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 msgr2=0x5599cd6ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:10.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.440+0000 7f543affd640 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.440+0000 7f543affd640 1 -- 172.21.15.60:0/3008224148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cd59dd60 con 0x5599cd6a1d00 2024-07-24T21:53:10.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.442+0000 7f543affd640 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a0260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5434011ad0 tx=0x7f543401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c2eb86e6de8513 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.443+0000 7f54217fa640 1 -- 172.21.15.60:0/3008224148 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400da90 con 0x5599cd6a1d00 2024-07-24T21:53:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.443+0000 7f54217fa640 1 -- 172.21.15.60:0/3008224148 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543400dc30 con 0x5599cd6a1d00 2024-07-24T21:53:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.444+0000 7f5443945740 1 -- 172.21.15.60:0/3008224148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 msgr2=0x5599cd6a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.444+0000 7f5443945740 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a0260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5434011ad0 tx=0x7f543401d000 comp rx=0 tx=0).stop 2024-07-24T21:53:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.446+0000 7f5443945740 1 -- 172.21.15.60:0/3008224148 shutdown_connections 2024-07-24T21:53:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.446+0000 7f5443945740 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.446+0000 7f5443945740 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.446+0000 7f5443945740 1 --2- 172.21.15.60:0/3008224148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a0260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.446+0000 7f5443945740 1 -- 172.21.15.60:0/3008224148 >> 172.21.15.60:0/3008224148 conn(0x5599cd66a750 msgr2=0x5599cd69e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.449+0000 7f5443945740 1 -- 172.21.15.60:0/3008224148 shutdown_connections 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.449+0000 7f5443945740 1 -- 172.21.15.60:0/3008224148 wait complete. 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.449+0000 7f5443945740 1 -- start start 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.449+0000 7f5443945740 1 Processor -- start 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5599cd4697a0 con 0x5599cd6ac5d0 2024-07-24T21:53:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599cd5a8020 con 0x5599cd6a07a0 2024-07-24T21:53:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599cd6af480 con 0x5599cd6a1d00 2024-07-24T21:53:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.450+0000 7f5443945740 10 client.0 _ll_register_callbacks cb 0x5599cd313090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.451+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.451+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47688/0 (socket says 172.21.15.60:47688) 2024-07-24T21:53:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.451+0000 7f543b7fe640 1 -- 172.21.15.60:0/2829531657 learned_addr learned my addr 172.21.15.60:0/2829531657 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543a7fc640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543b7fe640 1 -- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 msgr2=0x5599cd6a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543b7fe640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543b7fe640 1 -- 172.21.15.60:0/2829531657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 msgr2=0x5599cd6abfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:53:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543b7fe640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6abfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.452+0000 7f543b7fe640 1 -- 172.21.15.60:0/2829531657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cd5ac0d0 con 0x5599cd6ac5d0 2024-07-24T21:53:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.453+0000 7f543b7fe640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f542c0325d0 tx=0x7f542c032600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5bdfea2c258011 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.453+0000 7f541eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.455+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c032d90 con 0x5599cd6ac5d0 2024-07-24T21:53:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.455+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543400dc30 con 0x5599cd6ac5d0 2024-07-24T21:53:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.455+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7f542c005030 con 0x5599cd6ac5d0 2024-07-24T21:53:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.455+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f542c005230 con 0x5599cd6ac5d0 2024-07-24T21:53:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.456+0000 7f5443945740 10 client.16875 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.456+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5599cd375b90 con 0x5599cd6ac5d0 2024-07-24T21:53:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543400dc30 con 0x5599cd6ac5d0 2024-07-24T21:53:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f542c03e420 con 0x5599cd6ac5d0 2024-07-24T21:53:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 10 client.16875 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 10 client.16875 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:53:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 20 client.16875 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd6dea90 0x5599cd6e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 4 client.16875 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.458+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599cd6185f0 con 0x5599cd6dea90 2024-07-24T21:53:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.471+0000 7f543affd640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd6dea90 0x5599cd6e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.485+0000 7f541eff5640 10 client.16875 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.485+0000 7f543affd640 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd6dea90 0x5599cd6e0f10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740ae89aa0300b6f server_cookie=aaff74bda69773de in_seq=0 out_seq=0 2024-07-24T21:53:10.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.487+0000 7f541eff5640 1 -- 172.21.15.60:0/2829531657 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599cd6185f0 con 0x5599cd6dea90 2024-07-24T21:53:10.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.487+0000 7f541eff5640 10 client.16875 handle_command_reply: tid=0 2024-07-24T21:53:10.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.487+0000 7f5443945740 1 client.16875 shutdown 2024-07-24T21:53:10.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 msgr2=0x5599cd6a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f542c0325d0 tx=0x7f542c032600 comp rx=0 tx=0).stop 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 shutdown_connections 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd6dea90 0x5599cd6e0f10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6a9ae0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f541eff5640 0 client.16875 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 --2- 172.21.15.60:0/2829531657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 >> 172.21.15.60:0/2829531657 conn(0x5599cd66a750 msgr2=0x5599cd6a3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 shutdown_connections 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.488+0000 7f5443945740 1 -- 172.21.15.60:0/2829531657 wait complete. 2024-07-24T21:53:10.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.489+0000 7f5443945740 20 client.16875 trim_cache size 0 max 16384 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 Processor -- start 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 -- start start 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6ac5d0 0x5599cd6e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f54280113d0 con 0x5599cd6a1d00 2024-07-24T21:53:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f542800ebe0 con 0x5599cd6a07a0 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f5443945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f542800ca50 con 0x5599cd6ac5d0 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47696/0 (socket says 172.21.15.60:47696) 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.527+0000 7f543a7fc640 1 -- 172.21.15.60:0/2322965873 learned_addr learned my addr 172.21.15.60:0/2322965873 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 -- 172.21.15.60:0/2322965873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6ac5d0 msgr2=0x5599cd6e67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6ac5d0 0x5599cd6e67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 -- 172.21.15.60:0/2322965873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 msgr2=0x5599cd6a82e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a82e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 -- 172.21.15.60:0/2322965873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cd6997a0 con 0x5599cd6a1d00 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.528+0000 7f543a7fc640 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f54300ab840 tx=0x7f54300a7390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6e65bf62303c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.529+0000 7f5420ff9640 1 -- 172.21.15.60:0/2322965873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430020070 con 0x5599cd6a1d00 2024-07-24T21:53:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.529+0000 7f5420ff9640 1 -- 172.21.15.60:0/2322965873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5430015020 con 0x5599cd6a1d00 2024-07-24T21:53:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.530+0000 7f5443945740 1 -- 172.21.15.60:0/2322965873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 msgr2=0x5599cd69d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.530+0000 7f5443945740 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f54300ab840 tx=0x7f54300a7390 comp rx=0 tx=0).stop 2024-07-24T21:53:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.531+0000 7f5443945740 1 -- 172.21.15.60:0/2322965873 shutdown_connections 2024-07-24T21:53:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.531+0000 7f5443945740 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6ac5d0 0x5599cd6e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.531+0000 7f5443945740 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6a1d00 0x5599cd69d060 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.531+0000 7f5443945740 1 --2- 172.21.15.60:0/2322965873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6a07a0 0x5599cd6a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.531+0000 7f5443945740 1 -- 172.21.15.60:0/2322965873 >> 172.21.15.60:0/2322965873 conn(0x5599cd66a750 msgr2=0x5599cd69c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:10.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.535+0000 7f5443945740 1 -- 172.21.15.60:0/2322965873 shutdown_connections 2024-07-24T21:53:10.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.535+0000 7f5443945740 1 -- 172.21.15.60:0/2322965873 wait complete. 2024-07-24T21:53:10.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.536+0000 7f5443945740 1 -- start start 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.536+0000 7f5443945740 1 Processor -- start 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.536+0000 7f5443945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.536+0000 7f5443945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.537+0000 7f5443945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6e7220 0x5599cd6af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.537+0000 7f5443945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f542800d0a0 con 0x5599cd6ac5d0 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.537+0000 7f5443945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f542800e510 con 0x5599cd6e7220 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.537+0000 7f5443945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f542800d770 con 0x5599cd6a1d00 2024-07-24T21:53:10.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.537+0000 7f5443945740 10 client.0 _ll_register_callbacks cb 0x7f542c0094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.538+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.538+0000 7f543a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47704/0 (socket says 172.21.15.60:47704) 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.538+0000 7f543a7fc640 1 -- 172.21.15.60:0/1394609547 learned_addr learned my addr 172.21.15.60:0/1394609547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.538+0000 7f543a7fc640 1 -- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 msgr2=0x5599cd6b4800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.539+0000 7f543a7fc640 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6b4800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.539+0000 7f543a7fc640 1 -- 172.21.15.60:0/1394609547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6e7220 msgr2=0x5599cd6af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:10.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.539+0000 7f543a7fc640 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6e7220 0x5599cd6af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:10.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.539+0000 7f543a7fc640 1 -- 172.21.15.60:0/1394609547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cd6afe50 con 0x5599cd6ac5d0 2024-07-24T21:53:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.548+0000 7f543a7fc640 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f54300abad0 tx=0x7f543000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d0adc1f53ee85f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.548+0000 7f541eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.548+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54300ca020 con 0x5599cd6ac5d0 2024-07-24T21:53:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.548+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8609+0+0 (secure 0 0 0) 0x7f54300aada0 con 0x5599cd6ac5d0 2024-07-24T21:53:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.549+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5430015020 con 0x5599cd6ac5d0 2024-07-24T21:53:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.549+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5430020040 con 0x5599cd6ac5d0 2024-07-24T21:53:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.558+0000 7f5443945740 10 client.16881 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.558+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5599cd74a110 con 0x5599cd6ac5d0 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5430015020 con 0x5599cd6ac5d0 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f54300aa810 con 0x5599cd6ac5d0 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 10 client.16881 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 10 client.16881 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 20 client.16881 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd74a9e0 0x5599cd74ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 4 client.16881 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-24T21:53:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.562+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5599cd6a8c50 con 0x5599cd74a9e0 2024-07-24T21:53:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.568+0000 7f543affd640 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd74a9e0 0x5599cd74ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:10.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.579+0000 7f541eff5640 10 client.16881 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:10.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:10.579+0000 7f543affd640 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd74a9e0 0x5599cd74ce00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31725bd5d00dc3e1 server_cookie=9c8371b50130e968 in_seq=0 out_seq=0 2024-07-24T21:53:11.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:11 smithi060 ceph-mon[33000]: pgmap v574: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 867 op/s 2024-07-24T21:53:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:11 smithi123 ceph-mon[37143]: pgmap v574: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 867 op/s 2024-07-24T21:53:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:11 smithi086 ceph-mon[37128]: pgmap v574: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 867 op/s 2024-07-24T21:53:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:13 smithi060 ceph-mon[33000]: pgmap v575: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 1.27k op/s 2024-07-24T21:53:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:13 smithi123 ceph-mon[37143]: pgmap v575: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 1.27k op/s 2024-07-24T21:53:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:13 smithi086 ceph-mon[37128]: pgmap v575: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 1.27k op/s 2024-07-24T21:53:14.714 INFO:tasks.workunit.client.0.smithi060.stdout: 518 828262 581 585873 476 307996 719 2024-07-24T21:53:15.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:15 smithi060 ceph-mon[33000]: pgmap v576: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 1.77k op/s 2024-07-24T21:53:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:15 smithi123 ceph-mon[37143]: pgmap v576: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 1.77k op/s 2024-07-24T21:53:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:15 smithi086 ceph-mon[37128]: pgmap v576: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 1.77k op/s 2024-07-24T21:53:17.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:17 smithi060 ceph-mon[33000]: pgmap v577: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 2.27k op/s 2024-07-24T21:53:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:17 smithi123 ceph-mon[37143]: pgmap v577: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 2.27k op/s 2024-07-24T21:53:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:17 smithi086 ceph-mon[37128]: pgmap v577: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 2.27k op/s 2024-07-24T21:53:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:19 smithi060 ceph-mon[33000]: pgmap v578: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 2.36k op/s 2024-07-24T21:53:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:19 smithi123 ceph-mon[37143]: pgmap v578: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 2.36k op/s 2024-07-24T21:53:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:19 smithi086 ceph-mon[37128]: pgmap v578: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 2.36k op/s 2024-07-24T21:53:21.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:21 smithi060 ceph-mon[33000]: pgmap v579: 97 pgs: 24 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 2.85k op/s 2024-07-24T21:53:21.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:21 smithi123 ceph-mon[37143]: pgmap v579: 97 pgs: 24 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 2.85k op/s 2024-07-24T21:53:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:21 smithi086 ceph-mon[37128]: pgmap v579: 97 pgs: 24 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 2.85k op/s 2024-07-24T21:53:23.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:23 smithi060 ceph-mon[33000]: pgmap v580: 97 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 78 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 2.57k op/s 2024-07-24T21:53:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:23 smithi123 ceph-mon[37143]: pgmap v580: 97 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 78 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 2.57k op/s 2024-07-24T21:53:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:23 smithi086 ceph-mon[37128]: pgmap v580: 97 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 78 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 2.57k op/s 2024-07-24T21:53:24.714 INFO:tasks.workunit.client.0.smithi060.stdout: 518 0 0 0 0 28 0 2024-07-24T21:53:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.910+0000 7fdfef7fe640 1 -- 172.21.15.60:0/3779252344 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe004049020 con 0x55d9c2c80c60 2024-07-24T21:53:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.910+0000 7fdfef7fe640 10 client.26344 handle_command_reply: tid=0 2024-07-24T21:53:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.910+0000 7fe0153fe740 1 client.26344 shutdown 2024-07-24T21:53:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 msgr2=0x55d9c2c0f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe000029420 comp rx=0 tx=0).stop 2024-07-24T21:53:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 shutdown_connections 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d9c2c80c60 0x55d9c2c83080 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c2bf6350 0x55d9c2c0a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d9c2bea3c0 0x55d9c2c0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 --2- 172.21.15.60:0/3779252344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9c2be9aa0 0x55d9c2c0f5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 >> 172.21.15.60:0/3779252344 conn(0x55d9c2be78d0 msgr2=0x55d9c2be8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fdfef7fe640 0 client.26344 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 shutdown_connections 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 1 -- 172.21.15.60:0/3779252344 wait complete. 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.911+0000 7fe0153fe740 20 client.26344 trim_cache size 0 max 16384 2024-07-24T21:53:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 msgr2=0x7fdff40b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 0x7fdff40b6410 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe0000118d0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 msgr2=0x7fe00c1ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c970 tx=0x7fdffc00cd70 comp rx=0 tx=0).stop 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 shutdown_connections 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdff40b3f50 0x7fdff40b6410 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c11ba90 0x7fe00c1ad1a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe00c109f20 0x7fe00c1a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 --2- 172.21.15.60:0/2407492211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe00c109b40 0x7fe00c1a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 >> 172.21.15.60:0/2407492211 conn(0x7fe00c06d5a0 msgr2=0x7fe00c11c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 shutdown_connections 2024-07-24T21:53:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:24.913+0000 7fe0125cb640 1 -- 172.21.15.60:0/2407492211 wait complete. 2024-07-24T21:53:25.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:25 smithi060 ceph-mon[33000]: pgmap v581: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 2.26k op/s 2024-07-24T21:53:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:25 smithi123 ceph-mon[37143]: pgmap v581: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 2.26k op/s 2024-07-24T21:53:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:25 smithi086 ceph-mon[37128]: pgmap v581: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 15 MiB/s wr, 2.26k op/s 2024-07-24T21:53:27.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:27 smithi060 ceph-mon[33000]: pgmap v582: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.84k op/s 2024-07-24T21:53:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:27 smithi123 ceph-mon[37143]: pgmap v582: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.84k op/s 2024-07-24T21:53:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:27 smithi086 ceph-mon[37128]: pgmap v582: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.84k op/s 2024-07-24T21:53:29.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:29 smithi086 ceph-mon[37128]: osdmap e90: 12 total, 12 up, 12 in 2024-07-24T21:53:29.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:29 smithi086 ceph-mon[37128]: pgmap v584: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 1.10k op/s 2024-07-24T21:53:29.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:29 smithi060 ceph-mon[33000]: osdmap e90: 12 total, 12 up, 12 in 2024-07-24T21:53:29.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:29 smithi060 ceph-mon[33000]: pgmap v584: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 1.10k op/s 2024-07-24T21:53:29.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:29 smithi123 ceph-mon[37143]: osdmap e90: 12 total, 12 up, 12 in 2024-07-24T21:53:29.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:29 smithi123 ceph-mon[37143]: pgmap v584: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 1.10k op/s 2024-07-24T21:53:31.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:53:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:31.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:31 smithi060 ceph-mon[33000]: pgmap v585: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-07-24T21:53:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:31 smithi123 ceph-mon[37143]: pgmap v585: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-07-24T21:53:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:31 smithi086 ceph-mon[37128]: pgmap v585: 97 pgs: 97 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 -- 172.21.15.60:0/1432341861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 msgr2=0x7fd7a410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 --2- 172.21.15.60:0/1432341861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a410c4b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd794009900 tx=0x7fd79402a410 comp rx=0 tx=0).stop 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 -- 172.21.15.60:0/1432341861 shutdown_connections 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 --2- 172.21.15.60:0/1432341861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 --2- 172.21.15.60:0/1432341861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a410c4b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 --2- 172.21.15.60:0/1432341861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 0x7fd7a4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.255+0000 7fd7ac115640 1 -- 172.21.15.60:0/1432341861 >> 172.21.15.60:0/1432341861 conn(0x7fd7a40fd120 msgr2=0x7fd7a40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- 172.21.15.60:0/1432341861 shutdown_connections 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- 172.21.15.60:0/1432341861 wait complete. 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 Processor -- start 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- start start 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 0x7fd7a4196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41105b0 con 0x7fd7a4103fc0 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4110430 con 0x7fd7a410cb00 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.256+0000 7fd7ac115640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4110730 con 0x7fd7a41036a0 2024-07-24T21:53:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 0x7fd7a4196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48772/0 (socket says 172.21.15.60:48772) 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 -- 172.21.15.60:0/2282087836 learned_addr learned my addr 172.21.15.60:0/2282087836 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7aa6b2640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 -- 172.21.15.60:0/2282087836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 msgr2=0x7fd7a4196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 0x7fd7a4196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 -- 172.21.15.60:0/2282087836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 msgr2=0x7fd7a419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 -- 172.21.15.60:0/2282087836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a419cbf0 con 0x7fd7a4103fc0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7aa6b2640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd794005ab0 tx=0x7fd794003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.257+0000 7fd79affd640 1 -- 172.21.15.60:0/2282087836 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79403f070 con 0x7fd7a4103fc0 2024-07-24T21:53:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.258+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a419ce80 con 0x7fd7a4103fc0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.258+0000 7fd79affd640 1 -- 172.21.15.60:0/2282087836 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd794031b50 con 0x7fd7a4103fc0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.258+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41a35a0 con 0x7fd7a4103fc0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.258+0000 7fd79affd640 1 -- 172.21.15.60:0/2282087836 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794035030 con 0x7fd7a4103fc0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.261+0000 7fd7ae4c6740 1 Processor -- start 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.261+0000 7fd7ae4c6740 1 -- start start 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd79affd640 1 -- 172.21.15.60:0/2282087836 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd794042070 con 0x7fd7a4103fc0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 0x559626282a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 0x55962628e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55962628ef30 0x559626291320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5596261f9960 con 0x559626282680 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5596261e8e60 con 0x559626282fa0 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596261ecea0 con 0x55962628ef30 2024-07-24T21:53:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55962628ef30 0x559626291320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 0x55962628e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7aa6b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 0x559626282a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7aa6b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 0x559626282a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48782/0 (socket says 172.21.15.60:48782) 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.262+0000 7fd7aa6b2640 1 -- 172.21.15.60:0/356994258 learned_addr learned my addr 172.21.15.60:0/356994258 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/356994258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55962628ef30 msgr2=0x559626291320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55962628ef30 0x559626291320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/356994258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 msgr2=0x559626282a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 0x559626282a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/356994258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596261722d0 con 0x559626282fa0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 0x55962628e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c00 tx=0x7fd7a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a452a933cc7cf921 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd78ffff640 1 -- 172.21.15.60:0/356994258 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0004230 con 0x559626282fa0 2024-07-24T21:53:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd79affd640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 0x7fd7900ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd78ffff640 1 -- 172.21.15.60:0/356994258 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a00043d0 con 0x559626282fa0 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.263+0000 7fd78ffff640 1 -- 172.21.15.60:0/356994258 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0004c30 con 0x559626282fa0 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd79affd640 1 -- 172.21.15.60:0/2282087836 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fd794104510 con 0x7fd7a4103fc0 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7aa6b2640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 0x7fd7900ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/356994258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 msgr2=0x55962628e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 0x55962628e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c00 tx=0x7fd7a0011810 comp rx=0 tx=0).stop 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/356994258 shutdown_connections 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55962628ef30 0x559626291320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282fa0 0x55962628e9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/356994258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282680 0x559626282a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7aa6b2640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 0x7fd7900ba610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/356994258 >> 172.21.15.60:0/356994258 conn(0x5596262804b0 msgr2=0x559626281650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/356994258 shutdown_connections 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/356994258 wait complete. 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.264+0000 7fd7ae4c6740 1 -- start start 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 Processor -- start 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 0x55962628be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55962628ef30 0x55962628c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55962604c700 con 0x559626282fa0 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55962618b5a0 con 0x55962628ef30 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559626291bb0 con 0x559626282680 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7ae4c6740 10 client.0 _ll_register_callbacks cb 0x5596261ab910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49222/0 (socket says 172.21.15.60:49222) 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 0x55962628be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.265+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/317668382 learned_addr learned my addr 172.21.15.60:0/317668382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7aa6b2640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55962628ef30 0x55962628c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/317668382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55962628ef30 msgr2=0x55962628c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55962628ef30 0x55962628c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/317668382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 msgr2=0x55962628be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 0x55962628be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/317668382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962617e630 con 0x559626282680 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 0x55962628be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0019b70 tx=0x7fd7a0019ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f5581c7d02c448 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.266+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0028020 con 0x559626282680 2024-07-24T21:53:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a00043d0 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fd7a0024620 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd7a0033050 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 10 client.26365 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559625f49650 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a001f040 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd7a0018070 con 0x559626282680 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 10 client.26365 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 20 client.26365 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55962629e8d0 0x5596262a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 4 client.26365 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.267+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd7a00043d0 con 0x55962629e8d0 2024-07-24T21:53:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.268+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55962629e8d0 0x5596262a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.268+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55962629e8d0 0x5596262a0cc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95592bd04d34c082 server_cookie=eca434c46e7b932c in_seq=0 out_seq=0 2024-07-24T21:53:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.268+0000 7fd78d7fa640 10 client.26365 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd78d7fa640 1 -- 172.21.15.60:0/317668382 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd7a00043d0 con 0x55962629e8d0 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd78d7fa640 10 client.26365 handle_command_reply: tid=0 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 client.26365 shutdown 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 msgr2=0x55962628e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0019b70 tx=0x7fd7a0019ba0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 shutdown_connections 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55962629e8d0 0x5596262a0cc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55962628ef30 0x55962628c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559626282fa0 0x55962628be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd78d7fa640 0 client.26365 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:33.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.303+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/317668382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282680 0x55962628e9b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.304+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 >> 172.21.15.60:0/317668382 conn(0x5596262804b0 msgr2=0x559626286050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.304+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 shutdown_connections 2024-07-24T21:53:33.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.304+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/317668382 wait complete. 2024-07-24T21:53:33.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.304+0000 7fd7ae4c6740 20 client.26365 trim_cache size 0 max 16384 2024-07-24T21:53:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 Processor -- start 2024-07-24T21:53:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 -- start start 2024-07-24T21:53:33.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x55962627f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x5596262a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41aeaf0 con 0x55962628ef30 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41ac300 con 0x559626282680 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41aa170 con 0x559626282fa0 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x55962627f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:47058/0 (socket says 172.21.15.60:47058) 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/808913234 learned_addr learned my addr 172.21.15.60:0/808913234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7a96b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x55962627f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49232/0 (socket says 172.21.15.60:49232) 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.324+0000 7fd7aa6b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x5596262a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/808913234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 msgr2=0x55962627f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x55962627f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/808913234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 msgr2=0x5596262a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x5596262a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/808913234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962628c900 con 0x559626282680 2024-07-24T21:53:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.325+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c00 tx=0x7fd7a0019f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a574e342a4289e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.327+0000 7fd78f7fe640 1 -- 172.21.15.60:0/808913234 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0009e70 con 0x559626282680 2024-07-24T21:53:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.327+0000 7fd78f7fe640 1 -- 172.21.15.60:0/808913234 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a00054e0 con 0x559626282680 2024-07-24T21:53:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.327+0000 7fd78f7fe640 1 -- 172.21.15.60:0/808913234 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0018e60 con 0x559626282680 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/808913234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 msgr2=0x5596262867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009c00 tx=0x7fd7a0019f50 comp rx=0 tx=0).stop 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/808913234 shutdown_connections 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x5596262a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x55962627f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/808913234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x5596262867d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/808913234 >> 172.21.15.60:0/808913234 conn(0x5596262804b0 msgr2=0x55962627ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/808913234 shutdown_connections 2024-07-24T21:53:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/808913234 wait complete. 2024-07-24T21:53:33.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.328+0000 7fd7ae4c6740 1 -- start start 2024-07-24T21:53:33.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 Processor -- start 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x559626288ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x559626289430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x559626289970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41aa7c0 con 0x55962628ef30 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41abc30 con 0x559626282680 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41aae90 con 0x559626282fa0 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7ae4c6740 10 client.0 _ll_register_callbacks cb 0x7fd79c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7aa6b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x559626289970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7aa6b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x559626289970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48814/0 (socket says 172.21.15.60:48814) 2024-07-24T21:53:33.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7aa6b2640 1 -- 172.21.15.60:0/3276162990 learned_addr learned my addr 172.21.15.60:0/3276162990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7a9eb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x559626288ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.329+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x559626289430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 msgr2=0x559626289430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x559626289430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/3276162990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 msgr2=0x559626289970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x559626289970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 -- 172.21.15.60:0/3276162990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559626308520 con 0x559626282680 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7a9eb1640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x559626288ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55962628c460 tx=0x7fd7a0019850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5a2a6f01c9dd6d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0009e70 con 0x559626282680 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fd7a0018b50 con 0x559626282680 2024-07-24T21:53:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55962628b660 con 0x559626282680 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd7a0019b50 con 0x559626282680 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7ae4c6740 10 client.26342 fetch_fsmap learned FSMap version 37 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.330+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5596263097d0 con 0x559626282680 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55962628b660 con 0x559626282680 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd7a0047020 con 0x559626282680 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 10 client.26342 fetch_fsmap finished waiting for FSMap version 37 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 20 client.26342 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559626309e40 0x55962630c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 4 client.26342 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:53:33.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.331+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596261fb8b0 con 0x559626309e40 2024-07-24T21:53:33.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.332+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559626309e40 0x55962630c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:33.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.332+0000 7fd78d7fa640 10 client.26342 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:33.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.332+0000 7fd7a96b0640 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559626309e40 0x55962630c260 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad120c58438b5827 server_cookie=99dd311f2a15bafa in_seq=0 out_seq=0 2024-07-24T21:53:33.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:33 smithi060 ceph-mon[33000]: pgmap v586: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 472 op/s 2024-07-24T21:53:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.449+0000 7fd78d7fa640 1 -- 172.21.15.60:0/3276162990 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596261fb8b0 con 0x559626309e40 2024-07-24T21:53:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.449+0000 7fd78d7fa640 10 client.26342 handle_command_reply: tid=0 2024-07-24T21:53:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.449+0000 7fd7ae4c6740 1 client.26342 shutdown 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 msgr2=0x559626288ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x559626288ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55962628c460 tx=0x7fd7a0019850 comp rx=0 tx=0).stop 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 shutdown_connections 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559626309e40 0x55962630c260 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55962628ef30 0x559626289970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd78d7fa640 0 client.26342 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559626282fa0 0x559626289430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 --2- 172.21.15.60:0/3276162990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559626282680 0x559626288ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 >> 172.21.15.60:0/3276162990 conn(0x5596262804b0 msgr2=0x55962628dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 shutdown_connections 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 1 -- 172.21.15.60:0/3276162990 wait complete. 2024-07-24T21:53:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.450+0000 7fd7ae4c6740 20 client.26342 trim_cache size 0 max 16384 2024-07-24T21:53:33.454 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:53:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:53:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:53:33.454 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 msgr2=0x7fd7900ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 0x7fd7900ba610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c009500 comp rx=0 tx=0).stop 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 msgr2=0x7fd7a4197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd794005ab0 tx=0x7fd794003ae0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 shutdown_connections 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7900b8150 0x7fd7900ba610 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7a410cb00 0x7fd7a419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a4103fc0 0x7fd7a4197080 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 --2- 172.21.15.60:0/2282087836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7a41036a0 0x7fd7a4196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:33.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 >> 172.21.15.60:0/2282087836 conn(0x7fd7a40fd120 msgr2=0x7fd7a40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:33.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 shutdown_connections 2024-07-24T21:53:33.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:33.451+0000 7fd7ac115640 1 -- 172.21.15.60:0/2282087836 wait complete. 2024-07-24T21:53:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:33 smithi123 ceph-mon[37143]: pgmap v586: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 472 op/s 2024-07-24T21:53:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:33 smithi086 ceph-mon[37128]: pgmap v586: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 472 op/s 2024-07-24T21:53:34.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:53:34.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:53:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:35 smithi123 ceph-mon[37143]: pgmap v587: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 466 op/s 2024-07-24T21:53:35.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:35 smithi060 ceph-mon[33000]: pgmap v587: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 466 op/s 2024-07-24T21:53:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:35 smithi086 ceph-mon[37128]: pgmap v587: 97 pgs: 97 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 466 op/s 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 -- 172.21.15.60:0/809815823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c1079e0 msgr2=0x7fa40c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 --2- 172.21.15.60:0/809815823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c1079e0 0x7fa40c10e1e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ca0 tx=0x7fa40402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 -- 172.21.15.60:0/809815823 shutdown_connections 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 --2- 172.21.15.60:0/809815823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c1079e0 0x7fa40c10e1e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 --2- 172.21.15.60:0/809815823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 --2- 172.21.15.60:0/809815823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c101650 0x7fa40c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 -- 172.21.15.60:0/809815823 >> 172.21.15.60:0/809815823 conn(0x7fa40c0fd0c0 msgr2=0x7fa40c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 -- 172.21.15.60:0/809815823 shutdown_connections 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.420+0000 7fa411b23640 1 -- 172.21.15.60:0/809815823 wait complete. 2024-07-24T21:53:36.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 Processor -- start 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 -- start start 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c10f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c1079e0 0x7fa40c10f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1127f0 con 0x7fa40c101650 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112670 con 0x7fa40c107060 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa411b23640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c112970 con 0x7fa40c1079e0 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39120/0 (socket says 172.21.15.60:39120) 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa40b7fe640 1 -- 172.21.15.60:0/2117146620 learned_addr learned my addr 172.21.15.60:0/2117146620 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa40affd640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c10f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.421+0000 7fa40bfff640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c1079e0 0x7fa40c10f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 -- 172.21.15.60:0/2117146620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c1079e0 msgr2=0x7fa40c10f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c1079e0 0x7fa40c10f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 -- 172.21.15.60:0/2117146620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 msgr2=0x7fa40c10f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c10f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 -- 172.21.15.60:0/2117146620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a7020 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40affd640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa40b7fe640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00cd80 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3fc0054d0 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a7310 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc010690 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.422+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a7800 con 0x7fa40c101650 2024-07-24T21:53:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.423+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c101a30 con 0x7fa40c101650 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.424+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa3fc004e90 con 0x7fa40c101650 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.425+0000 7fa408ff9640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 0x7fa3f40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.425+0000 7fa40affd640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 0x7fa3f40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.425+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fa3fc0e4410 con 0x7fa40c101650 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.426+0000 7fa40affd640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 0x7fa3f40b6200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa40c1107e0 tx=0x7fa400005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.429+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa3fc0ab6f0 con 0x7fa40c101650 2024-07-24T21:53:36.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.665+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c02e000 con 0x7fa40c101650 2024-07-24T21:53:36.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:36 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:53:36.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.665+0000 7fa408ff9640 1 -- 172.21.15.60:0/2117146620 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11919 (secure 0 0 0) 0x7fa3fc0aaff0 con 0x7fa40c101650 2024-07-24T21:53:36.668 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:53:36.668 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"btime":"2024-07-24T21:53:35:326256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:53:34.943486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:53:36.668 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 38 2024-07-24T21:53:36.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 msgr2=0x7fa3f40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:36.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 0x7fa3f40b6200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa40c1107e0 tx=0x7fa400005f90 comp rx=0 tx=0).stop 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 msgr2=0x7fa40c10ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00edf0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 shutdown_connections 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3f40b3d40 0x7fa3f40b6200 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c1079e0 0x7fa40c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c107060 0x7fa40c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 --2- 172.21.15.60:0/2117146620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c101650 0x7fa40c10ecf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 >> 172.21.15.60:0/2117146620 conn(0x7fa40c0fd0c0 msgr2=0x7fa40c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 shutdown_connections 2024-07-24T21:53:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:36.668+0000 7fa411b23640 1 -- 172.21.15.60:0/2117146620 wait complete. 2024-07-24T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:36 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:53:36.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:36 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:53:37.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:53:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:37 smithi086 ceph-mon[37128]: pgmap v588: 97 pgs: 97 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 589 op/s 2024-07-24T21:53:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:37 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2117146620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:37 smithi123 ceph-mon[37143]: pgmap v588: 97 pgs: 97 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 589 op/s 2024-07-24T21:53:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:37 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2117146620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:37.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:37 smithi060 ceph-mon[33000]: pgmap v588: 97 pgs: 97 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 589 op/s 2024-07-24T21:53:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:37 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2117146620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 -- 172.21.15.60:0/3548878327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd94106f90 msgr2=0x7efd941073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 --2- 172.21.15.60:0/3548878327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd94106f90 0x7efd941073d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efd88009ca0 tx=0x7efd8802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 -- 172.21.15.60:0/3548878327 shutdown_connections 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 --2- 172.21.15.60:0/3548878327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94107910 0x7efd9410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 --2- 172.21.15.60:0/3548878327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd94106f90 0x7efd941073d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 --2- 172.21.15.60:0/3548878327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd941015e0 0x7efd941019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 -- 172.21.15.60:0/3548878327 >> 172.21.15.60:0/3548878327 conn(0x7efd940782e0 msgr2=0x7efd940ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 -- 172.21.15.60:0/3548878327 shutdown_connections 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.009+0000 7efd988a9640 1 -- 172.21.15.60:0/3548878327 wait complete. 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 Processor -- start 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 -- start start 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94106f90 0x7efd940fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd94107910 0x7efd940fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efd94111530 con 0x7efd941015e0 2024-07-24T21:53:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd941113b0 con 0x7efd94106f90 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd988a9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd941116b0 con 0x7efd94107910 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd9259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd9259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39144/0 (socket says 172.21.15.60:39144) 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd9259c640 1 -- 172.21.15.60:0/1366962735 learned_addr learned my addr 172.21.15.60:0/1366962735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd94107910 0x7efd940fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.010+0000 7efd91d9b640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94106f90 0x7efd940fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 -- 172.21.15.60:0/1366962735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd94107910 msgr2=0x7efd940fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd94107910 0x7efd940fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 -- 172.21.15.60:0/1366962735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94106f90 msgr2=0x7efd940fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94106f90 0x7efd940fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 -- 172.21.15.60:0/1366962735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941a7010 con 0x7efd941015e0 2024-07-24T21:53:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd9259c640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd8400e9f0 tx=0x7efd8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd837fe640 1 -- 172.21.15.60:0/1366962735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400cd80 con 0x7efd941015e0 2024-07-24T21:53:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd837fe640 1 -- 172.21.15.60:0/1366962735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd840108e0 con 0x7efd941015e0 2024-07-24T21:53:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a7300 con 0x7efd941015e0 2024-07-24T21:53:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd837fe640 1 -- 172.21.15.60:0/1366962735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84018980 con 0x7efd941015e0 2024-07-24T21:53:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.011+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a7850 con 0x7efd941015e0 2024-07-24T21:53:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.014+0000 7efd9b601740 1 Processor -- start 2024-07-24T21:53:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.014+0000 7efd9b601740 1 -- start start 2024-07-24T21:53:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.014+0000 7efd9b601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.014+0000 7efd9b601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.014+0000 7efd9b601740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd9b601740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557cec24d280 con 0x557cec2e4170 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd9b601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cec23c780 con 0x557cec2d78c0 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd9b601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557cec2407c0 con 0x557cec2d81e0 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd837fe640 1 -- 172.21.15.60:0/1366962735 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7efd84010430 con 0x7efd941015e0 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39150/0 (socket says 172.21.15.60:39150) 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 -- 172.21.15.60:0/486600050 learned_addr learned my addr 172.21.15.60:0/486600050 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd92d9d640 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd9259c640 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 -- 172.21.15.60:0/486600050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 msgr2=0x557cec2e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 -- 172.21.15.60:0/486600050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 msgr2=0x557cec2d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 -- 172.21.15.60:0/486600050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cebfa0790 con 0x557cec2e4170 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd91d9b640 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd88009ca0 tx=0x7efd88005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64490c81de85910b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd80ff9640 1 -- 172.21.15.60:0/486600050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88031db0 con 0x557cec2e4170 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd80ff9640 1 -- 172.21.15.60:0/486600050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd88036030 con 0x557cec2e4170 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.015+0000 7efd80ff9640 1 -- 172.21.15.60:0/486600050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880042f0 con 0x557cec2e4170 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd837fe640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 0x7efd7c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 0x7efd7c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd837fe640 1 -- 172.21.15.60:0/1366962735 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7efd840e4340 con 0x7efd941015e0 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 -- 172.21.15.60:0/486600050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 msgr2=0x557cec2e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd88009ca0 tx=0x7efd88005fa0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 0x7efd7c0b6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005fd0 tx=0x7efd8c005f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 -- 172.21.15.60:0/486600050 shutdown_connections 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e6560 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 --2- 172.21.15.60:0/486600050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 -- 172.21.15.60:0/486600050 >> 172.21.15.60:0/486600050 conn(0x557cec2d56f0 msgr2=0x557cec2d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 -- 172.21.15.60:0/486600050 shutdown_connections 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.016+0000 7efd9b601740 1 -- 172.21.15.60:0/486600050 wait complete. 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 -- start start 2024-07-24T21:53:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 Processor -- start 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557cec176ab0 con 0x557cec2e4170 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cec1c49e0 con 0x557cec2d78c0 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557cec2e6c70 con 0x557cec2d81e0 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9b601740 10 client.0 _ll_register_callbacks cb 0x557cebf4ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39158/0 (socket says 172.21.15.60:39158) 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 -- 172.21.15.60:0/3827551735 learned_addr learned my addr 172.21.15.60:0/3827551735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd91d9b640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd9259c640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 -- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 msgr2=0x557cec2e0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 -- 172.21.15.60:0/3827551735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 msgr2=0x557cec2e37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2e37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.017+0000 7efd92d9d640 1 -- 172.21.15.60:0/3827551735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cec019620 con 0x557cec2e4170 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd92d9d640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd8c01c900 tx=0x7efd8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c82198d30afded5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c02c060 con 0x557cec2e4170 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7efd8c016030 con 0x557cec2e4170 2024-07-24T21:53:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557cec1ebce0 con 0x557cec2e4170 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd8c00f4e0 con 0x557cec2e4170 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 10 client.16917 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557cec2f2390 con 0x557cec2e4170 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557cec1ebce0 con 0x557cec2e4170 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7efd8c027750 con 0x557cec2e4170 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 10 client.16917 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 20 client.16917 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec2f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 4 client.16917 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.018+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cec24f1d0 con 0x557cec2f38f0 2024-07-24T21:53:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.019+0000 7efd9259c640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec2f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.019+0000 7efd9259c640 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec2f5ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52176bb7b6e56458 server_cookie=7dc9d984a6a9b8b8 in_seq=0 out_seq=0 2024-07-24T21:53:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.019+0000 7efd79ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd79ffb640 1 -- 172.21.15.60:0/3827551735 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cec24f1d0 con 0x557cec2f38f0 2024-07-24T21:53:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd79ffb640 10 client.16917 handle_command_reply: tid=0 2024-07-24T21:53:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 client.16917 shutdown 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 msgr2=0x557cec2e1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd8c01c900 tx=0x7efd8c004730 comp rx=0 tx=0).stop 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 shutdown_connections 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec2f5ce0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2e1210 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd79ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 --2- 172.21.15.60:0/3827551735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2e37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 >> 172.21.15.60:0/3827551735 conn(0x557cec2d56f0 msgr2=0x557cec2db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 shutdown_connections 2024-07-24T21:53:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.023+0000 7efd9b601740 1 -- 172.21.15.60:0/3827551735 wait complete. 2024-07-24T21:53:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.024+0000 7efd9b601740 20 client.16917 trim_cache size 0 max 16384 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.038+0000 7efd9b601740 1 Processor -- start 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.038+0000 7efd9b601740 1 -- start start 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efd941b2c90 con 0x557cec2e4170 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd941b0500 con 0x557cec2d78c0 2024-07-24T21:53:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9b601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd941ae370 con 0x557cec2d81e0 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39174/0 (socket says 172.21.15.60:39174) 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 -- 172.21.15.60:0/1976570040 learned_addr learned my addr 172.21.15.60:0/1976570040 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd91d9b640 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd9259c640 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 -- 172.21.15.60:0/1976570040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 msgr2=0x557cec2fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 -- 172.21.15.60:0/1976570040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 msgr2=0x557cec2d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 -- 172.21.15.60:0/1976570040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cec2f6050 con 0x557cec2e4170 2024-07-24T21:53:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd92d9d640 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0108c0 tx=0x7efd8c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d824c28b4a56bf6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd7bfff640 1 -- 172.21.15.60:0/1976570040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c027420 con 0x557cec2e4170 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.039+0000 7efd7bfff640 1 -- 172.21.15.60:0/1976570040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd8c00f680 con 0x557cec2e4170 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd7bfff640 1 -- 172.21.15.60:0/1976570040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c02f7d0 con 0x557cec2e4170 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- 172.21.15.60:0/1976570040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 msgr2=0x557cec2fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0108c0 tx=0x7efd8c01c560 comp rx=0 tx=0).stop 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- 172.21.15.60:0/1976570040 shutdown_connections 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2e4170 0x557cec2fbb30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2d81e0 0x557cec2fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 --2- 172.21.15.60:0/1976570040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d78c0 0x557cec2d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- 172.21.15.60:0/1976570040 >> 172.21.15.60:0/1976570040 conn(0x557cec2d56f0 msgr2=0x557cec2d3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- 172.21.15.60:0/1976570040 shutdown_connections 2024-07-24T21:53:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- 172.21.15.60:0/1976570040 wait complete. 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.040+0000 7efd9b601740 1 -- start start 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 Processor -- start 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d81e0 0x557cec2e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2e4170 0x557cec35c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efd941ae9c0 con 0x557cec2d78c0 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd941afe30 con 0x557cec2d81e0 2024-07-24T21:53:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd941af090 con 0x557cec2e4170 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9b601740 10 client.0 _ll_register_callbacks cb 0x7efd8c00a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39188/0 (socket says 172.21.15.60:39188) 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 -- 172.21.15.60:0/3539984530 learned_addr learned my addr 172.21.15.60:0/3539984530 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd91d9b640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d81e0 0x557cec2e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd92d9d640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2e4170 0x557cec35c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 -- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2e4170 msgr2=0x557cec35c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2e4170 0x557cec35c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 -- 172.21.15.60:0/3539984530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d81e0 msgr2=0x557cec2e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d81e0 0x557cec2e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 -- 172.21.15.60:0/3539984530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cec35ccf0 con 0x557cec2d78c0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd9259c640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd840e1ac0 tx=0x7efd8400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cb43b3ae680fb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.041+0000 7efd79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840aaae0 con 0x557cec2d78c0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7efd840a7260 con 0x557cec2d78c0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd8c00f680 con 0x557cec2d78c0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd840aafd0 con 0x557cec2d78c0 2024-07-24T21:53:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 10 client.16923 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557cec35e060 con 0x557cec2d78c0 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd8c00f680 con 0x557cec2d78c0 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7efd840bf030 con 0x557cec2d78c0 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 10 client.16923 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 20 client.16923 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec360820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 4 client.16923 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cec24ac70 con 0x557cec2f38f0 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.042+0000 7efd91d9b640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec360820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.043+0000 7efd79ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:38.043+0000 7efd91d9b640 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec360820 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1475ca94b0b724b6 server_cookie=2d28d171f0fa9293 in_seq=0 out_seq=0 2024-07-24T21:53:39.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:39 smithi060 ceph-mon[33000]: pgmap v589: 97 pgs: 97 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-07-24T21:53:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.623+0000 7f5710b1b640 1 -- 172.21.15.60:0/1498897040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 msgr2=0x7f570c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.623+0000 7f5710b1b640 1 --2- 172.21.15.60:0/1498897040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c10a360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5704009ca0 tx=0x7f570402a960 comp rx=0 tx=0).stop 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.624+0000 7f5710b1b640 1 -- 172.21.15.60:0/1498897040 shutdown_connections 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.624+0000 7f5710b1b640 1 --2- 172.21.15.60:0/1498897040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c11ba90 0x7f570c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.624+0000 7f5710b1b640 1 --2- 172.21.15.60:0/1498897040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c10a360 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.624+0000 7f5710b1b640 1 --2- 172.21.15.60:0/1498897040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c109b40 0x7f570c110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.624+0000 7f5710b1b640 1 -- 172.21.15.60:0/1498897040 >> 172.21.15.60:0/1498897040 conn(0x7f570c06d980 msgr2=0x7f570c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.625+0000 7f5710b1b640 1 -- 172.21.15.60:0/1498897040 shutdown_connections 2024-07-24T21:53:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.625+0000 7f5710b1b640 1 -- 172.21.15.60:0/1498897040 wait complete. 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 Processor -- start 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 -- start start 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c109b40 0x7f570c118e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 0x7f570c1121c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f570c121550 con 0x7f570c11ba90 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f570c1213d0 con 0x7f570c109f20 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.626+0000 7f5710b1b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f570c1216d0 con 0x7f570c109b40 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.628+0000 7f5709d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f5709d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40138/0 (socket says 172.21.15.60:40138) 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 0x7f570c1121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 0x7f570c1121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39202/0 (socket says 172.21.15.60:39202) 2024-07-24T21:53:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f570ad9d640 1 -- 172.21.15.60:0/2867257815 learned_addr learned my addr 172.21.15.60:0/2867257815 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f570a59c640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c109b40 0x7f570c118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f5709d9b640 1 -- 172.21.15.60:0/2867257815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c109b40 msgr2=0x7f570c118e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.629+0000 7f5709d9b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c109b40 0x7f570c118e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.630+0000 7f5709d9b640 1 -- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 msgr2=0x7f570c1121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.630+0000 7f5709d9b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 0x7f570c1121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.630+0000 7f5709d9b640 1 -- 172.21.15.60:0/2867257815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c112a80 con 0x7f570c109f20 2024-07-24T21:53:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.636+0000 7f5709d9b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f5704030c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.641+0000 7f56fb7fe640 1 -- 172.21.15.60:0/2867257815 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704047070 con 0x7f570c109f20 2024-07-24T21:53:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.641+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c112d10 con 0x7f570c109f20 2024-07-24T21:53:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.641+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c117cb0 con 0x7f570c109f20 2024-07-24T21:53:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.642+0000 7f56fb7fe640 1 -- 172.21.15.60:0/2867257815 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570402fec0 con 0x7f570c109f20 2024-07-24T21:53:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.642+0000 7f56fb7fe640 1 -- 172.21.15.60:0/2867257815 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704034040 con 0x7f570c109f20 2024-07-24T21:53:39.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.649+0000 7f5712ecc740 1 Processor -- start 2024-07-24T21:53:39.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.651+0000 7f56fb7fe640 1 -- 172.21.15.60:0/2867257815 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5704037020 con 0x7f570c109f20 2024-07-24T21:53:39.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.652+0000 7f56fb7fe640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 0x7f56f40b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.652+0000 7f56fb7fe640 1 -- 172.21.15.60:0/2867257815 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f57041041d0 con 0x7f570c109f20 2024-07-24T21:53:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.658+0000 7f570ad9d640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 0x7f56f40b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.658+0000 7f570ad9d640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 0x7f56f40b6410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f570000bbe0 tx=0x7f5700005f30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 -- start start 2024-07-24T21:53:39.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5562f3e389b0 con 0x5562f3ec16d0 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562f3e27eb0 con 0x5562f3ecdf80 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.659+0000 7f5712ecc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562f3e2bef0 con 0x5562f3ec1ff0 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.660+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.660+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40154/0 (socket says 172.21.15.60:40154) 2024-07-24T21:53:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.660+0000 7f570ad9d640 1 -- 172.21.15.60:0/3711970147 learned_addr learned my addr 172.21.15.60:0/3711970147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:39.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f5709d9b640 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 -- 172.21.15.60:0/3711970147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 msgr2=0x5562f3ecda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 -- 172.21.15.60:0/3711970147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 msgr2=0x5562f3ec1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 -- 172.21.15.60:0/3711970147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3db1320 con 0x5562f3ecdf80 2024-07-24T21:53:39.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.662+0000 7f570ad9d640 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f570000b3b0 tx=0x7f5700012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be806f95ca3cd3f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.669+0000 7f56f8ff9640 1 -- 172.21.15.60:0/3711970147 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700022070 con 0x5562f3ecdf80 2024-07-24T21:53:39.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.669+0000 7f56f8ff9640 1 -- 172.21.15.60:0/3711970147 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5700011e50 con 0x5562f3ecdf80 2024-07-24T21:53:39.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.669+0000 7f56f8ff9640 1 -- 172.21.15.60:0/3711970147 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000109c0 con 0x5562f3ecdf80 2024-07-24T21:53:39.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.670+0000 7f5712ecc740 1 -- 172.21.15.60:0/3711970147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 msgr2=0x5562f3ed0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.670+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f570000b3b0 tx=0x7f5700012000 comp rx=0 tx=0).stop 2024-07-24T21:53:39.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.678+0000 7f5712ecc740 1 -- 172.21.15.60:0/3711970147 shutdown_connections 2024-07-24T21:53:39.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.678+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ed0370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.678+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.678+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3711970147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.678+0000 7f5712ecc740 1 -- 172.21.15.60:0/3711970147 >> 172.21.15.60:0/3711970147 conn(0x5562f3ebf500 msgr2=0x5562f3ec06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.683+0000 7f5712ecc740 1 -- 172.21.15.60:0/3711970147 shutdown_connections 2024-07-24T21:53:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.686+0000 7f5712ecc740 1 -- 172.21.15.60:0/3711970147 wait complete. 2024-07-24T21:53:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.686+0000 7f5712ecc740 1 -- start start 2024-07-24T21:53:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 Processor -- start 2024-07-24T21:53:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec16d0 0x5562f3ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5562f3c8b700 con 0x5562f3ecdf80 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562f3dca5f0 con 0x5562f3ec16d0 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562f3ed0c50 con 0x5562f3ec1ff0 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39218/0 (socket says 172.21.15.60:39218) 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f570ad9d640 1 -- 172.21.15.60:0/2987298389 learned_addr learned my addr 172.21.15.60:0/2987298389 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.687+0000 7f5712ecc740 10 client.0 _ll_register_callbacks cb 0x5562f3dea960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.690+0000 7f5709d9b640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.691+0000 7f570ad9d640 1 -- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 msgr2=0x5562f3ecae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.691+0000 7f570ad9d640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.691+0000 7f570ad9d640 1 -- 172.21.15.60:0/2987298389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec16d0 msgr2=0x5562f3ecda10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.691+0000 7f570ad9d640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec16d0 0x5562f3ecda10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.691+0000 7f570ad9d640 1 -- 172.21.15.60:0/2987298389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f3dbd680 con 0x5562f3ecdf80 2024-07-24T21:53:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.692+0000 7f570ad9d640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5700010540 tx=0x7f57000053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5970506c53a02734 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.695+0000 7f56f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.695+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700022070 con 0x5562f3ecdf80 2024-07-24T21:53:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.695+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f5700010950 con 0x5562f3ecdf80 2024-07-24T21:53:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.695+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700011e50 con 0x5562f3ecdf80 2024-07-24T21:53:39.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.700+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5700038050 con 0x5562f3ecdf80 2024-07-24T21:53:39.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.701+0000 7f5712ecc740 10 client.16932 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:39.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.701+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5562f3b88650 con 0x5562f3ecdf80 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.702+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57000055e0 con 0x5562f3ecdf80 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.702+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f570001e450 con 0x5562f3ecdf80 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 10 client.16932 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 10 client.16932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 20 client.16932 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3edfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 4 client.16932 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5700011e50 con 0x5562f3edda20 2024-07-24T21:53:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.703+0000 7f570a59c640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3edfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.724+0000 7f56f1ffb640 10 client.16932 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:53:39.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.724+0000 7f570a59c640 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3edfe10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c27093ef50416805 server_cookie=d26fc04f83b9add3 in_seq=0 out_seq=0 2024-07-24T21:53:39.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:39 smithi123 ceph-mon[37143]: pgmap v589: 97 pgs: 97 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-07-24T21:53:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:39 smithi086 ceph-mon[37128]: pgmap v589: 97 pgs: 97 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.824+0000 7f56f1ffb640 1 -- 172.21.15.60:0/2987298389 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5700011e50 con 0x5562f3edda20 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f56f1ffb640 10 client.16932 handle_command_reply: tid=0 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 client.16932 shutdown 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 msgr2=0x5562f3ecb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5700010540 tx=0x7f57000053b0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 shutdown_connections 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3edfe10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ecb610 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f56f1ffb640 0 client.16932 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3ecae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 --2- 172.21.15.60:0/2987298389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec16d0 0x5562f3ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.825+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 >> 172.21.15.60:0/2987298389 conn(0x5562f3ebf500 msgr2=0x5562f3ec50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.826+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 shutdown_connections 2024-07-24T21:53:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.826+0000 7f5712ecc740 1 -- 172.21.15.60:0/2987298389 wait complete. 2024-07-24T21:53:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.826+0000 7f5712ecc740 20 client.16932 trim_cache size 0 max 16384 2024-07-24T21:53:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.853+0000 7f5712ecc740 1 Processor -- start 2024-07-24T21:53:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.853+0000 7f5712ecc740 1 -- start start 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3eccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec011320 con 0x5562f3ec16d0 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00eb90 con 0x5562f3ecdf80 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.854+0000 7f5712ecc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00ca00 con 0x5562f3ec1ff0 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40164/0 (socket says 172.21.15.60:40164) 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f570ad9d640 1 -- 172.21.15.60:0/3486146679 learned_addr learned my addr 172.21.15.60:0/3486146679 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:39.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f570a59c640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f5709d9b640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3eccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f5709d9b640 1 -- 172.21.15.60:0/3486146679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 msgr2=0x5562f3ee5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.855+0000 7f5709d9b640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.856+0000 7f5709d9b640 1 -- 172.21.15.60:0/3486146679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 msgr2=0x5562f3ec5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.856+0000 7f5709d9b640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.856+0000 7f5709d9b640 1 -- 172.21.15.60:0/3486146679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3ec1e20 con 0x5562f3ec1ff0 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.856+0000 7f570ad9d640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.856+0000 7f570a59c640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.857+0000 7f5709d9b640 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3eccba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5704101890 tx=0x7f57040ca540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb32bfdcf4dd5046 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.857+0000 7f56f3fff640 1 -- 172.21.15.60:0/3486146679 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57040ca8e0 con 0x5562f3ec1ff0 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.857+0000 7f56f3fff640 1 -- 172.21.15.60:0/3486146679 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57040ce070 con 0x5562f3ec1ff0 2024-07-24T21:53:39.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.857+0000 7f56f3fff640 1 -- 172.21.15.60:0/3486146679 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704101b10 con 0x5562f3ec1ff0 2024-07-24T21:53:39.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.858+0000 7f5712ecc740 1 -- 172.21.15.60:0/3486146679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 msgr2=0x5562f3eccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.858+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3eccba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5704101890 tx=0x7f57040ca540 comp rx=0 tx=0).stop 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.859+0000 7f5712ecc740 1 -- 172.21.15.60:0/3486146679 shutdown_connections 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.859+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ecdf80 0x5562f3ee5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.859+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec1ff0 0x5562f3eccba0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.859+0000 7f5712ecc740 1 --2- 172.21.15.60:0/3486146679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ec16d0 0x5562f3ec5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.859+0000 7f5712ecc740 1 -- 172.21.15.60:0/3486146679 >> 172.21.15.60:0/3486146679 conn(0x5562f3ebf500 msgr2=0x5562f3ecc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.860+0000 7f5712ecc740 1 -- 172.21.15.60:0/3486146679 shutdown_connections 2024-07-24T21:53:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.860+0000 7f5712ecc740 1 -- 172.21.15.60:0/3486146679 wait complete. 2024-07-24T21:53:39.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.860+0000 7f5712ecc740 1 -- start start 2024-07-24T21:53:39.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.861+0000 7f5712ecc740 1 Processor -- start 2024-07-24T21:53:39.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.861+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 0x5562f3ed67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.861+0000 7f5712ecc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 0x5562f3ed1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.862+0000 7f570a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 0x5562f3ed67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.862+0000 7f570a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 0x5562f3ed67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38616/0 (socket says 172.21.15.60:38616) 2024-07-24T21:53:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.862+0000 7f570a59c640 1 -- 172.21.15.60:0/153133083 learned_addr learned my addr 172.21.15.60:0/153133083 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.862+0000 7f5709d9b640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 0x5562f3ed1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.863+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ed23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.863+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00d050 con 0x5562f3ecdf80 2024-07-24T21:53:39.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.863+0000 7f570ad9d640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ed23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.863+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00e4c0 con 0x5562f3ec1ff0 2024-07-24T21:53:39.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00d720 con 0x5562f3ec16d0 2024-07-24T21:53:39.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5712ecc740 10 client.0 _ll_register_callbacks cb 0x7f570000fde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:39.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5709d9b640 1 -- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 msgr2=0x5562f3ed67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5709d9b640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 0x5562f3ed67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5709d9b640 1 -- 172.21.15.60:0/153133083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 msgr2=0x5562f3ed23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5709d9b640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ed23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.864+0000 7f5709d9b640 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f3f47440 con 0x5562f3ec1ff0 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.865+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57040ce070 con 0x5562f3ec1ff0 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.865+0000 7f5709d9b640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 0x5562f3ed1cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf340 tx=0x7f57040ca260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fdb7c4da60f51e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.865+0000 7f56f1ffb640 10 client.26378 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:53:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.865+0000 7f56f1ffb640 1 -- 172.21.15.60:0/153133083 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57040e9070 con 0x5562f3ec1ff0 2024-07-24T21:53:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.866+0000 7f56f1ffb640 1 -- 172.21.15.60:0/153133083 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f57040c6950 con 0x5562f3ec1ff0 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.866+0000 7f56f1ffb640 1 -- 172.21.15.60:0/153133083 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f57040ca8e0 con 0x5562f3ec1ff0 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.866+0000 7f5712ecc740 10 client.26378 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.866+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=38}) -- 0x5562f3f48480 con 0x5562f3ec1ff0 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.866+0000 7f56f1ffb640 1 -- 172.21.15.60:0/153133083 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f57040c9aa0 con 0x5562f3ec1ff0 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 10 client.26378 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 10 client.26378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 20 client.26378 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3f4abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 4 client.26378 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:53:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.867+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562f3e3a900 con 0x5562f3edda20 2024-07-24T21:53:39.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.872+0000 7f570a59c640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3f4abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:39.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.874+0000 7f570a59c640 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3f4abc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=183c689b6afe29f6 server_cookie=415c15ce83c842d5 in_seq=0 out_seq=0 2024-07-24T21:53:39.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.874+0000 7f56f1ffb640 10 client.26378 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:53:39.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f56f1ffb640 1 -- 172.21.15.60:0/153133083 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562f3e3a900 con 0x5562f3edda20 2024-07-24T21:53:39.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f56f1ffb640 10 client.26378 handle_command_reply: tid=0 2024-07-24T21:53:39.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 client.26378 shutdown 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 msgr2=0x5562f3ed1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 0x5562f3ed1cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf340 tx=0x7f57040ca260 comp rx=0 tx=0).stop 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 shutdown_connections 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5562f3edda20 0x5562f3f4abc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f56f1ffb640 0 client.26378 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562f3ecdf80 0x5562f3ed23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562f3ec1ff0 0x5562f3ed1cf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 --2- 172.21.15.60:0/153133083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562f3ec16d0 0x5562f3ed67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.885+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 >> 172.21.15.60:0/153133083 conn(0x5562f3ebf500 msgr2=0x5562f3ec92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.886+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 shutdown_connections 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.886+0000 7f5712ecc740 1 -- 172.21.15.60:0/153133083 wait complete. 2024-07-24T21:53:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.886+0000 7f5712ecc740 20 client.26378 trim_cache size 0 max 16384 2024-07-24T21:53:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:53:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 msgr2=0x7f56f40b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 0x7f56f40b6410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f570000bbe0 tx=0x7f5700005f30 comp rx=0 tx=0).stop 2024-07-24T21:53:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 msgr2=0x7f570c1193b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f5704030c00 comp rx=0 tx=0).stop 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 shutdown_connections 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f56f40b3f50 0x7f56f40b6410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f570c11ba90 0x7f570c1121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c109f20 0x7f570c1193b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 --2- 172.21.15.60:0/2867257815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570c109b40 0x7f570c118e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 >> 172.21.15.60:0/2867257815 conn(0x7f570c06d980 msgr2=0x7f570c11c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 shutdown_connections 2024-07-24T21:53:39.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:39.887+0000 7f5710b1b640 1 -- 172.21.15.60:0/2867257815 wait complete. 2024-07-24T21:53:40.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:53:41.170 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:41.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:41 smithi060 ceph-mon[33000]: pgmap v590: 97 pgs: 97 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-07-24T21:53:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:41 smithi123 ceph-mon[37143]: pgmap v590: 97 pgs: 97 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-07-24T21:53:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:41 smithi086 ceph-mon[37128]: pgmap v590: 97 pgs: 97 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 -- 172.21.15.60:0/193107256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c107a20 msgr2=0x7f1c7c10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 --2- 172.21.15.60:0/193107256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c107a20 0x7f1c7c10e220 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009ca0 tx=0x7f1c7402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 -- 172.21.15.60:0/193107256 shutdown_connections 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 --2- 172.21.15.60:0/193107256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c107a20 0x7f1c7c10e220 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 --2- 172.21.15.60:0/193107256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c1074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 --2- 172.21.15.60:0/193107256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c101690 0x7f1c7c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 -- 172.21.15.60:0/193107256 >> 172.21.15.60:0/193107256 conn(0x7f1c7c0fd100 msgr2=0x7f1c7c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 -- 172.21.15.60:0/193107256 shutdown_connections 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.040+0000 7f1c8332a640 1 -- 172.21.15.60:0/193107256 wait complete. 2024-07-24T21:53:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 Processor -- start 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 -- start start 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101690 0x7f1c7c0fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c107a20 0x7f1c7c0fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c111650 con 0x7f1c7c1070a0 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1114d0 con 0x7f1c7c101690 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c8332a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1117d0 con 0x7f1c7c107a20 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39268/0 (socket says 172.21.15.60:39268) 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c808c5640 1 -- 172.21.15.60:0/1738611896 learned_addr learned my addr 172.21.15.60:0/1738611896 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c818c7640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c107a20 0x7f1c7c0fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.041+0000 7f1c810c6640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101690 0x7f1c7c0fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 -- 172.21.15.60:0/1738611896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c107a20 msgr2=0x7f1c7c0fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c107a20 0x7f1c7c0fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 -- 172.21.15.60:0/1738611896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101690 msgr2=0x7f1c7c0fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101690 0x7f1c7c0fd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 -- 172.21.15.60:0/1738611896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c1a7140 con 0x7f1c7c1070a0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c727fc640 1 -- 172.21.15.60:0/1738611896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00cd80 con 0x7f1c7c1070a0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c727fc640 1 -- 172.21.15.60:0/1738611896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c0108e0 con 0x7f1c7c1070a0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1a7430 con 0x7f1c7c1070a0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c727fc640 1 -- 172.21.15.60:0/1738611896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c018980 con 0x7f1c7c1070a0 2024-07-24T21:53:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.042+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c1a7970 con 0x7f1c7c1070a0 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c727fc640 1 -- 172.21.15.60:0/1738611896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1c6c010430 con 0x7f1c7c1070a0 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 Processor -- start 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 -- start start 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a80ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a80e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55978a7769b0 con 0x55978a7ffff0 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55978a765eb0 con 0x55978a80bf80 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c856db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55978a769ef0 con 0x55978a7ff6d0 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c818c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c818c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38632/0 (socket says 172.21.15.60:38632) 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c818c7640 1 -- 172.21.15.60:0/2646719131 learned_addr learned my addr 172.21.15.60:0/2646719131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.045+0000 7f1c808c5640 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a80e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 -- 172.21.15.60:0/2646719131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 msgr2=0x55978a80e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a80e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 -- 172.21.15.60:0/2646719131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 msgr2=0x55978a80ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a80ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 -- 172.21.15.60:0/2646719131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978a6ef320 con 0x55978a7ff6d0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c808c5640 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a80e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c727fc640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 0x7f1c680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c810c6640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 0x7f1c680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c818c7640 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c74035000 tx=0x7f1c74005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56dd643466473460 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c727fc640 1 -- 172.21.15.60:0/1738611896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f1c6c0e42b0 con 0x7f1c7c1070a0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c677fe640 1 -- 172.21.15.60:0/2646719131 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74031cf0 con 0x55978a7ff6d0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c677fe640 1 -- 172.21.15.60:0/2646719131 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c74036030 con 0x55978a7ff6d0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.046+0000 7f1c677fe640 1 -- 172.21.15.60:0/2646719131 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74004040 con 0x55978a7ff6d0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- 172.21.15.60:0/2646719131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 msgr2=0x55978a7ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c810c6640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 0x7f1c680b6200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55978a7fe3e0 tx=0x7f1c78009840 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c74035000 tx=0x7f1c74005ef0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- 172.21.15.60:0/2646719131 shutdown_connections 2024-07-24T21:53:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a80e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a80ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 --2- 172.21.15.60:0/2646719131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a7ffab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- 172.21.15.60:0/2646719131 >> 172.21.15.60:0/2646719131 conn(0x55978a7fd500 msgr2=0x55978a7fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- 172.21.15.60:0/2646719131 shutdown_connections 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- 172.21.15.60:0/2646719131 wait complete. 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.047+0000 7f1c856db740 1 -- start start 2024-07-24T21:53:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 Processor -- start 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 0x55978a808ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a80bf80 0x55978a809950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55978a5c9700 con 0x55978a80bf80 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55978a7085f0 con 0x55978a7ff6d0 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55978a80ea20 con 0x55978a7ffff0 2024-07-24T21:53:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c856db740 10 client.0 _ll_register_callbacks cb 0x55978a728960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38634/0 (socket says 172.21.15.60:38634) 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c808c5640 1 -- 172.21.15.60:0/441053248 learned_addr learned my addr 172.21.15.60:0/441053248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.048+0000 7f1c810c6640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 0x55978a808ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 -- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 msgr2=0x55978a808ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 0x55978a808ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 -- 172.21.15.60:0/441053248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a80bf80 msgr2=0x55978a809950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a80bf80 0x55978a809950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 -- 172.21.15.60:0/441053248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978a6fb680 con 0x55978a7ffff0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c810c6640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 0x55978a808ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c808c5640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0a81b0 tx=0x7f1c6c0aadd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1037b8df33e23cf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0aaa50 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f1c6c0a71d0 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c74036030 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c0aaf40 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c856db740 10 client.26419 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55978a4c6650 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.049+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c74036030 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1c6c018bb0 con 0x55978a7ffff0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 10 client.26419 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 10 client.26419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 20 client.26419 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a81b8a0 0x55978a81dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 4 client.26419 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55978a778900 con 0x55978a81b8a0 2024-07-24T21:53:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c810c6640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a81b8a0 0x55978a81dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c64ff9640 10 client.26419 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.050+0000 7f1c810c6640 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a81b8a0 0x55978a81dd10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=393cc58024a296a8 server_cookie=b68959070c451973 in_seq=0 out_seq=0 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c64ff9640 1 -- 172.21.15.60:0/441053248 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55978a778900 con 0x55978a81b8a0 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c64ff9640 10 client.26419 handle_command_reply: tid=0 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 client.26419 shutdown 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 msgr2=0x55978a809410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0a81b0 tx=0x7f1c6c0aadd0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 shutdown_connections 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a81b8a0 0x55978a81dd10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a80bf80 0x55978a809950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ffff0 0x55978a809410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c64ff9640 0 client.26419 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:53:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 --2- 172.21.15.60:0/441053248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ff6d0 0x55978a808ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 >> 172.21.15.60:0/441053248 conn(0x55978a7fd500 msgr2=0x55978a8030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 shutdown_connections 2024-07-24T21:53:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.051+0000 7f1c856db740 1 -- 172.21.15.60:0/441053248 wait complete. 2024-07-24T21:53:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.052+0000 7f1c856db740 20 client.26419 trim_cache size 0 max 16384 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.065+0000 7f1c856db740 1 Processor -- start 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.065+0000 7f1c856db740 1 -- start start 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 0x55978a823620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a80bf80 0x55978a823b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1b2ce0 con 0x55978a7ff6d0 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39300/0 (socket says 172.21.15.60:39300) 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 -- 172.21.15.60:0/105216507 learned_addr learned my addr 172.21.15.60:0/105216507 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c808c5640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 0x55978a823620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c818c7640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a80bf80 0x55978a823b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1b0550 con 0x55978a7ffff0 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1ae3c0 con 0x55978a80bf80 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 -- 172.21.15.60:0/105216507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a80bf80 msgr2=0x55978a823b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a80bf80 0x55978a823b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 -- 172.21.15.60:0/105216507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 msgr2=0x55978a823620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 0x55978a823620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c810c6640 1 -- 172.21.15.60:0/105216507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978a81e390 con 0x55978a7ff6d0 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.066+0000 7f1c808c5640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 0x55978a823620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c810c6640 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009b60 tx=0x7f1c78014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3bf00c132adf57 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c66ffd640 1 -- 172.21.15.60:0/105216507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78031420 con 0x55978a7ff6d0 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c66ffd640 1 -- 172.21.15.60:0/105216507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c78031a70 con 0x55978a7ff6d0 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c66ffd640 1 -- 172.21.15.60:0/105216507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78028a80 con 0x55978a7ff6d0 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 msgr2=0x55978a80afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009b60 tx=0x7f1c78014000 comp rx=0 tx=0).stop 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 shutdown_connections 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a80bf80 0x55978a823b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a7ffff0 0x55978a823620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 --2- 172.21.15.60:0/105216507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ff6d0 0x55978a80afd0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.067+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 >> 172.21.15.60:0/105216507 conn(0x55978a7fd500 msgr2=0x55978a803030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 shutdown_connections 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- 172.21.15.60:0/105216507 wait complete. 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- start start 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 Processor -- start 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a80fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a8855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1aea10 con 0x55978a7ffff0 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1afe80 con 0x55978a80bf80 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1af0e0 con 0x55978a7ff6d0 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.068+0000 7f1c856db740 10 client.0 _ll_register_callbacks cb 0x7f1c6c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c810c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a80fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39316/0 (socket says 172.21.15.60:39316) 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 -- 172.21.15.60:0/1871581545 learned_addr learned my addr 172.21.15.60:0/1871581545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c818c7640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a8855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 -- 172.21.15.60:0/1871581545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 msgr2=0x55978a80fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a80fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 -- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 msgr2=0x55978a8855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a8855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 -- 172.21.15.60:0/1871581545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978a810c20 con 0x55978a7ffff0 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c818c7640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a8855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c808c5640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c023040 tx=0x7f1c6c0e1ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476dca335435db9b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c018d30 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f1c6c0aa1a0 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c78031a70 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c0aaa50 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c856db740 10 client.16950 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.069+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55978a886c10 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c78031a70 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1c6c0cb020 con 0x55978a7ffff0 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 10 client.16950 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 10 client.16950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 20 client.16950 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a887220 0x55978a889640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 4 client.16950 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:53:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55978a7743a0 con 0x55978a887220 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.070+0000 7f1c810c6640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a887220 0x55978a889640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c810c6640 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a887220 0x55978a889640 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaecd1a917c1d8f6 server_cookie=d309cff10641d7a0 in_seq=0 out_seq=0 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c64ff9640 10 client.16950 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c64ff9640 1 -- 172.21.15.60:0/1871581545 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55978a7743a0 con 0x55978a887220 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c64ff9640 10 client.16950 handle_command_reply: tid=0 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 client.16950 shutdown 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 msgr2=0x55978a810370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c023040 tx=0x7f1c6c0e1ed0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 shutdown_connections 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55978a887220 0x55978a889640 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55978a80bf80 0x55978a8855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55978a7ffff0 0x55978a810370 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 --2- 172.21.15.60:0/1871581545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55978a7ff6d0 0x55978a80fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 >> 172.21.15.60:0/1871581545 conn(0x55978a7fd500 msgr2=0x55978a7fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c64ff9640 0 client.16950 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 shutdown_connections 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 1 -- 172.21.15.60:0/1871581545 wait complete. 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.071+0000 7f1c856db740 20 client.16950 trim_cache size 0 max 16384 2024-07-24T21:53:43.078 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 msgr2=0x7f1c680b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 0x7f1c680b6200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55978a7fe3e0 tx=0x7f1c78009840 comp rx=0 tx=0).stop 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 msgr2=0x7f1c7c0fdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 shutdown_connections 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1c680b3d40 0x7f1c680b6200 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c7c107a20 0x7f1c7c0fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c7c1070a0 0x7f1c7c0fdb40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 --2- 172.21.15.60:0/1738611896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101690 0x7f1c7c0fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 >> 172.21.15.60:0/1738611896 conn(0x7f1c7c0fd100 msgr2=0x7f1c7c06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 shutdown_connections 2024-07-24T21:53:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:43.072+0000 7f1c8332a640 1 -- 172.21.15.60:0/1738611896 wait complete. 2024-07-24T21:53:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:43 smithi123 ceph-mon[37143]: pgmap v591: 97 pgs: 97 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-07-24T21:53:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:43 smithi086 ceph-mon[37128]: pgmap v591: 97 pgs: 97 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-07-24T21:53:43.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:43 smithi060 ceph-mon[33000]: pgmap v591: 97 pgs: 97 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-07-24T21:53:44.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:53:44.408 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:45 smithi123 ceph-mon[37143]: pgmap v592: 97 pgs: 97 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-07-24T21:53:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:45 smithi086 ceph-mon[37128]: pgmap v592: 97 pgs: 97 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-07-24T21:53:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:45 smithi060 ceph-mon[33000]: pgmap v592: 97 pgs: 97 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 754 op/s 2024-07-24T21:53:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.488+0000 7efd79ffb640 1 -- 172.21.15.60:0/3539984530 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557cec24ac70 con 0x557cec2f38f0 2024-07-24T21:53:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.488+0000 7efd79ffb640 10 client.16923 handle_command_reply: tid=0 2024-07-24T21:53:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.488+0000 7efd9b601740 1 client.16923 shutdown 2024-07-24T21:53:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 msgr2=0x557cec2e7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd840e1ac0 tx=0x7efd8400cf20 comp rx=0 tx=0).stop 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 shutdown_connections 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557cec2f38f0 0x557cec360820 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cec2e4170 0x557cec35c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd79ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cec2d81e0 0x557cec2e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 --2- 172.21.15.60:0/3539984530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557cec2d78c0 0x557cec2e7680 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 >> 172.21.15.60:0/3539984530 conn(0x557cec2d56f0 msgr2=0x557cec2d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 shutdown_connections 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 1 -- 172.21.15.60:0/3539984530 wait complete. 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.489+0000 7efd9b601740 20 client.16923 trim_cache size 0 max 16384 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:53:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 msgr2=0x7efd7c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 0x7efd7c0b6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005fd0 tx=0x7efd8c005f60 comp rx=0 tx=0).stop 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 msgr2=0x7efd940fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd8400e9f0 tx=0x7efd8400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 shutdown_connections 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7efd7c0b3d40 0x7efd7c0b6200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd94107910 0x7efd940fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd94106f90 0x7efd940fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 --2- 172.21.15.60:0/1366962735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd941015e0 0x7efd940fd780 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 >> 172.21.15.60:0/1366962735 conn(0x7efd940782e0 msgr2=0x7efd940698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 shutdown_connections 2024-07-24T21:53:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:46.491+0000 7efd988a9640 1 -- 172.21.15.60:0/1366962735 wait complete. 2024-07-24T21:53:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/262790978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78101670 msgr2=0x7f7d78101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/262790978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78101670 0x7f7d78101a50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009900 tx=0x7f7d6802a410 comp rx=0 tx=0).stop 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/262790978 shutdown_connections 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/262790978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78107a00 0x7f7d7810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/262790978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d781074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/262790978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78101670 0x7f7d78101a50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/262790978 >> 172.21.15.60:0/262790978 conn(0x7f7d780fd0c0 msgr2=0x7f7d780ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/262790978 shutdown_connections 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.566+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/262790978 wait complete. 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.567+0000 7f7d7e4f4640 1 Processor -- start 2024-07-24T21:53:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.567+0000 7f7d7e4f4640 1 -- start start 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 0x7f7d7810d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d7810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 0x7f7d781a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d7810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 0x7f7d7810d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781127e0 con 0x7f7d78107a00 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78112660 con 0x7f7d78101670 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7e4f4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78112960 con 0x7f7d78107080 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 0x7f7d7810d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36190/0 (socket says 172.21.15.60:36190) 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 0x7f7d781a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d77fff640 1 -- 172.21.15.60:0/363566226 learned_addr learned my addr 172.21.15.60:0/363566226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d7810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55126/0 (socket says 172.21.15.60:55126) 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 -- 172.21.15.60:0/363566226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 msgr2=0x7f7d7810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d7810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 -- 172.21.15.60:0/363566226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 msgr2=0x7f7d7810d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 0x7f7d7810d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.568+0000 7f7d7ca91640 1 -- 172.21.15.60:0/363566226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d781a20b0 con 0x7f7d78107a00 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 0x7f7d781a18e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009fd0 tx=0x7f7d7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d757fa640 1 -- 172.21.15.60:0/363566226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000ce20 con 0x7f7d78107a00 2024-07-24T21:53:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d781a23a0 con 0x7f7d78107a00 2024-07-24T21:53:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d781abca0 con 0x7f7d78107a00 2024-07-24T21:53:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d757fa640 1 -- 172.21.15.60:0/363566226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d70005020 con 0x7f7d78107a00 2024-07-24T21:53:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.569+0000 7f7d757fa640 1 -- 172.21.15.60:0/363566226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70010670 con 0x7f7d78107a00 2024-07-24T21:53:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 Processor -- start 2024-07-24T21:53:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d757fa640 1 -- 172.21.15.60:0/363566226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7d70004a40 con 0x7f7d78107a00 2024-07-24T21:53:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 -- start start 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768f9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 0x55d768fad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d768f159b0 con 0x55d768f9e6d0 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d768f04eb0 con 0x55d768faaf80 2024-07-24T21:53:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d808a5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d768f08ef0 con 0x55d768f9eff0 2024-07-24T21:53:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.573+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55130/0 (socket says 172.21.15.60:55130) 2024-07-24T21:53:46.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d77fff640 1 -- 172.21.15.60:0/1854877344 learned_addr learned my addr 172.21.15.60:0/1854877344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:46.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 0x55d768fad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768f9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 -- 172.21.15.60:0/1854877344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 msgr2=0x55d768faaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 -- 172.21.15.60:0/1854877344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 msgr2=0x55d768f9eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768f9eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 -- 172.21.15.60:0/1854877344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d768e8e320 con 0x55d768faaf80 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d77fff640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.574+0000 7f7d777fe640 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 0x55d768fad370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0126c0 tx=0x7f7d6c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f867a54ed7a4427 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d757fa640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 0x7f7d600b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d5e7fc640 1 -- 172.21.15.60:0/1854877344 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c012c40 con 0x55d768faaf80 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d5e7fc640 1 -- 172.21.15.60:0/1854877344 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d6c012de0 con 0x55d768faaf80 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d5e7fc640 1 -- 172.21.15.60:0/1854877344 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c004a00 con 0x55d768faaf80 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d77fff640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 0x7f7d600b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d757fa640 1 -- 172.21.15.60:0/363566226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f7d700e4490 con 0x7f7d78107a00 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d808a5740 1 -- 172.21.15.60:0/1854877344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 msgr2=0x55d768fad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d808a5740 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 0x55d768fad370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0126c0 tx=0x7f7d6c009650 comp rx=0 tx=0).stop 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.575+0000 7f7d77fff640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 0x7f7d600b6280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d768f9d3e0 tx=0x7f7d680058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 -- 172.21.15.60:0/1854877344 shutdown_connections 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768faaf80 0x55d768fad370 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9eff0 0x55d768faaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 --2- 172.21.15.60:0/1854877344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768f9eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 -- 172.21.15.60:0/1854877344 >> 172.21.15.60:0/1854877344 conn(0x55d768f9c500 msgr2=0x55d768f9d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 -- 172.21.15.60:0/1854877344 shutdown_connections 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 -- 172.21.15.60:0/1854877344 wait complete. 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 -- start start 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.576+0000 7f7d808a5740 1 Processor -- start 2024-07-24T21:53:46.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768fa7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d768d68700 con 0x55d768f9e6d0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d768ea75f0 con 0x55d768f9eff0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d768fadc00 con 0x55d768faaf80 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d808a5740 10 client.0 _ll_register_callbacks cb 0x55d768ec7960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36198/0 (socket says 172.21.15.60:36198) 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 -- 172.21.15.60:0/3364919986 learned_addr learned my addr 172.21.15.60:0/3364919986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768fa7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 -- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 msgr2=0x55d768fa8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 -- 172.21.15.60:0/3364919986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 msgr2=0x55d768fa7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768fa7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d777fe640 1 -- 172.21.15.60:0/3364919986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d768e9a680 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.577+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768fa7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005590 tx=0x7f7d6c010a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c32324a564114cc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d577fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c012c40 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6c012de0 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f7d6c010db0 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c012020 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d808a5740 10 client.26408 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d768c65650 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.578+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d6c012de0 con 0x55d768f9eff0 2024-07-24T21:53:46.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f7d6c02f450 con 0x55d768f9eff0 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 10 client.26408 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 10 client.26408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 20 client.26408 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d768fbaa30 0x55d768fbce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 4 client.26408 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d768f17900 con 0x55d768fbaa30 2024-07-24T21:53:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d768fbaa30 0x55d768fbce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.579+0000 7f7d577fe640 10 client.26408 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.580+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d768fbaa30 0x55d768fbce20 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be92d1e937ddfaab server_cookie=6fa738e4d8755bf8 in_seq=0 out_seq=0 2024-07-24T21:53:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.581+0000 7f7d577fe640 1 -- 172.21.15.60:0/3364919986 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d768f17900 con 0x55d768fbaa30 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.581+0000 7f7d577fe640 10 client.26408 handle_command_reply: tid=0 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 client.26408 shutdown 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 msgr2=0x55d768fa8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005590 tx=0x7f7d6c010a00 comp rx=0 tx=0).stop 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 shutdown_connections 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d768fbaa30 0x55d768fbce20 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fa8430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3364919986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9e6d0 0x55d768fa7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d577fe640 0 client.26408 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 >> 172.21.15.60:0/3364919986 conn(0x55d768f9c500 msgr2=0x55d768fa20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 shutdown_connections 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 1 -- 172.21.15.60:0/3364919986 wait complete. 2024-07-24T21:53:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.582+0000 7f7d808a5740 20 client.26408 trim_cache size 0 max 16384 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 Processor -- start 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 -- start start 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fc25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640111e0 con 0x55d768faaf80 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400ea50 con 0x55d768f9eff0 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.612+0000 7f7d808a5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400c8c0 con 0x55d768f9e6d0 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d7ca91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fc25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55136/0 (socket says 172.21.15.60:55136) 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 -- 172.21.15.60:0/4112905207 learned_addr learned my addr 172.21.15.60:0/4112905207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d7ca91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39388/0 (socket says 172.21.15.60:39388) 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 -- 172.21.15.60:0/4112905207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 msgr2=0x55d768fc25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fc25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 -- 172.21.15.60:0/4112905207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 msgr2=0x55d768fc2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 -- 172.21.15.60:0/4112905207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d768f9ee20 con 0x55d768f9e6d0 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d777fe640 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fc25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.613+0000 7f7d77fff640 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009230 tx=0x7f7d6803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c393107cccbefb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.614+0000 7f7d5dffb640 1 -- 172.21.15.60:0/4112905207 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68035040 con 0x55d768f9e6d0 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.614+0000 7f7d5dffb640 1 -- 172.21.15.60:0/4112905207 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d68004240 con 0x55d768f9e6d0 2024-07-24T21:53:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.614+0000 7f7d5dffb640 1 -- 172.21.15.60:0/4112905207 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68053720 con 0x55d768f9e6d0 2024-07-24T21:53:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- 172.21.15.60:0/4112905207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 msgr2=0x55d768fa2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009230 tx=0x7f7d6803f000 comp rx=0 tx=0).stop 2024-07-24T21:53:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- 172.21.15.60:0/4112905207 shutdown_connections 2024-07-24T21:53:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768faaf80 0x55d768fc2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9eff0 0x55d768fc25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 --2- 172.21.15.60:0/4112905207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768f9e6d0 0x55d768fa2820 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- 172.21.15.60:0/4112905207 >> 172.21.15.60:0/4112905207 conn(0x55d768f9c500 msgr2=0x55d768fa9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- 172.21.15.60:0/4112905207 shutdown_connections 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- 172.21.15.60:0/4112905207 wait complete. 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 -- start start 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.615+0000 7f7d808a5740 1 Processor -- start 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 0x55d768fb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 0x55d768fa4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400cf10 con 0x55d768f9eff0 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400e380 con 0x55d768f9e6d0 2024-07-24T21:53:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400d5e0 con 0x55d768faaf80 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d808a5740 10 client.0 _ll_register_callbacks cb 0x7f7d6c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55146/0 (socket says 172.21.15.60:55146) 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 0x55d768fa4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 -- 172.21.15.60:0/3300277799 learned_addr learned my addr 172.21.15.60:0/3300277799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 0x55d768fb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 -- 172.21.15.60:0/3300277799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 msgr2=0x55d768fb3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 0x55d768fb3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 -- 172.21.15.60:0/3300277799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 msgr2=0x55d768fa4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 0x55d768fa4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d7ca91640 1 -- 172.21.15.60:0/3300277799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d768fa56f0 con 0x55d768faaf80 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 0x55d768fa4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.616+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 0x55d768fb3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d7ca91640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c760 tx=0x7f7d700ab7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e858e355a60a37ae server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d577fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700b40c0 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f7d700b3e40 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d68004240 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7d700c5b50 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d808a5740 10 client.26443 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d7690254c0 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.617+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d68004240 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f7d700cc070 con 0x55d768faaf80 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 10 client.26443 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:46.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 20 client.26443 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:46.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d769025be0 0x55d769028000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:46.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 4 client.26443 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:53:46.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d768f133a0 con 0x55d769025be0 2024-07-24T21:53:46.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d769025be0 0x55d769028000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d577fe640 10 client.26443 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.618+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d769025be0 0x55d769028000 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfa0a23937a2971 server_cookie=50d4e8c050a722a8 in_seq=0 out_seq=0 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d577fe640 1 -- 172.21.15.60:0/3300277799 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d768f133a0 con 0x55d769025be0 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d577fe640 10 client.26443 handle_command_reply: tid=0 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 client.26443 shutdown 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 msgr2=0x55d768fa5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c760 tx=0x7f7d700ab7d0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 shutdown_connections 2024-07-24T21:53:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d769025be0 0x55d769028000 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d768faaf80 0x55d768fa5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d768f9eff0 0x55d768fa4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 --2- 172.21.15.60:0/3300277799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d768f9e6d0 0x55d768fb3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 >> 172.21.15.60:0/3300277799 conn(0x55d768f9c500 msgr2=0x55d768fa7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d577fe640 0 client.26443 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 shutdown_connections 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 1 -- 172.21.15.60:0/3300277799 wait complete. 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.619+0000 7f7d808a5740 20 client.26443 trim_cache size 0 max 16384 2024-07-24T21:53:46.625 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:53:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 msgr2=0x7f7d600b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 0x7f7d600b6280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d768f9d3e0 tx=0x7f7d680058d0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 msgr2=0x7f7d781a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 0x7f7d781a18e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009fd0 tx=0x7f7d7000ef10 comp rx=0 tx=0).stop 2024-07-24T21:53:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 shutdown_connections 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7d600b3dc0 0x7f7d600b6280 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d78107a00 0x7f7d781a18e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d78107080 0x7f7d7810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 --2- 172.21.15.60:0/363566226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d78101670 0x7f7d7810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 >> 172.21.15.60:0/363566226 conn(0x7f7d780fd0c0 msgr2=0x7f7d78103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.620+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 shutdown_connections 2024-07-24T21:53:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:46.621+0000 7f7d7e4f4640 1 -- 172.21.15.60:0/363566226 wait complete. 2024-07-24T21:53:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:47 smithi123 ceph-mon[37143]: pgmap v593: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-07-24T21:53:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:47 smithi086 ceph-mon[37128]: pgmap v593: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-07-24T21:53:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:47 smithi060 ceph-mon[33000]: pgmap v593: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 939 op/s 2024-07-24T21:53:48.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:53:48.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:53:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:49.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:49 smithi060 ceph-mon[33000]: pgmap v594: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-07-24T21:53:49.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:49 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:53:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:49 smithi123 ceph-mon[37143]: pgmap v594: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-07-24T21:53:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:49 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:53:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:49 smithi086 ceph-mon[37128]: pgmap v594: 97 pgs: 97 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-07-24T21:53:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:49 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:53:50.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f541eff5640 1 -- 172.21.15.60:0/1394609547 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5599cd6a8c50 con 0x5599cd74a9e0 2024-07-24T21:53:50.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f541eff5640 10 client.16881 handle_command_reply: tid=0 2024-07-24T21:53:50.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 client.16881 shutdown 2024-07-24T21:53:50.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 msgr2=0x5599cd6b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:50.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f54300abad0 tx=0x7f543000bf60 comp rx=0 tx=0).stop 2024-07-24T21:53:50.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 shutdown_connections 2024-07-24T21:53:50.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5599cd74a9e0 0x5599cd74ce00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cd6e7220 0x5599cd6af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5599cd6ac5d0 0x5599cd6b4d40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 --2- 172.21.15.60:0/1394609547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599cd6a1d00 0x5599cd6b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f541eff5640 0 client.16881 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.071+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 >> 172.21.15.60:0/1394609547 conn(0x5599cd66a750 msgr2=0x5599cd69d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 shutdown_connections 2024-07-24T21:53:50.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5443945740 1 -- 172.21.15.60:0/1394609547 wait complete. 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5443945740 20 client.16881 trim_cache size 0 max 16384 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 38, 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 18, 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "b8eda14d": { 2024-07-24T21:53:50.076 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 18, 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:53:50.077 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:50.078 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 msgr2=0x7f54240b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 0x7f54240b6410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f5434009bb0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 msgr2=0x7f543c074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f543000d950 tx=0x7f543000de10 comp rx=0 tx=0).stop 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 shutdown_connections 2024-07-24T21:53:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f54240b3f50 0x7f54240b6410 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f543c1079f0 0x7f543c06d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c107070 0x7f543c074650 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 --2- 172.21.15.60:0/4121997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f543c101660 0x7f543c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.072+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 >> 172.21.15.60:0/4121997238 conn(0x7f543c0fd0f0 msgr2=0x7f543c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.073+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 shutdown_connections 2024-07-24T21:53:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:50.073+0000 7f5441594640 1 -- 172.21.15.60:0/4121997238 wait complete. 2024-07-24T21:53:51.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:51 smithi060 ceph-mon[33000]: pgmap v595: 97 pgs: 97 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-07-24T21:53:51.536 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b8eda14d', quiesce took 64.5 seconds. Will release after: -34.5 2024-07-24T21:53:51.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:53:51.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:51 smithi123 ceph-mon[37143]: pgmap v595: 97 pgs: 97 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-07-24T21:53:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:51 smithi086 ceph-mon[37128]: pgmap v595: 97 pgs: 97 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-07-24T21:53:52.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 -- 172.21.15.60:0/1443661998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103f80 msgr2=0x7fabb010c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 --2- 172.21.15.60:0/1443661998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103f80 0x7fabb010c470 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faba4009900 tx=0x7faba402a410 comp rx=0 tx=0).stop 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 -- 172.21.15.60:0/1443661998 shutdown_connections 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 --2- 172.21.15.60:0/1443661998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 --2- 172.21.15.60:0/1443661998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103f80 0x7fabb010c470 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 --2- 172.21.15.60:0/1443661998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103660 0x7fabb0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 -- 172.21.15.60:0/1443661998 >> 172.21.15.60:0/1443661998 conn(0x7fabb00fd0c0 msgr2=0x7fabb00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 -- 172.21.15.60:0/1443661998 shutdown_connections 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.754+0000 7fabb5407640 1 -- 172.21.15.60:0/1443661998 wait complete. 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 Processor -- start 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 -- start start 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103f80 0x7fabb0101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb0101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112790 con 0x7fabb0103660 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112610 con 0x7fabb0103f80 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabb5407640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112910 con 0x7fabb010cac0 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39416/0 (socket says 172.21.15.60:39416) 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabaeffd640 1 -- 172.21.15.60:0/2933157923 learned_addr learned my addr 172.21.15.60:0/2933157923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb0101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.755+0000 7fabae7fc640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103f80 0x7fabb0101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 -- 172.21.15.60:0/2933157923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 msgr2=0x7fabb0101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb0101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 -- 172.21.15.60:0/2933157923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103f80 msgr2=0x7fabb0101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103f80 0x7fabb0101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 -- 172.21.15.60:0/2933157923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb01a70e0 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faba000b4a0 tx=0x7faba000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fab97fff640 1 -- 172.21.15.60:0/2933157923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0014070 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fab97fff640 1 -- 172.21.15.60:0/2933157923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0010ac0 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01a73d0 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fab97fff640 1 -- 172.21.15.60:0/2933157923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0017c30 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.756+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a7960 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fab97fff640 1 -- 172.21.15.60:0/2933157923 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7fabb0103660 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 Processor -- start 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 -- start start 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422636310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x565422644bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5654225ad9f0 con 0x565422636850 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56542259cef0 con 0x565422635f30 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabb77b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5654225a0f30 con 0x5654226427e0 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.759+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39426/0 (socket says 172.21.15.60:39426) 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 -- 172.21.15.60:0/235538063 learned_addr learned my addr 172.21.15.60:0/235538063 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fab97fff640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 0x7fab980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422636310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 0x7fab980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fab97fff640 1 -- 172.21.15.60:0/2933157923 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7faba00e3760 con 0x7fabb0103660 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabae7fc640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x565422644bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 -- 172.21.15.60:0/235538063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 msgr2=0x565422644bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x565422644bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 -- 172.21.15.60:0/235538063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 msgr2=0x565422636310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422636310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 -- 172.21.15.60:0/235538063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654222fd930 con 0x565422636850 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422636310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaeffd640 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faba00a7660 tx=0x7faba0017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975363a66a1b6865 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 0x7fab980b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fabb01033d0 tx=0x7faba8004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fab957fa640 1 -- 172.21.15.60:0/235538063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba001e070 con 0x565422636850 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fab957fa640 1 -- 172.21.15.60:0/235538063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00aaaf0 con 0x565422636850 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.760+0000 7fab957fa640 1 -- 172.21.15.60:0/235538063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0078040 con 0x565422636850 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 -- 172.21.15.60:0/235538063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 msgr2=0x5654226422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faba00a7660 tx=0x7faba0017650 comp rx=0 tx=0).stop 2024-07-24T21:53:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 -- 172.21.15.60:0/235538063 shutdown_connections 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x565422644bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x5654226422a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 --2- 172.21.15.60:0/235538063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422636310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 -- 172.21.15.60:0/235538063 >> 172.21.15.60:0/235538063 conn(0x565422633d60 msgr2=0x565422634f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 -- 172.21.15.60:0/235538063 shutdown_connections 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.761+0000 7fabb77b8740 1 -- 172.21.15.60:0/235538063 wait complete. 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 -- start start 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 Processor -- start 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x56542263f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565422636850 0x56542263fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x565422400890 con 0x5654226427e0 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56542253f390 con 0x565422635f30 2024-07-24T21:53:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565422645400 con 0x565422636850 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabb77b8740 10 client.0 _ll_register_callbacks cb 0x565422549500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565422636850 0x56542263fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39442/0 (socket says 172.21.15.60:39442) 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 -- 172.21.15.60:0/2634326164 learned_addr learned my addr 172.21.15.60:0/2634326164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x56542263f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 -- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565422636850 msgr2=0x56542263fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565422636850 0x56542263fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 -- 172.21.15.60:0/2634326164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 msgr2=0x56542263f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x56542263f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaf7fe640 1 -- 172.21.15.60:0/2634326164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565422539d60 con 0x5654226427e0 2024-07-24T21:53:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.762+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x56542263f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faba8012c80 tx=0x7faba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b2cbd73bff9d9b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba801a070 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7faba8029600 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba00aaaf0 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faba8029840 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabb77b8740 10 client.16980 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5654226513d0 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00aaaf0 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7faba800f040 con 0x5654226427e0 2024-07-24T21:53:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabb77b8740 10 client.16980 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.763+0000 7fabb77b8740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabb77b8740 20 client.16980 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226528e0 0x565422654d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabb77b8740 4 client.16980 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654225af940 con 0x5654226528e0 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226528e0 0x565422654d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fab92ff5640 10 client.16980 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.764+0000 7fabaeffd640 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226528e0 0x565422654d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd83636603dabd11 server_cookie=a0f926f786f89c5a in_seq=0 out_seq=0 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fab92ff5640 1 -- 172.21.15.60:0/2634326164 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654225af940 con 0x5654226528e0 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fab92ff5640 10 client.16980 handle_command_reply: tid=0 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 client.16980 shutdown 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 msgr2=0x565422640240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faba8012c80 tx=0x7faba8004a40 comp rx=0 tx=0).stop 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 shutdown_connections 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226528e0 0x565422654d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654226427e0 0x565422640240 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fab92ff5640 0 client.16980 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565422636850 0x56542263fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 --2- 172.21.15.60:0/2634326164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x56542263f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 >> 172.21.15.60:0/2634326164 conn(0x565422633d60 msgr2=0x565422639900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 shutdown_connections 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 1 -- 172.21.15.60:0/2634326164 wait complete. 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.765+0000 7fabb77b8740 20 client.16980 trim_cache size 0 max 16384 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 Processor -- start 2024-07-24T21:53:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 -- start start 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 0x56542265a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x56542265ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b2de0 con 0x565422635f30 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b0650 con 0x565422636850 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabb77b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01ae4c0 con 0x5654226427e0 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39450/0 (socket says 172.21.15.60:39450) 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 -- 172.21.15.60:0/869334245 learned_addr learned my addr 172.21.15.60:0/869334245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabae7fc640 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 0x56542265a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 -- 172.21.15.60:0/869334245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 msgr2=0x56542265ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x56542265ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 -- 172.21.15.60:0/869334245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 msgr2=0x56542265a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 0x56542265a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 -- 172.21.15.60:0/869334245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56542262e2a0 con 0x565422635f30 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabae7fc640 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 0x56542265a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fabaeffd640 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faba0009420 tx=0x7faba0010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51d8bfdf9313efc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.784+0000 7fab94ff9640 1 -- 172.21.15.60:0/869334245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00e9020 con 0x565422635f30 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fab94ff9640 1 -- 172.21.15.60:0/869334245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0078040 con 0x565422635f30 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fab94ff9640 1 -- 172.21.15.60:0/869334245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba001e040 con 0x565422635f30 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 -- 172.21.15.60:0/869334245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 msgr2=0x56542263a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faba0009420 tx=0x7faba0010f90 comp rx=0 tx=0).stop 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 -- 172.21.15.60:0/869334245 shutdown_connections 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x56542265ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422636850 0x56542265a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 --2- 172.21.15.60:0/869334245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422635f30 0x56542263a080 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 -- 172.21.15.60:0/869334245 >> 172.21.15.60:0/869334245 conn(0x565422633d60 msgr2=0x56542263ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 -- 172.21.15.60:0/869334245 shutdown_connections 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.785+0000 7fabb77b8740 1 -- 172.21.15.60:0/869334245 wait complete. 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 -- start start 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 Processor -- start 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422647630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x5654226480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01aeb10 con 0x565422636850 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01aff80 con 0x565422635f30 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01af1e0 con 0x5654226427e0 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabb77b8740 10 client.0 _ll_register_callbacks cb 0x7faba8014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39454/0 (socket says 172.21.15.60:39454) 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 -- 172.21.15.60:0/4196872655 learned_addr learned my addr 172.21.15.60:0/4196872655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabaeffd640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422647630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabaf7fe640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x5654226480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 -- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 msgr2=0x5654226480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x5654226480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 -- 172.21.15.60:0/4196872655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 msgr2=0x565422647630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422647630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.786+0000 7fabae7fc640 1 -- 172.21.15.60:0/4196872655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56542263a5c0 con 0x565422636850 2024-07-24T21:53:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabae7fc640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faba4005ab0 tx=0x7faba4031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c149b71a980a369 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4035040 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7faba40040c0 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0078040 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faba40042c0 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 10 client.16986 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x565422640660 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0078040 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7faba4004960 con 0x565422636850 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 10 client.16986 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 20 client.16986 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226beeb0 0x5654226c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 4 client.16986 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.787+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654225ab3e0 con 0x5654226beeb0 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.788+0000 7fabaeffd640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226beeb0 0x5654226c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.788+0000 7fab92ff5640 10 client.16986 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.788+0000 7fabaeffd640 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226beeb0 0x5654226c12d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4810f4b2821f639e server_cookie=80cd5ffe537f0915 in_seq=0 out_seq=0 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fab92ff5640 1 -- 172.21.15.60:0/4196872655 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654225ab3e0 con 0x5654226beeb0 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fab92ff5640 10 client.16986 handle_command_reply: tid=0 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 client.16986 shutdown 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 msgr2=0x565422647b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faba4005ab0 tx=0x7faba4031970 comp rx=0 tx=0).stop 2024-07-24T21:53:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 shutdown_connections 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5654226beeb0 0x5654226c12d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654226427e0 0x5654226480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565422636850 0x565422647b70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 --2- 172.21.15.60:0/4196872655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565422635f30 0x565422647630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fab92ff5640 0 client.16986 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 >> 172.21.15.60:0/4196872655 conn(0x565422633d60 msgr2=0x56542264d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 shutdown_connections 2024-07-24T21:53:52.054 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "9d0501ec-3c36-411d-861c-4f83c49329c6", 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 1 -- 172.21.15.60:0/4196872655 wait complete. 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.789+0000 7fabb77b8740 20 client.16986 trim_cache size 0 max 16384 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 msgr2=0x7fab980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 0x7fab980b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fabb01033d0 tx=0x7faba8004ab0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 msgr2=0x7fabb0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faba000b4a0 tx=0x7faba000b960 comp rx=0 tx=0).stop 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 shutdown_connections 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fab980b3d40 0x7fab980b6200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabb010cac0 0x7fabb0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0103f80 0x7fabb0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 --2- 172.21.15.60:0/2933157923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabb0103660 0x7fabb0101310 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 >> 172.21.15.60:0/2933157923 conn(0x7fabb00fd0c0 msgr2=0x7fabb00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 shutdown_connections 2024-07-24T21:53:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:51.790+0000 7fabb5407640 1 -- 172.21.15.60:0/2933157923 wait complete. 2024-07-24T21:53:52.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:52 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:53:52.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:52 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:53:52.743 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:53:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:53:52.332+0000 7f827ed99640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x200000020bc [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-60/ auth{1=1,2=1} fragtree_t(*^3) v214509 ap=3 f(v10 m2024-07-24T21:52:38.944843+0000 158=158+0) n(v16 rc2024-07-24T21:52:38.944843+0000 b2308062 159=158+1) old_inodes=4 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:18245 nref=3)) caps={15300=pAsLsXsFs/pAsLsXsFsxcral@24} | dirtyscattered=0 request=1 lock=2 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55de93085080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x200000020bc:[<0x10000000399/blog-60 v124597>,<0x10000000208/blogtest_in v1201>,<0x10000000201/src v2125>,<0x10000000006/blogbench-1.0 v1847>,<0x10000000005/tmp v82>,<0x100000001fb/client.0 v70>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v86>,<0x10000000001/sv_1 v109>,<0x10000000000/qa v100>,<0x1/volumes v94>]//[]","memoryvalue":"(2)0x200000020bc:[<0x10000000399/blog-60 v214509>,<0x10000000208/blogtest_in v2278>,<0x10000000201/src v2535>,<0x10000000006/blogbench-1.0 v2181>,<0x10000000005/tmp v414>,<0x100000001fb/client.0 v376>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v369>,<0x10000000001/sv_1 v413>,<0x10000000000/qa v392>,<0x1/volumes v146>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:51:58.888205+0000 157=157+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:52:38.944843+0000 b2308062 159=158+1)","memory_value.dirstat":"f(v10 m2024-07-24T21:52:38.944843+0000 158=158+0)","memory_value.rstat":"n(v16 rc2024-07-24T21:52:38.944843+0000 b2308062 159=158+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:53:52.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:52 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:53:52.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:52 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:53:52.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:52 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:53:52.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:52 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:53:53.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:53:53.039 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: pgmap v596: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-07-24T21:53:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:53 smithi123 ceph-mon[37143]: Scrub error on inode 0x200000020bc (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-60) see mds.i log and `damage ls` output for details 2024-07-24T21:53:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:53:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:53:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:53:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: pgmap v596: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-07-24T21:53:53.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:53 smithi086 ceph-mon[37128]: Scrub error on inode 0x200000020bc (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-60) see mds.i log and `damage ls` output for details 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: pgmap v596: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-07-24T21:53:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:53 smithi060 ceph-mon[33000]: Scrub error on inode 0x200000020bc (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-60) see mds.i log and `damage ls` output for details 2024-07-24T21:53:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:55.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:55 smithi060 ceph-mon[33000]: pgmap v597: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 1.06k op/s 2024-07-24T21:53:55.657 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:55 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:55.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:55 smithi123 ceph-mon[37143]: pgmap v597: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 1.06k op/s 2024-07-24T21:53:55.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:55 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:55 smithi086 ceph-mon[37128]: pgmap v597: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.5 MiB/s wr, 1.06k op/s 2024-07-24T21:53:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:55 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:55.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.946+0000 7f9d99e4b640 1 -- 172.21.15.60:0/3353032132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 msgr2=0x7f9d94101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.946+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/3353032132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d94101830 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009900 tx=0x7f9d8402a410 comp rx=0 tx=0).stop 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 -- 172.21.15.60:0/3353032132 shutdown_connections 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/3353032132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94106890 0x7f9d94106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/3353032132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94101d70 0x7f9d94106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/3353032132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d94101830 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 -- 172.21.15.60:0/3353032132 >> 172.21.15.60:0/3353032132 conn(0x7f9d940fd0c0 msgr2=0x7f9d940ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 -- 172.21.15.60:0/3353032132 shutdown_connections 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.947+0000 7f9d99e4b640 1 -- 172.21.15.60:0/3353032132 wait complete. 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 Processor -- start 2024-07-24T21:53:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 -- start start 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94101d70 0x7f9d9419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94106890 0x7f9d941a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94118d50 con 0x7f9d94101450 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94118bd0 con 0x7f9d94106890 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d99e4b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94118ed0 con 0x7f9d94101d70 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d937fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d937fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51858/0 (socket says 172.21.15.60:51858) 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d937fe640 1 -- 172.21.15.60:0/2498260534 learned_addr learned my addr 172.21.15.60:0/2498260534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d93fff640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94106890 0x7f9d941a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.948+0000 7f9d92ffd640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94101d70 0x7f9d9419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 -- 172.21.15.60:0/2498260534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94101d70 msgr2=0x7f9d9419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94101d70 0x7f9d9419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 -- 172.21.15.60:0/2498260534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94106890 msgr2=0x7f9d941a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94106890 0x7f9d941a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 -- 172.21.15.60:0/2498260534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d941a52f0 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d937fe640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005940 tx=0x7f9d840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8403f030 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d941a5580 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d840044c0 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.949+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941abca0 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.950+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84049060 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.951+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d94106240 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.953+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9d84042030 con 0x7f9d94101450 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.954+0000 7f9d90ff9640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 0x7f9d7c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.954+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f9d84104710 con 0x7f9d94101450 2024-07-24T21:53:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.954+0000 7f9d92ffd640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 0x7f9d7c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.955+0000 7f9d92ffd640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 0x7f9d7c0b6280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9d941005a0 tx=0x7f9d88005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:55.959+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9d840cb340 con 0x7f9d94101450 2024-07-24T21:53:56.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.298+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d941abed0 con 0x7f9d94101450 2024-07-24T21:53:56.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.298+0000 7f9d90ff9640 1 -- 172.21.15.60:0/2498260534 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11919 (secure 0 0 0) 0x7f9d840cf890 con 0x7f9d94101450 2024-07-24T21:53:56.300 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:53:56.300 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"btime":"2024-07-24T21:53:35:326256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:53:34.943486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:53:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 38 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 msgr2=0x7f9d7c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 0x7f9d7c0b6280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9d941005a0 tx=0x7f9d88005f90 comp rx=0 tx=0).stop 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 msgr2=0x7f9d9419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005940 tx=0x7f9d840040c0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 shutdown_connections 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9d7c0b3dc0 0x7f9d7c0b6280 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d94106890 0x7f9d941a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d94101d70 0x7f9d9419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 --2- 172.21.15.60:0/2498260534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d94101450 0x7f9d9419f240 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.301+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 >> 172.21.15.60:0/2498260534 conn(0x7f9d940fd0c0 msgr2=0x7f9d941041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.302+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 shutdown_connections 2024-07-24T21:53:56.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.302+0000 7f9d99e4b640 1 -- 172.21.15.60:0/2498260534 wait complete. 2024-07-24T21:53:56.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2498260534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.477+0000 7ffa5301f640 1 -- 172.21.15.60:0/2447119029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c1079e0 msgr2=0x7ffa4c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.477+0000 7ffa5301f640 1 --2- 172.21.15.60:0/2447119029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c10e1e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ca0 tx=0x7ffa4402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 -- 172.21.15.60:0/2447119029 shutdown_connections 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 --2- 172.21.15.60:0/2447119029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c10e1e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 --2- 172.21.15.60:0/2447119029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c107060 0x7ffa4c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 --2- 172.21.15.60:0/2447119029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 -- 172.21.15.60:0/2447119029 >> 172.21.15.60:0/2447119029 conn(0x7ffa4c0fd0c0 msgr2=0x7ffa4c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 -- 172.21.15.60:0/2447119029 shutdown_connections 2024-07-24T21:53:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.478+0000 7ffa5301f640 1 -- 172.21.15.60:0/2447119029 wait complete. 2024-07-24T21:53:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 Processor -- start 2024-07-24T21:53:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 -- start start 2024-07-24T21:53:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c0fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c111660 con 0x7ffa4c107060 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1114e0 con 0x7ffa4c1079e0 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.479+0000 7ffa5301f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1117e0 con 0x7ffa4c101650 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c0fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51878/0 (socket says 172.21.15.60:51878) 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:36736/0 (socket says 172.21.15.60:36736) 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa4bfff640 1 -- 172.21.15.60:0/1800966569 learned_addr learned my addr 172.21.15.60:0/1800966569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 -- 172.21.15.60:0/1800966569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 msgr2=0x7ffa4c0fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 -- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 msgr2=0x7ffa4c0fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 -- 172.21.15.60:0/1800966569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c1a7130 con 0x7ffa4c1079e0 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.480+0000 7ffa515bc640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c0fe040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005e30 tx=0x7ffa440317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.481+0000 7ffa49ffb640 1 -- 172.21.15.60:0/1800966569 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44003b70 con 0x7ffa4c1079e0 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.481+0000 7ffa49ffb640 1 -- 172.21.15.60:0/1800966569 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44003d10 con 0x7ffa4c1079e0 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.481+0000 7ffa49ffb640 1 -- 172.21.15.60:0/1800966569 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4402f970 con 0x7ffa4c1079e0 2024-07-24T21:53:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.481+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c1a73c0 con 0x7ffa4c1079e0 2024-07-24T21:53:56.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.481+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1a78b0 con 0x7ffa4c1079e0 2024-07-24T21:53:56.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.484+0000 7ffa553d0740 1 Processor -- start 2024-07-24T21:53:56.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa49ffb640 1 -- 172.21.15.60:0/1800966569 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ffa4402fb10 con 0x7ffa4c1079e0 2024-07-24T21:53:56.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 -- start start 2024-07-24T21:53:56.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abcf66b0 0x5594abcf6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5594abc6d990 con 0x5594abd02f60 2024-07-24T21:53:56.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594abc5ce90 con 0x5594abcf66b0 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa553d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5594abc60ed0 con 0x5594abcf6fd0 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abcf66b0 0x5594abcf6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51886/0 (socket says 172.21.15.60:51886) 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa4bfff640 1 -- 172.21.15.60:0/2693287550 learned_addr learned my addr 172.21.15.60:0/2693287550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.485+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 -- 172.21.15.60:0/2693287550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 msgr2=0x5594abd02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 -- 172.21.15.60:0/2693287550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abcf66b0 msgr2=0x5594abcf6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abcf66b0 0x5594abcf6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 -- 172.21.15.60:0/2693287550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594abbe6300 con 0x5594abd02f60 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffa38004600 tx=0x7ffa38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688df370dc9ca382 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa49ffb640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 0x7ffa3c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa36ffd640 1 -- 172.21.15.60:0/2693287550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38016030 con 0x5594abd02f60 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa36ffd640 1 -- 172.21.15.60:0/2693287550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3800fe50 con 0x5594abd02f60 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa36ffd640 1 -- 172.21.15.60:0/2693287550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38019040 con 0x5594abd02f60 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa49ffb640 1 -- 172.21.15.60:0/1800966569 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7ffa44105110 con 0x7ffa4c1079e0 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.486+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 0x7ffa3c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 -- 172.21.15.60:0/2693287550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 msgr2=0x5594abd05350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 0x7ffa3c0b6200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5594abcf53c0 tx=0x7ffa40005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ffa38004600 tx=0x7ffa38005f90 comp rx=0 tx=0).stop 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 -- 172.21.15.60:0/2693287550 shutdown_connections 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abd02f60 0x5594abd05350 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 --2- 172.21.15.60:0/2693287550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abcf66b0 0x5594abcf6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 -- 172.21.15.60:0/2693287550 >> 172.21.15.60:0/2693287550 conn(0x5594abcf44e0 msgr2=0x5594abcf5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 -- 172.21.15.60:0/2693287550 shutdown_connections 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.487+0000 7ffa553d0740 1 -- 172.21.15.60:0/2693287550 wait complete. 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.488+0000 7ffa553d0740 1 -- start start 2024-07-24T21:53:56.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.488+0000 7ffa553d0740 1 Processor -- start 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.488+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.488+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf6fd0 0x5594abd004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.488+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa553d0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5594abac0700 con 0x5594abcf6fd0 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa553d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594abbff5d0 con 0x5594abd02f60 2024-07-24T21:53:56.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa553d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5594abd05b80 con 0x5594abcf66b0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:36764/0 (socket says 172.21.15.60:36764) 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 -- 172.21.15.60:0/1390585499 learned_addr learned my addr 172.21.15.60:0/1390585499 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf6fd0 0x5594abd004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa515bc640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa553d0740 10 client.0 _ll_register_callbacks cb 0x5594abc1f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 -- 172.21.15.60:0/1390585499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 msgr2=0x5594abd00a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd00a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 -- 172.21.15.60:0/1390585499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf6fd0 msgr2=0x5594abd004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf6fd0 0x5594abd004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 -- 172.21.15.60:0/1390585499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594abbf2660 con 0x5594abcf66b0 2024-07-24T21:53:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa515bc640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd00a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa4001f9b0 tx=0x7ffa40004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c755c3a2848b39 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.489+0000 7ffa2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4001d620 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7ffa4001d7c0 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3800fe50 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffa4001da00 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 10 client.26476 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5594ab9bd650 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3800fe50 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ffa400293f0 con 0x5594abcf66b0 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 10 client.26476 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:56.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 20 client.26476 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.490+0000 7ffa553d0740 4 client.26476 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.491+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594abc6f8e0 con 0x5594abd12b90 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.491+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.491+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd14f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efccc66e9abfaf8d server_cookie=7a08e86dd584fda in_seq=0 out_seq=0 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.491+0000 7ffa2bfff640 10 client.26476 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa2bfff640 1 -- 172.21.15.60:0/1390585499 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594abc6f8e0 con 0x5594abd12b90 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa2bfff640 10 client.26476 handle_command_reply: tid=0 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 client.26476 shutdown 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 msgr2=0x5594abcfff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa4001f9b0 tx=0x7ffa40004730 comp rx=0 tx=0).stop 2024-07-24T21:53:56.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 shutdown_connections 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd14f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa2bfff640 0 client.26476 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.492+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd00a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf6fd0 0x5594abd004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 1 --2- 172.21.15.60:0/1390585499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf66b0 0x5594abcfff80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 >> 172.21.15.60:0/1390585499 conn(0x5594abcf44e0 msgr2=0x5594abcfa080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 shutdown_connections 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 1 -- 172.21.15.60:0/1390585499 wait complete. 2024-07-24T21:53:56.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.493+0000 7ffa553d0740 20 client.26476 trim_cache size 0 max 16384 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.516+0000 7ffa553d0740 1 Processor -- start 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.516+0000 7ffa553d0740 1 -- start start 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.516+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abcfa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.516+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd1a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa553d0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1ae970 con 0x5594abcf66b0 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa553d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1ac1e0 con 0x5594abd02f60 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abcfa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa553d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1aa050 con 0x5594abcf6fd0 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd1a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59118/0 (socket says 172.21.15.60:59118) 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd1a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:36766/0 (socket says 172.21.15.60:36766) 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 -- 172.21.15.60:0/453523114 learned_addr learned my addr 172.21.15.60:0/453523114 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 -- 172.21.15.60:0/453523114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 msgr2=0x5594abd1a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd1a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 -- 172.21.15.60:0/453523114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 msgr2=0x5594abcfa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abcfa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 -- 172.21.15.60:0/453523114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594abd152f0 con 0x5594abd02f60 2024-07-24T21:53:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.517+0000 7ffa515bc640 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa44031a60 tx=0x7ffa440095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b8b25c56ac5e50 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.518+0000 7ffa367fc640 1 -- 172.21.15.60:0/453523114 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440e6070 con 0x5594abd02f60 2024-07-24T21:53:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.518+0000 7ffa367fc640 1 -- 172.21.15.60:0/453523114 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44036040 con 0x5594abd02f60 2024-07-24T21:53:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.518+0000 7ffa367fc640 1 -- 172.21.15.60:0/453523114 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44102b50 con 0x5594abd02f60 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.518+0000 7ffa553d0740 1 -- 172.21.15.60:0/453523114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 msgr2=0x5594abd1ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.518+0000 7ffa553d0740 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa44031a60 tx=0x7ffa440095c0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 -- 172.21.15.60:0/453523114 shutdown_connections 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd1ad80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd1a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 --2- 172.21.15.60:0/453523114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abcfa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 -- 172.21.15.60:0/453523114 >> 172.21.15.60:0/453523114 conn(0x5594abcf44e0 msgr2=0x5594abcfe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 -- 172.21.15.60:0/453523114 shutdown_connections 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 -- 172.21.15.60:0/453523114 wait complete. 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 -- start start 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.519+0000 7ffa553d0740 1 Processor -- start 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd7c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1aa6a0 con 0x5594abcf66b0 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1abb10 con 0x5594abd02f60 2024-07-24T21:53:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1aad70 con 0x5594abcf6fd0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa553d0740 10 client.0 _ll_register_callbacks cb 0x7ffa40009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa50dbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51922/0 (socket says 172.21.15.60:51922) 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa50dbb640 1 -- 172.21.15.60:0/3028056790 learned_addr learned my addr 172.21.15.60:0/3028056790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.520+0000 7ffa515bc640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd7c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 -- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 msgr2=0x5594abd07960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd07960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 -- 172.21.15.60:0/3028056790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 msgr2=0x5594abd7c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd7c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 -- 172.21.15.60:0/3028056790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594abd7cdf0 con 0x5594abcf66b0 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa515bc640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd7c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa50dbb640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffa40029800 tx=0x7ffa40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da389097065a80fa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4001dda0 con 0x5594abcf66b0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7ffa4003c480 con 0x5594abcf66b0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa44036040 con 0x5594abcf66b0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffa4004a050 con 0x5594abcf66b0 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa553d0740 10 client.17010 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:56.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.521+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5594abd7e100 con 0x5594abcf66b0 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44036040 con 0x5594abcf66b0 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4003ccc0 con 0x5594abcf66b0 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 10 client.17010 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 20 client.17010 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 4 client.17010 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594abc6b380 con 0x5594abd12b90 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.522+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa4bfff640 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd808e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15596dea6f191cd6 server_cookie=9f1fe127cf70fc54 in_seq=0 out_seq=0 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa2bfff640 10 client.17010 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa2bfff640 1 -- 172.21.15.60:0/3028056790 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594abc6b380 con 0x5594abd12b90 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa2bfff640 10 client.17010 handle_command_reply: tid=0 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 client.17010 shutdown 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 msgr2=0x5594abd07420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffa40029800 tx=0x7ffa40017000 comp rx=0 tx=0).stop 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 shutdown_connections 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5594abd12b90 0x5594abd808e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594abd02f60 0x5594abd7c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5594abcf6fd0 0x5594abd07960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 --2- 172.21.15.60:0/3028056790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594abcf66b0 0x5594abd07420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 >> 172.21.15.60:0/3028056790 conn(0x5594abcf44e0 msgr2=0x5594abcfebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa2bfff640 0 client.17010 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 shutdown_connections 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.523+0000 7ffa553d0740 1 -- 172.21.15.60:0/3028056790 wait complete. 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.524+0000 7ffa553d0740 20 client.17010 trim_cache size 0 max 16384 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:53:56.530 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: "9d0501ec-3c36-411d-861c-4f83c49329c6": { 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "9d0501ec-3c36-411d-861c-4f83c49329c6", 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:53:56.531 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.524+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 msgr2=0x7ffa3c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.524+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 0x7ffa3c0b6200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5594abcf53c0 tx=0x7ffa40005f90 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.524+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 msgr2=0x7ffa4c0fe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.524+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c0fe040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005e30 tx=0x7ffa440317c0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 shutdown_connections 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ffa3c0b3d40 0x7ffa3c0b6200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa4c1079e0 0x7ffa4c0fe040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c107060 0x7ffa4c0fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 --2- 172.21.15.60:0/1800966569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa4c101650 0x7ffa4c0fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 >> 172.21.15.60:0/1800966569 conn(0x7ffa4c0fd0c0 msgr2=0x7ffa4c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 shutdown_connections 2024-07-24T21:53:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:56.525+0000 7ffa5301f640 1 -- 172.21.15.60:0/1800966569 wait complete. 2024-07-24T21:53:56.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:56 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2498260534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:56.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:56 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2498260534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:53:57.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id b8eda14d --release --await' 2024-07-24T21:53:57.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id b8eda14d --release --await 2024-07-24T21:53:57.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:57 smithi060 ceph-mon[33000]: pgmap v598: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 2.13k op/s 2024-07-24T21:53:57.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:57 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:57 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:57 smithi123 ceph-mon[37143]: pgmap v598: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 2.13k op/s 2024-07-24T21:53:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:57 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:57 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.762 INFO:tasks.cephfs.filesystem:scrub status for tag:9d0501ec-3c36-411d-861c-4f83c49329c6 - {'path': '/', 'tag': '9d0501ec-3c36-411d-861c-4f83c49329c6', 'options': 'recursive,force'} 2024-07-24T21:53:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:57 smithi086 ceph-mon[37128]: pgmap v598: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 2.13k op/s 2024-07-24T21:53:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:57 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:57 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:53:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:53:59.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:53:59 smithi060 ceph-mon[33000]: pgmap v599: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 3.06k op/s 2024-07-24T21:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.596+0000 7fc7e6641640 1 -- 172.21.15.60:0/3859929767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e0103490 msgr2=0x7fc7e0103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.596+0000 7fc7e6641640 1 --2- 172.21.15.60:0/3859929767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e0103490 0x7fc7e0103870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009900 tx=0x7fc7d002a410 comp rx=0 tx=0).stop 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 -- 172.21.15.60:0/3859929767 shutdown_connections 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 --2- 172.21.15.60:0/3859929767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e010ca40 0x7fc7e010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 --2- 172.21.15.60:0/3859929767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103e90 0x7fc7e010c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 --2- 172.21.15.60:0/3859929767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e0103490 0x7fc7e0103870 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 -- 172.21.15.60:0/3859929767 >> 172.21.15.60:0/3859929767 conn(0x7fc7e00782e0 msgr2=0x7fc7e00ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 -- 172.21.15.60:0/3859929767 shutdown_connections 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 -- 172.21.15.60:0/3859929767 wait complete. 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 Processor -- start 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.597+0000 7fc7e6641640 1 -- start start 2024-07-24T21:53:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103490 0x7fc7e0101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e0103e90 0x7fc7e0101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0112650 con 0x7fc7e010ca40 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01124d0 con 0x7fc7e0103490 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e6641640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01127d0 con 0x7fc7e0103e90 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51936/0 (socket says 172.21.15.60:51936) 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 -- 172.21.15.60:0/4198324687 learned_addr learned my addr 172.21.15.60:0/4198324687 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e0103e90 0x7fc7e0101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7dffff640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103490 0x7fc7e0101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 -- 172.21.15.60:0/4198324687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e0103e90 msgr2=0x7fc7e0101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e0103e90 0x7fc7e0101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 -- 172.21.15.60:0/4198324687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103490 msgr2=0x7fc7e0101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103490 0x7fc7e0101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.598+0000 7fc7e4bde640 1 -- 172.21.15.60:0/4198324687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01a7080 con 0x7fc7e010ca40 2024-07-24T21:53:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e9d0 tx=0x7fc7d800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7ddffb640 1 -- 172.21.15.60:0/4198324687 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8004d20 con 0x7fc7e010ca40 2024-07-24T21:53:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01a7370 con 0x7fc7e010ca40 2024-07-24T21:53:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7ddffb640 1 -- 172.21.15.60:0/4198324687 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d8005020 con 0x7fc7e010ca40 2024-07-24T21:53:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7ddffb640 1 -- 172.21.15.60:0/4198324687 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80116b0 con 0x7fc7e010ca40 2024-07-24T21:53:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.599+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a78c0 con 0x7fc7e010ca40 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 Processor -- start 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7ddffb640 1 -- 172.21.15.60:0/4198324687 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc7d80051d0 con 0x7fc7e010ca40 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 -- start start 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8608c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55acf8572280 con 0x55acf85fc8c0 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55acf8561780 con 0x55acf8609170 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.602+0000 7fc7e9397740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55acf85657c0 con 0x55acf85fd1e0 2024-07-24T21:53:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7dffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8608c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51944/0 (socket says 172.21.15.60:51944) 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 -- 172.21.15.60:0/140392967 learned_addr learned my addr 172.21.15.60:0/140392967 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 -- 172.21.15.60:0/140392967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 msgr2=0x55acf8608c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8608c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 -- 172.21.15.60:0/140392967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 msgr2=0x55acf860b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 -- 172.21.15.60:0/140392967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf82c5790 con 0x55acf85fc8c0 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7dffff640 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8608c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7ddffb640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 0x7fc7cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8013610 tx=0x7fc7c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f861891bb9c35 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7dffff640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 0x7fc7cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.603+0000 7fc7ddffb640 1 -- 172.21.15.60:0/4198324687 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fc7d80e40c0 con 0x7fc7e010ca40 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7d67fc640 1 -- 172.21.15.60:0/140392967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800d710 con 0x55acf85fc8c0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7d67fc640 1 -- 172.21.15.60:0/140392967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c800dd60 con 0x55acf85fc8c0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7d67fc640 1 -- 172.21.15.60:0/140392967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800fcf0 con 0x55acf85fc8c0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7e9397740 1 -- 172.21.15.60:0/140392967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 msgr2=0x55acf85fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7e9397740 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8013610 tx=0x7fc7c800a710 comp rx=0 tx=0).stop 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.604+0000 7fc7dffff640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 0x7fc7cc0b6280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0031440 tx=0x7fc7d0005e60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 -- 172.21.15.60:0/140392967 shutdown_connections 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8608c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 --2- 172.21.15.60:0/140392967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf85fcca0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 -- 172.21.15.60:0/140392967 >> 172.21.15.60:0/140392967 conn(0x55acf85fa6f0 msgr2=0x55acf85fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 -- 172.21.15.60:0/140392967 shutdown_connections 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 -- 172.21.15.60:0/140392967 wait complete. 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 -- start start 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.605+0000 7fc7e9397740 1 Processor -- start 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8605e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf8606340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55acf849bab0 con 0x55acf85fc8c0 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55acf84e99e0 con 0x55acf8609170 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55acf860be50 con 0x55acf85fd1e0 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e9397740 10 client.0 _ll_register_callbacks cb 0x55acf8271a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51950/0 (socket says 172.21.15.60:51950) 2024-07-24T21:53:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7dffff640 1 -- 172.21.15.60:0/1090703608 learned_addr learned my addr 172.21.15.60:0/1090703608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8605e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf8606340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1090703608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 msgr2=0x55acf8606340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf8606340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1090703608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 msgr2=0x55acf8608920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1090703608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf833e620 con 0x55acf85fd1e0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7dffff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8605e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f910 tx=0x7fc7c800a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2063ad9fdea97320 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.606+0000 7fc7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8005030 con 0x55acf85fd1e0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55acf8510ce0 con 0x55acf85fd1e0 2024-07-24T21:53:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fc7c80051d0 con 0x55acf85fd1e0 2024-07-24T21:53:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc7c80053d0 con 0x55acf85fd1e0 2024-07-24T21:53:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 10 client.26494 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55acf8617640 con 0x55acf85fd1e0 2024-07-24T21:53:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55acf8510ce0 con 0x55acf85fd1e0 2024-07-24T21:53:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fc7c80046c0 con 0x55acf85fd1e0 2024-07-24T21:53:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 10 client.26494 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.607+0000 7fc7e9397740 20 client.26494 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8618a40 0x55acf861ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7e9397740 4 client.26494 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acf85741d0 con 0x55acf8618a40 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7dffff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8618a40 0x55acf861ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7dffff640 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8618a40 0x55acf861ae30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fd62f7a2e64ea2 server_cookie=4ba3d733347ca1ed in_seq=0 out_seq=0 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.608+0000 7fc7bf7fe640 10 client.26494 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/1090703608 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acf85741d0 con 0x55acf8618a40 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7bf7fe640 10 client.26494 handle_command_reply: tid=0 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 client.26494 shutdown 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 msgr2=0x55acf8605e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8605e00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f910 tx=0x7fc7c800a780 comp rx=0 tx=0).stop 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 shutdown_connections 2024-07-24T21:53:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8618a40 0x55acf861ae30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf8606340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fd1e0 0x55acf8605e00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7bf7fe640 0 client.26494 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1090703608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fc8c0 0x55acf8608920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 >> 172.21.15.60:0/1090703608 conn(0x55acf85fa6f0 msgr2=0x55acf8600220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 shutdown_connections 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 1 -- 172.21.15.60:0/1090703608 wait complete. 2024-07-24T21:53:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.609+0000 7fc7e9397740 20 client.26494 trim_cache size 0 max 16384 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 Processor -- start 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 -- start start 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf85fc8c0 0x55acf8614f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 0x55acf8620520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01b2bb0 con 0x55acf85fd1e0 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01b0420 con 0x55acf85fc8c0 2024-07-24T21:53:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.626+0000 7fc7e9397740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01ae290 con 0x55acf8609170 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51964/0 (socket says 172.21.15.60:51964) 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1354878748 learned_addr learned my addr 172.21.15.60:0/1354878748 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7dffff640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf85fc8c0 0x55acf8614f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 0x55acf8620520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1354878748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 msgr2=0x55acf8620520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 0x55acf8620520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1354878748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf85fc8c0 msgr2=0x55acf8614f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf85fc8c0 0x55acf8614f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 -- 172.21.15.60:0/1354878748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf86042e0 con 0x55acf85fd1e0 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 0x55acf8620520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8004ae0 tx=0x7fc7c800a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9694dc62b3d0c39f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d5ffb640 1 -- 172.21.15.60:0/1354878748 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8033070 con 0x55acf85fd1e0 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d5ffb640 1 -- 172.21.15.60:0/1354878748 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c8005180 con 0x55acf85fd1e0 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.627+0000 7fc7d5ffb640 1 -- 172.21.15.60:0/1354878748 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800d710 con 0x55acf85fd1e0 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- 172.21.15.60:0/1354878748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 msgr2=0x55acf8601280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8004ae0 tx=0x7fc7c800a9c0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- 172.21.15.60:0/1354878748 shutdown_connections 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf8609170 0x55acf8620520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf8601280 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 --2- 172.21.15.60:0/1354878748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf85fc8c0 0x55acf8614f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- 172.21.15.60:0/1354878748 >> 172.21.15.60:0/1354878748 conn(0x55acf85fa6f0 msgr2=0x55acf8600b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- 172.21.15.60:0/1354878748 shutdown_connections 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- 172.21.15.60:0/1354878748 wait complete. 2024-07-24T21:53:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.628+0000 7fc7e9397740 1 -- start start 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 Processor -- start 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fc8c0 0x55acf8611900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01ae8e0 con 0x55acf85fd1e0 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01afd50 con 0x55acf8609170 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01aefb0 con 0x55acf85fc8c0 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e9397740 10 client.0 _ll_register_callbacks cb 0x7fc7d0005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51976/0 (socket says 172.21.15.60:51976) 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 -- 172.21.15.60:0/2351224108 learned_addr learned my addr 172.21.15.60:0/2351224108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7e4bde640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7dffff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fc8c0 0x55acf8611900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 -- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fc8c0 msgr2=0x55acf8611900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fc8c0 0x55acf8611900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 -- 172.21.15.60:0/2351224108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 msgr2=0x55acf860c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.629+0000 7fc7d7fff640 1 -- 172.21.15.60:0/2351224108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf86814e0 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7d7fff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f910 tx=0x7fc7c8013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c3e570e933bd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c803c020 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c8005180 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fc7c8004e60 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8047050 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 10 client.17028 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55acf8682690 con 0x55acf85fd1e0 2024-07-24T21:53:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c8005180 con 0x55acf85fd1e0 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fc7c8027640 con 0x55acf85fd1e0 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 10 client.17028 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 20 client.17028 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8682d70 0x55acf8685190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 4 client.17028 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.630+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acf856fc70 con 0x55acf8682d70 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.631+0000 7fc7dffff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8682d70 0x55acf8685190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.631+0000 7fc7dffff640 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8682d70 0x55acf8685190 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a94295aa0160da server_cookie=628373b43e363b53 in_seq=0 out_seq=0 2024-07-24T21:53:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.631+0000 7fc7bf7fe640 10 client.17028 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:53:59 smithi123 ceph-mon[37143]: pgmap v599: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 3.06k op/s 2024-07-24T21:53:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:53:59 smithi086 ceph-mon[37128]: pgmap v599: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 3.06k op/s 2024-07-24T21:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.795+0000 7fc7bf7fe640 1 -- 172.21.15.60:0/2351224108 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acf856fc70 con 0x55acf8682d70 2024-07-24T21:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7bf7fe640 10 client.17028 handle_command_reply: tid=0 2024-07-24T21:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 client.17028 shutdown 2024-07-24T21:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 msgr2=0x55acf860c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f910 tx=0x7fc7c8013f60 comp rx=0 tx=0).stop 2024-07-24T21:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 shutdown_connections 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55acf8682d70 0x55acf8685190 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7bf7fe640 0 client.17028 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55acf8609170 0x55acf860c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55acf85fd1e0 0x55acf860c0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 --2- 172.21.15.60:0/2351224108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acf85fc8c0 0x55acf8611900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 >> 172.21.15.60:0/2351224108 conn(0x55acf85fa6f0 msgr2=0x55acf8607a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 shutdown_connections 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 1 -- 172.21.15.60:0/2351224108 wait complete. 2024-07-24T21:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.796+0000 7fc7e9397740 20 client.17028 trim_cache size 0 max 16384 2024-07-24T21:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 msgr2=0x7fc7cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 0x7fc7cc0b6280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0031440 tx=0x7fc7d0005e60 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 msgr2=0x7fc7e0101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e9d0 tx=0x7fc7d800edd0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 shutdown_connections 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7cc0b3dc0 0x7fc7cc0b6280 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e010ca40 0x7fc7e0101d90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7e0103e90 0x7fc7e0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 --2- 172.21.15.60:0/4198324687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e0103490 0x7fc7e0101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 >> 172.21.15.60:0/4198324687 conn(0x7fc7e00782e0 msgr2=0x7fc7e00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 shutdown_connections 2024-07-24T21:53:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:53:59.797+0000 7fc7e6641640 1 -- 172.21.15.60:0/4198324687 wait complete. 2024-07-24T21:53:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 -- 172.21.15.60:0/1552209996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541036a0 msgr2=0x7f3b54103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 --2- 172.21.15.60:0/1552209996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541036a0 0x7f3b54103a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009900 tx=0x7f3b4402a410 comp rx=0 tx=0).stop 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 -- 172.21.15.60:0/1552209996 shutdown_connections 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 --2- 172.21.15.60:0/1552209996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b5410cb00 0x7f3b5410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 --2- 172.21.15.60:0/1552209996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b5410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 --2- 172.21.15.60:0/1552209996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541036a0 0x7f3b54103a80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.845+0000 7f3b59d07640 1 -- 172.21.15.60:0/1552209996 >> 172.21.15.60:0/1552209996 conn(0x7f3b540fd100 msgr2=0x7f3b540ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.846+0000 7f3b59d07640 1 -- 172.21.15.60:0/1552209996 shutdown_connections 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.846+0000 7f3b59d07640 1 -- 172.21.15.60:0/1552209996 wait complete. 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.846+0000 7f3b59d07640 1 Processor -- start 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.846+0000 7f3b59d07640 1 -- start start 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b54199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 0x7f3b5419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54112740 con 0x7f3b5410cb00 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541125c0 con 0x7f3b54103fc0 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b59d07640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541128c0 con 0x7f3b541036a0 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 0x7f3b5419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:36854/0 (socket says 172.21.15.60:36854) 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 -- 172.21.15.60:0/3882331333 learned_addr learned my addr 172.21.15.60:0/3882331333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b54199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 -- 172.21.15.60:0/3882331333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 msgr2=0x7f3b54199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b54199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 -- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 msgr2=0x7f3b5419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 0x7f3b5419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 -- 172.21.15.60:0/3882331333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5419ed80 con 0x7f3b541036a0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b54199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 0x7f3b5419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b537fe640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3b44005940 tx=0x7f3b440040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b50ff9640 1 -- 172.21.15.60:0/3882331333 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4403f030 con 0x7f3b541036a0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.847+0000 7f3b50ff9640 1 -- 172.21.15.60:0/3882331333 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b440044c0 con 0x7f3b541036a0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.848+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5419f010 con 0x7f3b541036a0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.848+0000 7f3b50ff9640 1 -- 172.21.15.60:0/3882331333 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44035070 con 0x7f3b541036a0 2024-07-24T21:53:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.848+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541a5730 con 0x7f3b541036a0 2024-07-24T21:53:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 Processor -- start 2024-07-24T21:53:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 -- start start 2024-07-24T21:53:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b50ff9640 1 -- 172.21.15.60:0/3882331333 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3b44042030 con 0x7f3b541036a0 2024-07-24T21:53:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ebeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ec16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b085e29dd0 con 0x55b085eb3350 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b085e192d0 con 0x55b085ebf2e0 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b085e1d310 con 0x55b085eb2a30 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ebeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.850+0000 7f3b537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ebeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52004/0 (socket says 172.21.15.60:52004) 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b537fe640 1 -- 172.21.15.60:0/3482560456 learned_addr learned my addr 172.21.15.60:0/3482560456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ec16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 -- 172.21.15.60:0/3482560456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 msgr2=0x55b085ec16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ec16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 -- 172.21.15.60:0/3482560456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 msgr2=0x55b085ebeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ebeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 -- 172.21.15.60:0/3482560456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b085b79b00 con 0x55b085eb2a30 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b50ff9640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 0x7f3b3c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb2e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c013af0 tx=0x7f3b4c00e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3acce445580a4c34 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b537fe640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 0x7f3b3c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b39ffb640 1 -- 172.21.15.60:0/3482560456 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00ebc0 con 0x55b085eb2a30 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b39ffb640 1 -- 172.21.15.60:0/3482560456 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b4c00ed60 con 0x55b085eb2a30 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b39ffb640 1 -- 172.21.15.60:0/3482560456 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c01c410 con 0x55b085eb2a30 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.851+0000 7f3b50ff9640 1 -- 172.21.15.60:0/3882331333 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b44104410 con 0x7f3b541036a0 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3482560456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 msgr2=0x55b085eb2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b537fe640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 0x7f3b3c0b6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b440cacf0 tx=0x7f3b44109040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb2e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c013af0 tx=0x7f3b4c00e3b0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3482560456 shutdown_connections 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ec16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ebeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3482560456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb2e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3482560456 >> 172.21.15.60:0/3482560456 conn(0x55b085eb0860 msgr2=0x55b085eb1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3482560456 shutdown_connections 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3482560456 wait complete. 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.852+0000 7f3b5c0b8740 1 -- start start 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 Processor -- start 2024-07-24T21:53:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085ebc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085eb3350 0x55b085ebc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b085c7cbb0 con 0x55b085ebf2e0 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b085dbb900 con 0x55b085eb3350 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b085ec1f60 con 0x55b085eb2a30 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b5c0b8740 10 client.0 _ll_register_callbacks cb 0x55b085dc5c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52016/0 (socket says 172.21.15.60:52016) 2024-07-24T21:53:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b53fff640 1 -- 172.21.15.60:0/2904037779 learned_addr learned my addr 172.21.15.60:0/2904037779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b53fff640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085ebc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.853+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085eb3350 0x55b085ebc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 -- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 msgr2=0x55b085ebc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085ebc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 -- 172.21.15.60:0/2904037779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085eb3350 msgr2=0x55b085ebc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085eb3350 0x55b085ebc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 -- 172.21.15.60:0/2904037779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b085bf0960 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b53fff640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c013fd0 tx=0x7f3b4c005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476590001c0cefd5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003b80 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b4c00ed60 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b4c003d20 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c004ad0 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b5c0b8740 10 client.17040 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b085ecdaf0 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b4c00ed60 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.854+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3b4c028040 con 0x55b085ebf2e0 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 10 client.17040 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 10 client.17040 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:53:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 20 client.17040 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085ecee40 0x55b085ed1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 4 client.17040 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b085e2bd20 con 0x55b085ecee40 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085ecee40 0x55b085ed1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b32ffd640 10 client.17040 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.855+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085ecee40 0x55b085ed1230 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ab7e1ba31c8c1e server_cookie=d2e2d12e069e6a1a in_seq=0 out_seq=0 2024-07-24T21:53:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b32ffd640 1 -- 172.21.15.60:0/2904037779 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b085e2bd20 con 0x55b085ecee40 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b32ffd640 10 client.17040 handle_command_reply: tid=0 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 client.17040 shutdown 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 msgr2=0x55b085ebccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c013fd0 tx=0x7f3b4c005650 comp rx=0 tx=0).stop 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 shutdown_connections 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085ecee40 0x55b085ed1230 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085ebf2e0 0x55b085ebccd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085eb3350 0x55b085ebc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/2904037779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085ebc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 >> 172.21.15.60:0/2904037779 conn(0x55b085eb0860 msgr2=0x55b085eb6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b32ffd640 0 client.17040 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 shutdown_connections 2024-07-24T21:53:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/2904037779 wait complete. 2024-07-24T21:53:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.856+0000 7f3b5c0b8740 20 client.17040 trim_cache size 0 max 16384 2024-07-24T21:53:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.874+0000 7f3b5c0b8740 1 Processor -- start 2024-07-24T21:53:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.874+0000 7f3b5c0b8740 1 -- start start 2024-07-24T21:53:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.874+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ed6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541ae900 con 0x55b085eb3350 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541ac170 con 0x55b085ebf2e0 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541a9fe0 con 0x55b085eb2a30 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52024/0 (socket says 172.21.15.60:52024) 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 -- 172.21.15.60:0/336366922 learned_addr learned my addr 172.21.15.60:0/336366922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b53fff640 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ed6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 -- 172.21.15.60:0/336366922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 msgr2=0x55b085eb6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 -- 172.21.15.60:0/336366922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 msgr2=0x55b085ed6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ed6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 -- 172.21.15.60:0/336366922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b085eac190 con 0x55b085eb3350 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b53fff640 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.875+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3b5406bcf0 tx=0x7f3b48009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4837aaff5cc4d3a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.876+0000 7f3b397fa640 1 -- 172.21.15.60:0/336366922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48003e70 con 0x55b085eb3350 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.876+0000 7f3b397fa640 1 -- 172.21.15.60:0/336366922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b48012900 con 0x55b085eb3350 2024-07-24T21:53:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.876+0000 7f3b397fa640 1 -- 172.21.15.60:0/336366922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4801d990 con 0x55b085eb3350 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.876+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/336366922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 msgr2=0x55b085ed69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.876+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3b5406bcf0 tx=0x7f3b48009e80 comp rx=0 tx=0).stop 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/336366922 shutdown_connections 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085ed6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb3350 0x55b085ed69c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/336366922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb2a30 0x55b085eb6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/336366922 >> 172.21.15.60:0/336366922 conn(0x55b085eb0860 msgr2=0x55b085eb5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/336366922 shutdown_connections 2024-07-24T21:53:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/336366922 wait complete. 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 -- start start 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.877+0000 7f3b5c0b8740 1 Processor -- start 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb3350 0x55b085ec7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085eb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541aa630 con 0x55b085eb2a30 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541abaa0 con 0x55b085ebf2e0 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b541aad00 con 0x55b085eb3350 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b5c0b8740 10 client.0 _ll_register_callbacks cb 0x7f3b4c0139f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52036/0 (socket says 172.21.15.60:52036) 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 -- 172.21.15.60:0/3689283037 learned_addr learned my addr 172.21.15.60:0/3689283037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb3350 0x55b085ec7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085eb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 -- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb3350 msgr2=0x55b085ec7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb3350 0x55b085ec7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 -- 172.21.15.60:0/3689283037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 msgr2=0x55b085eb8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085eb8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 -- 172.21.15.60:0/3689283037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b085eb97a0 con 0x55b085eb2a30 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b53fff640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e8b0 tx=0x7f3b4c005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c86f2f26d56092a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.878+0000 7f3b32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:53:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c004e80 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b4c005020 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b48012900 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c003b80 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 10 client.17046 fetch_fsmap learned FSMap version 38 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b085f39740 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b48012900 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3b4c03c070 con 0x55b085eb2a30 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 10 client.17046 fetch_fsmap finished waiting for FSMap version 38 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 10 client.17046 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 20 client.17046 populate_metadata read hostname 'smithi060' 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085f39e60 0x55b085f3c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 4 client.17046 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b8eda14d", "release": true, "await": true}] 2024-07-24T21:53:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.879+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b8eda14d", "release": true, "await": true}) -- 0x55b085e277c0 con 0x55b085f39e60 2024-07-24T21:53:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.880+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085f39e60 0x55b085f3c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:53:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.880+0000 7f3b32ffd640 10 client.17046 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:53:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:53:59.880+0000 7f3b52ffd640 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085f39e60 0x55b085f3c280 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f39227f23477c5 server_cookie=719f522c5f1c4328 in_seq=0 out_seq=0 2024-07-24T21:54:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.343+0000 7f3b32ffd640 1 -- 172.21.15.60:0/3689283037 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b085e277c0 con 0x55b085f39e60 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.343+0000 7f3b32ffd640 10 client.17046 handle_command_reply: tid=0 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.343+0000 7f3b5c0b8740 1 client.17046 shutdown 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 msgr2=0x55b085ec75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e8b0 tx=0x7f3b4c005650 comp rx=0 tx=0).stop 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 shutdown_connections 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b085f39e60 0x55b085f3c280 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b085ebf2e0 0x55b085eb8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b085eb3350 0x55b085ec7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 --2- 172.21.15.60:0/3689283037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b085eb2a30 0x55b085ec75d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 >> 172.21.15.60:0/3689283037 conn(0x55b085eb0860 msgr2=0x55b085eb6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b32ffd640 0 client.17046 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 shutdown_connections 2024-07-24T21:54:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 1 -- 172.21.15.60:0/3689283037 wait complete. 2024-07-24T21:54:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.344+0000 7f3b5c0b8740 20 client.17046 trim_cache size 0 max 16384 2024-07-24T21:54:01.347 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 38, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 20, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "b8eda14d": { 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 20, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:54:01.348 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:01.349 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 msgr2=0x7f3b3c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 0x7f3b3c0b6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b440cacf0 tx=0x7f3b44109040 comp rx=0 tx=0).stop 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 msgr2=0x7f3b54198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3b44005940 tx=0x7f3b440040c0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 shutdown_connections 2024-07-24T21:54:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3b3c0b3d40 0x7f3b3c0b6200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b5410cb00 0x7f3b5419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b54103fc0 0x7f3b54199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 --2- 172.21.15.60:0/3882331333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b541036a0 0x7f3b54198cd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.345+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 >> 172.21.15.60:0/3882331333 conn(0x7f3b540fd100 msgr2=0x7f3b540fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.346+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 shutdown_connections 2024-07-24T21:54:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:01.346+0000 7f3b59d07640 1 -- 172.21.15.60:0/3882331333 wait complete. 2024-07-24T21:54:01.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:01 smithi060 ceph-mon[33000]: pgmap v600: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 3.41k op/s 2024-07-24T21:54:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:01 smithi123 ceph-mon[37143]: pgmap v600: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 3.41k op/s 2024-07-24T21:54:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:01 smithi086 ceph-mon[37128]: pgmap v600: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 3.41k op/s 2024-07-24T21:54:02.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b8eda14d', total seconds elapsed: 75.1 2024-07-24T21:54:02.050 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-24T21:54:02.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:54:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:02.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:54:02.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:54:02.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:02 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:54:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:54:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:54:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:02 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:54:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:54:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:54:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:02 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:54:03.644 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:03 smithi123 ceph-mon[37143]: pgmap v601: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 959 KiB/s wr, 3.42k op/s 2024-07-24T21:54:03.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:03 smithi060 ceph-mon[33000]: pgmap v601: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 959 KiB/s wr, 3.42k op/s 2024-07-24T21:54:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:03 smithi086 ceph-mon[37128]: pgmap v601: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 959 KiB/s wr, 3.42k op/s 2024-07-24T21:54:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 -- 172.21.15.60:0/3115274800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103f80 msgr2=0x7fbadc10c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3115274800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103f80 0x7fbadc10c470 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009900 tx=0x7fbad002a410 comp rx=0 tx=0).stop 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 -- 172.21.15.60:0/3115274800 shutdown_connections 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3115274800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3115274800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103f80 0x7fbadc10c470 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3115274800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103660 0x7fbadc103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 -- 172.21.15.60:0/3115274800 >> 172.21.15.60:0/3115274800 conn(0x7fbadc0fd0c0 msgr2=0x7fbadc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 -- 172.21.15.60:0/3115274800 shutdown_connections 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.984+0000 7fbae33a5640 1 -- 172.21.15.60:0/3115274800 wait complete. 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 Processor -- start 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 -- start start 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 0x7fbadc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc112700 con 0x7fbadc103660 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc112580 con 0x7fbadc10cac0 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae33a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc112880 con 0x7fbadc103f80 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52058/0 (socket says 172.21.15.60:52058) 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 -- 172.21.15.60:0/3154346513 learned_addr learned my addr 172.21.15.60:0/3154346513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae0940640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 0x7fbadc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1942640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 -- 172.21.15.60:0/3154346513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 msgr2=0x7fbadc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 0x7fbadc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 -- 172.21.15.60:0/3154346513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 msgr2=0x7fbadc19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:03.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1141640 1 -- 172.21.15.60:0/3154346513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc19ed40 con 0x7fbadc103660 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae0940640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 0x7fbadc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.985+0000 7fbae1942640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbae1141640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbad400c890 tx=0x7fbad400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004220 con 0x7fbadc103660 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad40043c0 con 0x7fbadc103660 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc19f030 con 0x7fbadc103660 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004a00 con 0x7fbadc103660 2024-07-24T21:54:03.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.986+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc1a56f0 con 0x7fbadc103660 2024-07-24T21:54:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.987+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc103a40 con 0x7fbadc103660 2024-07-24T21:54:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.988+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fbad40079b0 con 0x7fbadc103660 2024-07-24T21:54:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.989+0000 7fbada7fc640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 0x7fbac40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.989+0000 7fbae0940640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 0x7fbac40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.989+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7fbad40e3ec0 con 0x7fbadc103660 2024-07-24T21:54:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.990+0000 7fbae0940640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 0x7fbac40b6200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbad0000c00 tx=0x7fbad0032e00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:03.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:03.993+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fbad4014030 con 0x7fbadc103660 2024-07-24T21:54:04.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.234+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbadc02e000 con 0x7fbadc103660 2024-07-24T21:54:04.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.234+0000 7fbada7fc640 1 -- 172.21.15.60:0/3154346513 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11919 (secure 0 0 0) 0x7fbad40ab140 con 0x7fbadc103660 2024-07-24T21:54:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:54:04.239 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"btime":"2024-07-24T21:53:35:326256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:53:34.943486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 38 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 msgr2=0x7fbac40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 0x7fbac40b6200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbad0000c00 tx=0x7fbad0032e00 comp rx=0 tx=0).stop 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 msgr2=0x7fbadc198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbad400c890 tx=0x7fbad400cd50 comp rx=0 tx=0).stop 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 shutdown_connections 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbac40b3d40 0x7fbac40b6200 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbadc10cac0 0x7fbadc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:04.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbadc103f80 0x7fbadc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:04.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 --2- 172.21.15.60:0/3154346513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbadc103660 0x7fbadc198c90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:04.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 >> 172.21.15.60:0/3154346513 conn(0x7fbadc0fd0c0 msgr2=0x7fbadc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:04.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 shutdown_connections 2024-07-24T21:54:04.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:04.237+0000 7fbae33a5640 1 -- 172.21.15.60:0/3154346513 wait complete. 2024-07-24T21:54:04.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:04 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3154346513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:04.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:04 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3154346513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:04 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3154346513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:04.908 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-24T21:54:04.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-24T21:54:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:05.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:05 smithi060 ceph-mon[33000]: pgmap v602: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 587 KiB/s wr, 3.37k op/s 2024-07-24T21:54:05.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:05 smithi123 ceph-mon[37143]: pgmap v602: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 587 KiB/s wr, 3.37k op/s 2024-07-24T21:54:05.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:05 smithi086 ceph-mon[37128]: pgmap v602: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 587 KiB/s wr, 3.37k op/s 2024-07-24T21:54:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:06 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:54:06.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:06 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:54:06.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:06 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 -- 172.21.15.60:0/2892983786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2810ee00 msgr2=0x7f8c281111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 --2- 172.21.15.60:0/2892983786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2810ee00 0x7f8c281111f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009be0 tx=0x7f8c2002a750 comp rx=0 tx=0).stop 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 -- 172.21.15.60:0/2892983786 shutdown_connections 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 --2- 172.21.15.60:0/2892983786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2810ee00 0x7f8c281111f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 --2- 172.21.15.60:0/2892983786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28101b70 0x7f8c2810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 --2- 172.21.15.60:0/2892983786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c28101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 -- 172.21.15.60:0/2892983786 >> 172.21.15.60:0/2892983786 conn(0x7f8c280fd0a0 msgr2=0x7f8c280ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:06.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 -- 172.21.15.60:0/2892983786 shutdown_connections 2024-07-24T21:54:06.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.800+0000 7f8c301bf640 1 -- 172.21.15.60:0/2892983786 wait complete. 2024-07-24T21:54:06.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 Processor -- start 2024-07-24T21:54:06.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 -- start start 2024-07-24T21:54:06.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c2819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 0x7f8c2819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2810ee00 0x7f8c281a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c28114940 con 0x7f8c2810ee00 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281147c0 con 0x7f8c28101b70 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c301bf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c28114ac0 con 0x7f8c28101250 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c2819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c2819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39798/0 (socket says 172.21.15.60:39798) 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2810ee00 0x7f8c281a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c2d75a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 0x7f8c2819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.801+0000 7f8c2df5b640 1 -- 172.21.15.60:0/3835335398 learned_addr learned my addr 172.21.15.60:0/3835335398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 -- 172.21.15.60:0/3835335398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 msgr2=0x7f8c2819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c2819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 -- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2810ee00 msgr2=0x7f8c281a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2810ee00 0x7f8c281a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 -- 172.21.15.60:0/3835335398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c281a0ec0 con 0x7f8c28101b70 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 0x7f8c2819b350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e790 tx=0x7f8c1800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c1effd640 1 -- 172.21.15.60:0/3835335398 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800eea0 con 0x7f8c28101b70 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c281a11b0 con 0x7f8c28101b70 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c281a7a40 con 0x7f8c28101b70 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c1effd640 1 -- 172.21.15.60:0/3835335398 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c1800ccc0 con 0x7f8c28101b70 2024-07-24T21:54:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.802+0000 7f8c1effd640 1 -- 172.21.15.60:0/3835335398 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18018680 con 0x7f8c28101b70 2024-07-24T21:54:06.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.805+0000 7f8c32570740 1 Processor -- start 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 -- start start 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c1effd640 1 -- 172.21.15.60:0/3835335398 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8c18010430 con 0x7f8c28101b70 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bcdd155670 con 0x55bcdd1e1050 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bcdd144b70 con 0x55bcdd1eb920 2024-07-24T21:54:06.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c32570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bcdd148bb0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48492/0 (socket says 172.21.15.60:48492) 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c2e75c640 1 -- 172.21.15.60:0/1389559315 learned_addr learned my addr 172.21.15.60:0/1389559315 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.806+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 -- 172.21.15.60:0/1389559315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 msgr2=0x55bcdd1edd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1edd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 -- 172.21.15.60:0/1389559315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 msgr2=0x55bcdd1df5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 -- 172.21.15.60:0/1389559315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcdd0dcd30 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1edd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:06.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2e75c640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1dff10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009c80 tx=0x7f8c24011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983f1488c9c07250 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c1effd640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 0x7f8c140b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c13fff640 1 -- 172.21.15.60:0/1389559315 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004230 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c13fff640 1 -- 172.21.15.60:0/1389559315 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c240043d0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c13fff640 1 -- 172.21.15.60:0/1389559315 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004d80 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c2e75c640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 0x7f8c140b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.807+0000 7f8c1effd640 1 -- 172.21.15.60:0/3835335398 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f8c180e3ff0 con 0x7f8c28101b70 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- 172.21.15.60:0/1389559315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 msgr2=0x55bcdd1dff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c2e75c640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 0x7f8c140b6280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006010 tx=0x7f8c20004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1dff10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009c80 tx=0x7f8c24011810 comp rx=0 tx=0).stop 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- 172.21.15.60:0/1389559315 shutdown_connections 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1edd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1dff10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 --2- 172.21.15.60:0/1389559315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- 172.21.15.60:0/1389559315 >> 172.21.15.60:0/1389559315 conn(0x55bcdd1a9b10 msgr2=0x55bcdd1dd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- 172.21.15.60:0/1389559315 shutdown_connections 2024-07-24T21:54:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- 172.21.15.60:0/1389559315 wait complete. 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 -- start start 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.808+0000 7f8c32570740 1 Processor -- start 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 0x55bcdd1e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 0x55bcdd1e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bcdcfa87a0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bcdd0e6ff0 con 0x55bcdd1e1050 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bcdd1ee7e0 con 0x55bcdd1eb920 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c32570740 10 client.0 _ll_register_callbacks cb 0x55bcdce52090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 0x55bcdd1e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 0x55bcdd1e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39812/0 (socket says 172.21.15.60:39812) 2024-07-24T21:54:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48496/0 (socket says 172.21.15.60:48496) 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2e75c640 1 -- 172.21.15.60:0/1305726167 learned_addr learned my addr 172.21.15.60:0/1305726167 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 0x55bcdd1e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 msgr2=0x55bcdd1e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 0x55bcdd1e92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1305726167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 msgr2=0x55bcdd1e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.809+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1305726167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcdd0eb0a0 con 0x55bcdd1e1050 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 0x55bcdd1e8db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c180e1710 tx=0x7f8c180a6c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b968c24b33b797b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180e1e60 con 0x55bcdd1e1050 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f8c180b37f0 con 0x55bcdd1e1050 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c240043d0 con 0x55bcdd1e1050 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c180a6410 con 0x55bcdd1e1050 2024-07-24T21:54:06.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 10 client.26498 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bcdceb4b90 con 0x55bcdd1e1050 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c240043d0 con 0x55bcdd1e1050 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8c180920e0 con 0x55bcdd1e1050 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 10 client.26498 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 10 client.26498 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 20 client.26498 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.810+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd1fb9f0 0x55bcdd1fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.811+0000 7f8c32570740 4 client.26498 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.811+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcdd1575c0 con 0x55bcdd1fb9f0 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.811+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd1fb9f0 0x55bcdd1fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.811+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd1fb9f0 0x55bcdd1fde50 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c258a26d08c01259 server_cookie=10b7c6b05fcc1352 in_seq=0 out_seq=0 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.811+0000 7f8c117fa640 10 client.26498 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:06.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c117fa640 1 -- 172.21.15.60:0/1305726167 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcdd1575c0 con 0x55bcdd1fb9f0 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c117fa640 10 client.26498 handle_command_reply: tid=0 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 client.26498 shutdown 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 msgr2=0x55bcdd1e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 0x55bcdd1e8db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8c180e1710 tx=0x7f8c180a6c30 comp rx=0 tx=0).stop 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 shutdown_connections 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd1fb9f0 0x55bcdd1fde50 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1eb920 0x55bcdd1e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1e1050 0x55bcdd1e8db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 --2- 172.21.15.60:0/1305726167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c117fa640 0 client.26498 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 >> 172.21.15.60:0/1305726167 conn(0x55bcdd1a9b10 msgr2=0x55bcdd1e2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 shutdown_connections 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.812+0000 7f8c32570740 1 -- 172.21.15.60:0/1305726167 wait complete. 2024-07-24T21:54:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.813+0000 7f8c32570740 20 client.26498 trim_cache size 0 max 16384 2024-07-24T21:54:06.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 Processor -- start 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 -- start start 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c011200 con 0x55bcdd1e1050 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00ea40 con 0x55bcdd1eb920 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c32570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00c8b0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48500/0 (socket says 172.21.15.60:48500) 2024-07-24T21:54:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1034439313 learned_addr learned my addr 172.21.15.60:0/1034439313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2e75c640 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1034439313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 msgr2=0x55bcdd1ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1034439313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 msgr2=0x55bcdd1dce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1dce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.833+0000 7f8c2d75a640 1 -- 172.21.15.60:0/1034439313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcdd1eacf0 con 0x55bcdd1e1050 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.834+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c180e1d10 tx=0x7f8c1800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c446a8fe64ea2d3d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.834+0000 7f8c137fe640 1 -- 172.21.15.60:0/1034439313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180e1e60 con 0x55bcdd1e1050 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.834+0000 7f8c137fe640 1 -- 172.21.15.60:0/1034439313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c180aa790 con 0x55bcdd1e1050 2024-07-24T21:54:06.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.834+0000 7f8c137fe640 1 -- 172.21.15.60:0/1034439313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180b38c0 con 0x55bcdd1e1050 2024-07-24T21:54:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- 172.21.15.60:0/1034439313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 msgr2=0x55bcdd1df290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c180e1d10 tx=0x7f8c1800cf80 comp rx=0 tx=0).stop 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- 172.21.15.60:0/1034439313 shutdown_connections 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1eb920 0x55bcdd1dce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1e1050 0x55bcdd1df290 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 --2- 172.21.15.60:0/1034439313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- 172.21.15.60:0/1034439313 >> 172.21.15.60:0/1034439313 conn(0x55bcdd1a9b10 msgr2=0x55bcdd1dc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- 172.21.15.60:0/1034439313 shutdown_connections 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- 172.21.15.60:0/1034439313 wait complete. 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.835+0000 7f8c32570740 1 -- start start 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 Processor -- start 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 0x55bcdd1f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 0x55bcdd1eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00cf00 con 0x55bcdd1eb920 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00e370 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00d5d0 con 0x55bcdd1e1050 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c32570740 10 client.0 _ll_register_callbacks cb 0x7f8c18009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2e75c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 0x55bcdd1eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2d75a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 0x55bcdd1f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2df5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:57672/0 (socket says 172.21.15.60:57672) 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2df5b640 1 -- 172.21.15.60:0/4035901705 learned_addr learned my addr 172.21.15.60:0/4035901705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.836+0000 7f8c2d75a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 0x55bcdd1f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39830/0 (socket says 172.21.15.60:39830) 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 -- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 msgr2=0x55bcdd1f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 0x55bcdd1f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 -- 172.21.15.60:0/4035901705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 msgr2=0x55bcdd1eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 0x55bcdd1eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 -- 172.21.15.60:0/4035901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcdd1ef000 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2e75c640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 0x55bcdd1eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c2df5b640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009c80 tx=0x7f8c24004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65403aff22c33ad0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24032060 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f8c240035f0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c180aa790 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c2401e040 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c32570740 10 client.26504 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.837+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bcdd266ee0 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c180aa790 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8c24024040 con 0x55bcdd1dfaf0 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 10 client.26504 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 10 client.26504 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 20 client.26504 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd267560 0x55bcdd269980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 4 client.26504 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55bcdd153060 con 0x55bcdd267560 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.838+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd267560 0x55bcdd269980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.839+0000 7f8c117fa640 10 client.26504 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:06.839+0000 7f8c2d75a640 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd267560 0x55bcdd269980 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6461429ac935895 server_cookie=1a6f2a5e86d0f795 in_seq=0 out_seq=0 2024-07-24T21:54:07.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:07 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:54:07.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:07 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:54:07.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:07 smithi086 ceph-mon[37128]: pgmap v603: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 419 KiB/s wr, 3.16k op/s 2024-07-24T21:54:07.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:07 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:07 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:54:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:07 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:54:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:07 smithi123 ceph-mon[37143]: pgmap v603: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 419 KiB/s wr, 3.16k op/s 2024-07-24T21:54:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:07 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:07 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:54:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:07 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:54:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:07 smithi060 ceph-mon[33000]: pgmap v603: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 419 KiB/s wr, 3.16k op/s 2024-07-24T21:54:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:07 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:09.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:09 smithi123 ceph-mon[37143]: pgmap v604: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 292 KiB/s wr, 1.97k op/s 2024-07-24T21:54:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:09 smithi086 ceph-mon[37128]: pgmap v604: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 292 KiB/s wr, 1.97k op/s 2024-07-24T21:54:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:09 smithi060 ceph-mon[33000]: pgmap v604: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 292 KiB/s wr, 1.97k op/s 2024-07-24T21:54:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:11 smithi123 ceph-mon[37143]: pgmap v605: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 597 KiB/s wr, 799 op/s 2024-07-24T21:54:11.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:11 smithi086 ceph-mon[37128]: pgmap v605: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 597 KiB/s wr, 799 op/s 2024-07-24T21:54:11.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:11 smithi060 ceph-mon[33000]: pgmap v605: 97 pgs: 97 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 597 KiB/s wr, 799 op/s 2024-07-24T21:54:12.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:12 smithi123 ceph-mon[37143]: osdmap e91: 12 total, 12 up, 12 in 2024-07-24T21:54:12.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:12 smithi086 ceph-mon[37128]: osdmap e91: 12 total, 12 up, 12 in 2024-07-24T21:54:12.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:12 smithi060 ceph-mon[33000]: osdmap e91: 12 total, 12 up, 12 in 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3218835172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 msgr2=0x7f2b7410ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3218835172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 0x7f2b7410ee90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009860 tx=0x7f2b6c02a340 comp rx=0 tx=0).stop 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3218835172 shutdown_connections 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3218835172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 0x7f2b7410ee90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3218835172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74103ee0 0x7f2b7410c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3218835172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b741035c0 0x7f2b741039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3218835172 >> 172.21.15.60:0/3218835172 conn(0x7f2b740fd010 msgr2=0x7f2b740ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3218835172 shutdown_connections 2024-07-24T21:54:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.931+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3218835172 wait complete. 2024-07-24T21:54:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.932+0000 7f2b7b96f640 1 Processor -- start 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.932+0000 7f2b7b96f640 1 -- start start 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b741035c0 0x7f2b740741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 0x7f2b7406d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741148a0 con 0x7f2b74103ee0 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74114720 con 0x7f2b7410ca10 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b7b96f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74114a20 con 0x7f2b741035c0 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b78f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b78f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48518/0 (socket says 172.21.15.60:48518) 2024-07-24T21:54:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.933+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3646713426 learned_addr learned my addr 172.21.15.60:0/3646713426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3646713426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b741035c0 msgr2=0x7f2b740741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b741035c0 0x7f2b740741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3646713426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 msgr2=0x7f2b7406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 0x7f2b7406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3646713426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7406de50 con 0x7f2b74103ee0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.934+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400ef40 tx=0x7f2b6400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.935+0000 7f2b727fc640 1 -- 172.21.15.60:0/3646713426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64003970 con 0x7f2b74103ee0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.935+0000 7f2b727fc640 1 -- 172.21.15.60:0/3646713426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64004020 con 0x7f2b74103ee0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.935+0000 7f2b727fc640 1 -- 172.21.15.60:0/3646713426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400b860 con 0x7f2b74103ee0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.935+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7406e140 con 0x7f2b74103ee0 2024-07-24T21:54:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.935+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b74073070 con 0x7f2b74103ee0 2024-07-24T21:54:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.940+0000 7f2b727fc640 1 -- 172.21.15.60:0/3646713426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2b6401d020 con 0x7f2b74103ee0 2024-07-24T21:54:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.940+0000 7f2b7e79b740 1 Processor -- start 2024-07-24T21:54:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.941+0000 7f2b7e79b740 1 -- start start 2024-07-24T21:54:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.941+0000 7f2b727fc640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 0x7f2b5c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.941+0000 7f2b727fc640 1 -- 172.21.15.60:0/3646713426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f2b640e3fe0 con 0x7f2b74103ee0 2024-07-24T21:54:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.941+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 0x7f2b5c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.941+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 0x55a8bfff3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39870/0 (socket says 172.21.15.60:39870) 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7970b640 1 -- 172.21.15.60:0/1711513719 learned_addr learned my addr 172.21.15.60:0/1711513719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a8bff5c280 con 0x55a8bfff3540 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8bff4b780 con 0x55a8bffe75b0 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 0x55a8bfff3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.942+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8bff4f7c0 con 0x55a8bffe6c90 2024-07-24T21:54:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b7970b640 1 -- 172.21.15.60:0/1711513719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 msgr2=0x55a8bfff3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 0x55a8bfff3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b7970b640 1 -- 172.21.15.60:0/1711513719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 msgr2=0x55a8bfff5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b7970b640 1 -- 172.21.15.60:0/1711513719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8bfcaf790 con 0x55a8bffe6c90 2024-07-24T21:54:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.943+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 0x55a8bfff3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.944+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800ba80 tx=0x7f2b6800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25368075257daa4a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.944+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 0x7f2b5c0b6350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c035040 tx=0x7f2b6c030340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.944+0000 7f2b5b7fe640 1 -- 172.21.15.60:0/1711513719 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68016070 con 0x55a8bffe6c90 2024-07-24T21:54:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.945+0000 7f2b5b7fe640 1 -- 172.21.15.60:0/1711513719 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b680054d0 con 0x55a8bffe6c90 2024-07-24T21:54:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.945+0000 7f2b5b7fe640 1 -- 172.21.15.60:0/1711513719 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004e10 con 0x55a8bffe6c90 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 msgr2=0x55a8bffe7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800ba80 tx=0x7f2b6800bf40 comp rx=0 tx=0).stop 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 shutdown_connections 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe75b0 0x55a8bfff3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1711513719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bffe7070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 >> 172.21.15.60:0/1711513719 conn(0x55a8bffe4ac0 msgr2=0x55a8bffe5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 shutdown_connections 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.946+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1711513719 wait complete. 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.947+0000 7f2b7e79b740 1 -- start start 2024-07-24T21:54:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.947+0000 7f2b7e79b740 1 Processor -- start 2024-07-24T21:54:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.947+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.947+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe75b0 0x55a8bfff0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:57710/0 (socket says 172.21.15.60:57710) 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a8bfe85ab0 con 0x55a8bfff3540 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8bfed39e0 con 0x55a8bffe6c90 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8bfff6220 con 0x55a8bffe75b0 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b78f0a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe75b0 0x55a8bfff0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 -- 172.21.15.60:0/1663346385 learned_addr learned my addr 172.21.15.60:0/1663346385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7e79b740 10 client.0 _ll_register_callbacks cb 0x55a8bfc5ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 -- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe75b0 msgr2=0x55a8bfff0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe75b0 0x55a8bfff0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 -- 172.21.15.60:0/1663346385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 msgr2=0x55a8bfff0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.948+0000 7f2b7970b640 1 -- 172.21.15.60:0/1663346385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8bfd28620 con 0x55a8bffe6c90 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b7970b640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005940 tx=0x7f2b68004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee2c26d970cfcbb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68016030 con 0x55a8bffe6c90 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8bfeface0 con 0x55a8bffe6c90 2024-07-24T21:54:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f2b68004040 con 0x55a8bffe6c90 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b68004240 con 0x55a8bffe6c90 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b7e79b740 10 client.26513 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8c0001880 con 0x55a8bffe6c90 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.949+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8bfeface0 con 0x55a8bffe6c90 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f2b68022070 con 0x55a8bffe6c90 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 10 client.26513 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 20 client.26513 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c0002c90 0x55a8c0005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 4 client.26513 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.950+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8bff5e1d0 con 0x55a8c0002c90 2024-07-24T21:54:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.954+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c0002c90 0x55a8c0005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.960+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c0002c90 0x55a8c0005080 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32f529942f5d5fc server_cookie=2cfa07f4f2595d41 in_seq=0 out_seq=0 2024-07-24T21:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:12.960+0000 7f2b58ff9640 10 client.26513 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b58ff9640 1 -- 172.21.15.60:0/1663346385 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8bff5e1d0 con 0x55a8c0002c90 2024-07-24T21:54:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b58ff9640 10 client.26513 handle_command_reply: tid=0 2024-07-24T21:54:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 client.26513 shutdown 2024-07-24T21:54:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 msgr2=0x55a8bfff2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005940 tx=0x7f2b68004e00 comp rx=0 tx=0).stop 2024-07-24T21:54:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 shutdown_connections 2024-07-24T21:54:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c0002c90 0x55a8c0005080 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bfff3540 0x55a8bfff0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe75b0 0x55a8bfff0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/1663346385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfff2b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b58ff9640 0 client.26513 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.108+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 >> 172.21.15.60:0/1663346385 conn(0x55a8bffe4ac0 msgr2=0x55a8bffea660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.109+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 shutdown_connections 2024-07-24T21:54:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.109+0000 7f2b7e79b740 1 -- 172.21.15.60:0/1663346385 wait complete. 2024-07-24T21:54:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.109+0000 7f2b7e79b740 20 client.26513 trim_cache size 0 max 16384 2024-07-24T21:54:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.128+0000 7f2b7e79b740 1 Processor -- start 2024-07-24T21:54:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.128+0000 7f2b7e79b740 1 -- start start 2024-07-24T21:54:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.128+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bfff0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.128+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.128+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bfff3540 0x55a8bffea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2b600112a0 con 0x55a8bffe75b0 2024-07-24T21:54:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6000eae0 con 0x55a8bfff3540 2024-07-24T21:54:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6000c950 con 0x55a8bffe6c90 2024-07-24T21:54:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48540/0 (socket says 172.21.15.60:48540) 2024-07-24T21:54:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3860459887 learned_addr learned my addr 172.21.15.60:0/3860459887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bfff3540 0x55a8bffea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b7970b640 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bfff0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3860459887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 msgr2=0x55a8bfff0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bfff0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3860459887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bfff3540 msgr2=0x55a8bffea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bfff3540 0x55a8bffea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 -- 172.21.15.60:0/3860459887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8bffe7990 con 0x55a8bffe75b0 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000f80 tx=0x7f2b64004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd482e0da6724ffd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.129+0000 7f2b5affd640 1 -- 172.21.15.60:0/3860459887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b640ca070 con 0x55a8bffe75b0 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.130+0000 7f2b5affd640 1 -- 172.21.15.60:0/3860459887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b640b3ba0 con 0x55a8bffe75b0 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.130+0000 7f2b5affd640 1 -- 172.21.15.60:0/3860459887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b640e1a90 con 0x55a8bffe75b0 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.130+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3860459887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 msgr2=0x55a8bffeb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.130+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000f80 tx=0x7f2b64004440 comp rx=0 tx=0).stop 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3860459887 shutdown_connections 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bfff3540 0x55a8bffea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bffeb760 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3860459887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bffe6c90 0x55a8bfff0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3860459887 >> 172.21.15.60:0/3860459887 conn(0x55a8bffe4ac0 msgr2=0x55a8bffe9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3860459887 shutdown_connections 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3860459887 wait complete. 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 -- start start 2024-07-24T21:54:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.131+0000 7f2b7e79b740 1 Processor -- start 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bfffba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 0x55a8bffed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6000cfa0 con 0x55a8bffe75b0 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6000e410 con 0x55a8bffe6c90 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6000d670 con 0x55a8bfff3540 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7e79b740 10 client.0 _ll_register_callbacks cb 0x7f2b6800b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7970b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:57726/0 (socket says 172.21.15.60:57726) 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b7970b640 1 -- 172.21.15.60:0/3745808733 learned_addr learned my addr 172.21.15.60:0/3745808733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.132+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 0x55a8bffed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b78f0a640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bfffba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 -- 172.21.15.60:0/3745808733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 msgr2=0x55a8bfffb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 -- 172.21.15.60:0/3745808733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 msgr2=0x55a8bfffba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bfffba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 -- 172.21.15.60:0/3745808733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8bffed770 con 0x55a8bfff3540 2024-07-24T21:54:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b7970b640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b79f0c640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 0x55a8bffed230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009830 tx=0x7f2b6c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486554334761c099 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.133+0000 7f2b58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:54:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c04f070 con 0x55a8bfff3540 2024-07-24T21:54:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f2b6c030450 con 0x55a8bfff3540 2024-07-24T21:54:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b640b3ba0 con 0x55a8bfff3540 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b6c037040 con 0x55a8bfff3540 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b7e79b740 10 client.26554 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.134+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8bfff4140 con 0x55a8bfff3540 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b640b3ba0 con 0x55a8bfff3540 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f2b6c03a030 con 0x55a8bfff3540 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b7e79b740 10 client.26554 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b7e79b740 10 client.26554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b7e79b740 20 client.26554 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.135+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c006d920 0x55a8c006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.136+0000 7f2b7e79b740 4 client.26554 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:54:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.136+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8bff59c70 con 0x55a8c006d920 2024-07-24T21:54:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.137+0000 7f2b7970b640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c006d920 0x55a8c006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.146+0000 7f2b58ff9640 10 client.26554 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:13.146+0000 7f2b7970b640 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c006d920 0x55a8c006fd10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4513b4168cdadf75 server_cookie=302017b0d56720d4 in_seq=0 out_seq=0 2024-07-24T21:54:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:13 smithi123 ceph-mon[37143]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 655 KiB/s wr, 125 op/s 2024-07-24T21:54:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:13 smithi086 ceph-mon[37128]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 655 KiB/s wr, 125 op/s 2024-07-24T21:54:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:13 smithi060 ceph-mon[33000]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 655 KiB/s wr, 125 op/s 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b58ff9640 1 -- 172.21.15.60:0/3745808733 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8bff59c70 con 0x55a8c006d920 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b58ff9640 10 client.26554 handle_command_reply: tid=0 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 client.26554 shutdown 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 msgr2=0x55a8bffed230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 0x55a8bffed230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009830 tx=0x7f2b6c005f90 comp rx=0 tx=0).stop 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 shutdown_connections 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a8c006d920 0x55a8c006fd10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b58ff9640 0 client.26554 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8bfff3540 0x55a8bffed230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8bffe75b0 0x55a8bfffba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 --2- 172.21.15.60:0/3745808733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8bffe6c90 0x55a8bfffb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.566+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 >> 172.21.15.60:0/3745808733 conn(0x55a8bffe4ac0 msgr2=0x55a8bffe3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.567+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 shutdown_connections 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.567+0000 7f2b7e79b740 1 -- 172.21.15.60:0/3745808733 wait complete. 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.567+0000 7f2b7e79b740 20 client.26554 trim_cache size 0 max 16384 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:54:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 msgr2=0x7f2b5c0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 0x7f2b5c0b6350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c035040 tx=0x7f2b6c030340 comp rx=0 tx=0).stop 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 msgr2=0x7f2b74074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400ef40 tx=0x7f2b6400c800 comp rx=0 tx=0).stop 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 shutdown_connections 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2b5c0b3e90 0x7f2b5c0b6350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b7410ca10 0x7f2b7406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b74103ee0 0x7f2b74074710 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 --2- 172.21.15.60:0/3646713426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b741035c0 0x7f2b740741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 >> 172.21.15.60:0/3646713426 conn(0x7f2b740fd010 msgr2=0x7f2b740fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 shutdown_connections 2024-07-24T21:54:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:14.568+0000 7f2b7b96f640 1 -- 172.21.15.60:0/3646713426 wait complete. 2024-07-24T21:54:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:15 smithi123 ceph-mon[37143]: pgmap v608: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 648 KiB/s wr, 102 op/s 2024-07-24T21:54:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:15 smithi086 ceph-mon[37128]: pgmap v608: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 648 KiB/s wr, 102 op/s 2024-07-24T21:54:15.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:15 smithi060 ceph-mon[33000]: pgmap v608: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 648 KiB/s wr, 102 op/s 2024-07-24T21:54:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:17 smithi123 ceph-mon[37143]: pgmap v609: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 790 KiB/s wr, 112 op/s 2024-07-24T21:54:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:17 smithi086 ceph-mon[37128]: pgmap v609: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 790 KiB/s wr, 112 op/s 2024-07-24T21:54:17.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:17 smithi060 ceph-mon[33000]: pgmap v609: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 790 KiB/s wr, 112 op/s 2024-07-24T21:54:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:19 smithi123 ceph-mon[37143]: pgmap v610: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 715 KiB/s wr, 90 op/s 2024-07-24T21:54:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:19 smithi086 ceph-mon[37128]: pgmap v610: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 715 KiB/s wr, 90 op/s 2024-07-24T21:54:19.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:19 smithi060 ceph-mon[33000]: pgmap v610: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 715 KiB/s wr, 90 op/s 2024-07-24T21:54:21.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.692+0000 7f8c117fa640 1 -- 172.21.15.60:0/4035901705 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bcdd153060 con 0x55bcdd267560 2024-07-24T21:54:21.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.692+0000 7f8c117fa640 10 client.26504 handle_command_reply: tid=0 2024-07-24T21:54:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.692+0000 7f8c32570740 1 client.26504 shutdown 2024-07-24T21:54:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 msgr2=0x55bcdd1f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009c80 tx=0x7f8c24004270 comp rx=0 tx=0).stop 2024-07-24T21:54:21.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:21 smithi060 ceph-mon[33000]: pgmap v611: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 352 KiB/s wr, 67 op/s 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 shutdown_connections 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bcdd267560 0x55bcdd269980 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcdd1eb920 0x55bcdd1eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bcdd1e1050 0x55bcdd1f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c117fa640 0 client.26504 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 --2- 172.21.15.60:0/4035901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bcdd1dfaf0 0x55bcdd1f3b10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 >> 172.21.15.60:0/4035901705 conn(0x55bcdd1a9b10 msgr2=0x55bcdd1dbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 shutdown_connections 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 1 -- 172.21.15.60:0/4035901705 wait complete. 2024-07-24T21:54:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.693+0000 7f8c32570740 20 client.26504 trim_cache size 0 max 16384 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 39, 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 22, 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "cbe533b0": { 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 22, 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:54:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 msgr2=0x7f8c140b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 0x7f8c140b6280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006010 tx=0x7f8c20004a40 comp rx=0 tx=0).stop 2024-07-24T21:54:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 msgr2=0x7f8c2819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 0x7f8c2819b350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e790 tx=0x7f8c1800ec50 comp rx=0 tx=0).stop 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 shutdown_connections 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c140b3dc0 0x7f8c140b6280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2810ee00 0x7f8c281a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c28101b70 0x7f8c2819b350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 --2- 172.21.15.60:0/3835335398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c28101250 0x7f8c2819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 >> 172.21.15.60:0/3835335398 conn(0x7f8c280fd0a0 msgr2=0x7f8c281110e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 shutdown_connections 2024-07-24T21:54:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:21.694+0000 7f8c301bf640 1 -- 172.21.15.60:0/3835335398 wait complete. 2024-07-24T21:54:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:21 smithi123 ceph-mon[37143]: pgmap v611: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 352 KiB/s wr, 67 op/s 2024-07-24T21:54:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:21 smithi086 ceph-mon[37128]: pgmap v611: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 352 KiB/s wr, 67 op/s 2024-07-24T21:54:22.692 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cbe533b0', quiesce took 20.6 seconds. Will release after: 9.399999999999999 2024-07-24T21:54:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:23 smithi123 ceph-mon[37143]: pgmap v612: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-07-24T21:54:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:23 smithi086 ceph-mon[37128]: pgmap v612: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-07-24T21:54:23.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:23 smithi060 ceph-mon[33000]: pgmap v612: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 281 KiB/s wr, 55 op/s 2024-07-24T21:54:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:25 smithi123 ceph-mon[37143]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 236 KiB/s wr, 51 op/s 2024-07-24T21:54:25.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:25 smithi060 ceph-mon[33000]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 236 KiB/s wr, 51 op/s 2024-07-24T21:54:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:25 smithi086 ceph-mon[37128]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 236 KiB/s wr, 51 op/s 2024-07-24T21:54:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 -- 172.21.15.60:0/1662653321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 msgr2=0x7f9eec1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 --2- 172.21.15.60:0/1662653321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec1084e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009900 tx=0x7f9ee002a410 comp rx=0 tx=0).stop 2024-07-24T21:54:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 -- 172.21.15.60:0/1662653321 shutdown_connections 2024-07-24T21:54:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 --2- 172.21.15.60:0/1662653321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec10f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 --2- 172.21.15.60:0/1662653321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec1084e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 --2- 172.21.15.60:0/1662653321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 -- 172.21.15.60:0/1662653321 >> 172.21.15.60:0/1662653321 conn(0x7f9eec0fe0e0 msgr2=0x7f9eec100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 -- 172.21.15.60:0/1662653321 shutdown_connections 2024-07-24T21:54:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.673+0000 7f9ef1841640 1 -- 172.21.15.60:0/1662653321 wait complete. 2024-07-24T21:54:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 Processor -- start 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 -- start start 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1126e0 con 0x7f9eec1080a0 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112560 con 0x7f9eec108a20 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9ef1841640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112860 con 0x7f9eec1026a0 2024-07-24T21:54:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9eea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9eea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44858/0 (socket says 172.21.15.60:44858) 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.674+0000 7f9eea7fc640 1 -- 172.21.15.60:0/98850915 learned_addr learned my addr 172.21.15.60:0/98850915 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 -- 172.21.15.60:0/98850915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 msgr2=0x7f9eec198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 -- 172.21.15.60:0/98850915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 msgr2=0x7f9eec19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 -- 172.21.15.60:0/98850915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec19eca0 con 0x7f9eec1080a0 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ab0 tx=0x7f9ee0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9ef083f640 1 -- 172.21.15.60:0/98850915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003f070 con 0x7f9eec1080a0 2024-07-24T21:54:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9ef083f640 1 -- 172.21.15.60:0/98850915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee0031d60 con 0x7f9eec1080a0 2024-07-24T21:54:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec19ef30 con 0x7f9eec1080a0 2024-07-24T21:54:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9ef083f640 1 -- 172.21.15.60:0/98850915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0035030 con 0x7f9eec1080a0 2024-07-24T21:54:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.675+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a5650 con 0x7f9eec1080a0 2024-07-24T21:54:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 Processor -- start 2024-07-24T21:54:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef083f640 1 -- 172.21.15.60:0/98850915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9ee0042070 con 0x7f9eec1080a0 2024-07-24T21:54:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 -- start start 2024-07-24T21:54:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56301301d280 con 0x5630130a85b0 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56301300c780 con 0x5630130b4540 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9ef4596740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630130107c0 con 0x5630130a7c90 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9eeaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9eeaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44872/0 (socket says 172.21.15.60:44872) 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2219403705 learned_addr learned my addr 172.21.15.60:0/2219403705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.678+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2219403705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 msgr2=0x5630130a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2219403705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 msgr2=0x5630130b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130b6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 -- 172.21.15.60:0/2219403705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563012d70790 con 0x5630130a85b0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9ef083f640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 0x7f9ed40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00bd40 tx=0x7f9edc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d41f54bb5ca756 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 0x7f9ed40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9ef083f640 1 -- 172.21.15.60:0/98850915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f9ee0104890 con 0x7f9eec1080a0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9ed1ffb640 1 -- 172.21.15.60:0/2219403705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc015070 con 0x5630130a85b0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9ed1ffb640 1 -- 172.21.15.60:0/2219403705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc00be30 con 0x5630130a85b0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.679+0000 7f9ed1ffb640 1 -- 172.21.15.60:0/2219403705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0106e0 con 0x5630130a85b0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 -- 172.21.15.60:0/2219403705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 msgr2=0x5630130b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 0x7f9ed40b6200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005fd0 tx=0x7f9ee4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00bd40 tx=0x7f9edc00f730 comp rx=0 tx=0).stop 2024-07-24T21:54:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 -- 172.21.15.60:0/2219403705 shutdown_connections 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130b4000 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 --2- 172.21.15.60:0/2219403705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 -- 172.21.15.60:0/2219403705 >> 172.21.15.60:0/2219403705 conn(0x5630130a5ac0 msgr2=0x5630130a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 -- 172.21.15.60:0/2219403705 shutdown_connections 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.680+0000 7f9ef4596740 1 -- 172.21.15.60:0/2219403705 wait complete. 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 -- start start 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 Processor -- start 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a7c90 0x5630130b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a85b0 0x5630130b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563012f46ab0 con 0x5630130b4540 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563012f949e0 con 0x5630130a7c90 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630130b7220 con 0x5630130a85b0 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9ef4596740 10 client.0 _ll_register_callbacks cb 0x563012d1ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44874/0 (socket says 172.21.15.60:44874) 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/709871152 learned_addr learned my addr 172.21.15.60:0/709871152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a7c90 0x5630130b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.681+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a85b0 0x5630130b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a85b0 msgr2=0x5630130b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a85b0 0x5630130b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/709871152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a7c90 msgr2=0x5630130b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a7c90 0x5630130b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/709871152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563012de9620 con 0x5630130b4540 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4003f20 tx=0x7f9ee4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c31ee31b352cf07 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40047d0 con 0x5630130b4540 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f9ee401f430 con 0x5630130b4540 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563012fbbce0 con 0x5630130b4540 2024-07-24T21:54:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ee401f630 con 0x5630130b4540 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ef4596740 10 client.17100 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.682+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5630130c29c0 con 0x5630130b4540 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563012fbbce0 con 0x5630130b4540 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f9ee4027070 con 0x5630130b4540 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 10 client.17100 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 20 client.17100 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5630130c3f40 0x5630130c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 4 client.17100 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56301301f1d0 con 0x5630130c3f40 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5630130c3f40 0x5630130c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9ecaffd640 10 client.17100 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.683+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5630130c3f40 0x5630130c6370 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d4b3da9f8572a9 server_cookie=d6950ce84cb4214e in_seq=0 out_seq=0 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ecaffd640 1 -- 172.21.15.60:0/709871152 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56301301f1d0 con 0x5630130c3f40 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ecaffd640 10 client.17100 handle_command_reply: tid=0 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 client.17100 shutdown 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 msgr2=0x5630130b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4003f20 tx=0x7f9ee4004ab0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 shutdown_connections 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5630130c3f40 0x5630130c6370 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b1620 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a85b0 0x5630130b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 --2- 172.21.15.60:0/709871152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a7c90 0x5630130b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ecaffd640 0 client.17100 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.684+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 >> 172.21.15.60:0/709871152 conn(0x5630130a5ac0 msgr2=0x5630130ab660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.685+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 shutdown_connections 2024-07-24T21:54:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.685+0000 7f9ef4596740 1 -- 172.21.15.60:0/709871152 wait complete. 2024-07-24T21:54:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.685+0000 7f9ef4596740 20 client.17100 trim_cache size 0 max 16384 2024-07-24T21:54:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 Processor -- start 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 -- start start 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1b0b10 con 0x5630130a85b0 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1ae350 con 0x5630130b4540 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9ef4596740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1ac1c0 con 0x5630130a7c90 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9eea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9eea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44882/0 (socket says 172.21.15.60:44882) 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9eea7fc640 1 -- 172.21.15.60:0/1553643239 learned_addr learned my addr 172.21.15.60:0/1553643239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.700+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 -- 172.21.15.60:0/1553643239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 msgr2=0x5630130cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 -- 172.21.15.60:0/1553643239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 msgr2=0x5630130ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 -- 172.21.15.60:0/1553643239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630130b0e60 con 0x5630130a85b0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ec0 tx=0x7f9ee010a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf4714dd4bfed3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9ed17fa640 1 -- 172.21.15.60:0/1553643239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0049040 con 0x5630130a85b0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9ed17fa640 1 -- 172.21.15.60:0/1553643239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee00caf50 con 0x5630130a85b0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.701+0000 7f9ed17fa640 1 -- 172.21.15.60:0/1553643239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00cfa40 con 0x5630130a85b0 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- 172.21.15.60:0/1553643239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 msgr2=0x5630130cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ec0 tx=0x7f9ee010a550 comp rx=0 tx=0).stop 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- 172.21.15.60:0/1553643239 shutdown_connections 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130b4540 0x5630130ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130a85b0 0x5630130cc180 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 --2- 172.21.15.60:0/1553643239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- 172.21.15.60:0/1553643239 >> 172.21.15.60:0/1553643239 conn(0x5630130a5ac0 msgr2=0x5630130b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- 172.21.15.60:0/1553643239 shutdown_connections 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- 172.21.15.60:0/1553643239 wait complete. 2024-07-24T21:54:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 -- start start 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.702+0000 7f9ef4596740 1 Processor -- start 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a85b0 0x5630130b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1ac810 con 0x5630130b4540 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1adc80 con 0x5630130a85b0 2024-07-24T21:54:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1acee0 con 0x5630130a7c90 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ef4596740 10 client.0 _ll_register_callbacks cb 0x7f9ee4012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44888/0 (socket says 172.21.15.60:44888) 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/4013800426 learned_addr learned my addr 172.21.15.60:0/4013800426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eea7fc640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a85b0 0x5630130b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 msgr2=0x5630130bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130bccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/4013800426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a85b0 msgr2=0x5630130b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a85b0 0x5630130b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 -- 172.21.15.60:0/4013800426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630130a71a0 con 0x5630130b4540 2024-07-24T21:54:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9eeb7fe640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009600 tx=0x7f9ee4029670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d348713f250000b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.703+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4003d10 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8609+0+0 (secure 0 0 0) 0x7f9ee401f7d0 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee00caf50 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ee4040050 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 10 client.17106 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56301312eb80 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee00caf50 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f9ee4038020 con 0x5630130b4540 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 10 client.17106 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 20 client.17106 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56301312f3b0 0x5630131317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 4 client.17106 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:54:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56301301ac70 con 0x56301312f3b0 2024-07-24T21:54:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.704+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56301312f3b0 0x5630131317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.705+0000 7f9eeaffd640 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56301312f3b0 0x5630131317d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919c81b82da10817 server_cookie=38f1bbdfa904ac76 in_seq=0 out_seq=0 2024-07-24T21:54:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.705+0000 7f9ecaffd640 10 client.17106 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ecaffd640 1 -- 172.21.15.60:0/4013800426 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56301301ac70 con 0x56301312f3b0 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ecaffd640 10 client.17106 handle_command_reply: tid=0 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 client.17106 shutdown 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 msgr2=0x5630130b8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009600 tx=0x7f9ee4029670 comp rx=0 tx=0).stop 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 shutdown_connections 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56301312f3b0 0x5630131317d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630130b4540 0x5630130b8860 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630130a85b0 0x5630130b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ecaffd640 0 client.17106 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 --2- 172.21.15.60:0/4013800426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630130a7c90 0x5630130bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 >> 172.21.15.60:0/4013800426 conn(0x5630130a5ac0 msgr2=0x5630130a4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 shutdown_connections 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 1 -- 172.21.15.60:0/4013800426 wait complete. 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.725+0000 7f9ef4596740 20 client.17106 trim_cache size 0 max 16384 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:54:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:54:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:54:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 msgr2=0x7f9ed40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 0x7f9ed40b6200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005fd0 tx=0x7f9ee4009500 comp rx=0 tx=0).stop 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 msgr2=0x7f9eec199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ab0 tx=0x7f9ee0003c80 comp rx=0 tx=0).stop 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 shutdown_connections 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9ed40b3d40 0x7f9ed40b6200 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec108a20 0x7f9eec19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9eec1080a0 0x7f9eec199140 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 --2- 172.21.15.60:0/98850915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eec1026a0 0x7f9eec198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 >> 172.21.15.60:0/98850915 conn(0x7f9eec0fe0e0 msgr2=0x7f9eec1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 shutdown_connections 2024-07-24T21:54:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:27.726+0000 7f9ef1841640 1 -- 172.21.15.60:0/98850915 wait complete. 2024-07-24T21:54:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:27 smithi123 ceph-mon[37143]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 235 KiB/s wr, 59 op/s 2024-07-24T21:54:27.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:54:27.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:27 smithi060 ceph-mon[33000]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 235 KiB/s wr, 59 op/s 2024-07-24T21:54:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:27 smithi086 ceph-mon[37128]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 235 KiB/s wr, 59 op/s 2024-07-24T21:54:28.029 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:29.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:29 smithi060 ceph-mon[33000]: osdmap e92: 12 total, 12 up, 12 in 2024-07-24T21:54:29.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:29 smithi060 ceph-mon[33000]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-07-24T21:54:29.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:29 smithi123 ceph-mon[37143]: osdmap e92: 12 total, 12 up, 12 in 2024-07-24T21:54:29.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:29 smithi123 ceph-mon[37143]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-07-24T21:54:29.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:29 smithi086 ceph-mon[37128]: osdmap e92: 12 total, 12 up, 12 in 2024-07-24T21:54:29.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:29 smithi086 ceph-mon[37128]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-07-24T21:54:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 -- 172.21.15.60:0/2511495544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed41036a0 msgr2=0x7f4ed4103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 --2- 172.21.15.60:0/2511495544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed41036a0 0x7f4ed4103a80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009900 tx=0x7f4ec802a410 comp rx=0 tx=0).stop 2024-07-24T21:54:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 -- 172.21.15.60:0/2511495544 shutdown_connections 2024-07-24T21:54:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 --2- 172.21.15.60:0/2511495544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 --2- 172.21.15.60:0/2511495544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed4103fc0 0x7f4ed410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 --2- 172.21.15.60:0/2511495544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed41036a0 0x7f4ed4103a80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 -- 172.21.15.60:0/2511495544 >> 172.21.15.60:0/2511495544 conn(0x7f4ed40fd100 msgr2=0x7f4ed40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 -- 172.21.15.60:0/2511495544 shutdown_connections 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.691+0000 7f4edab89640 1 -- 172.21.15.60:0/2511495544 wait complete. 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 Processor -- start 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 -- start start 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 0x7f4ed419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed41a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41127d0 con 0x7f4ed4103fc0 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4112650 con 0x7f4ed41036a0 2024-07-24T21:54:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4edab89640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4112950 con 0x7f4ed410cb00 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4ed3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4ed3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44918/0 (socket says 172.21.15.60:44918) 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4ed3fff640 1 -- 172.21.15.60:0/3974430038 learned_addr learned my addr 172.21.15.60:0/3974430038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.692+0000 7f4ed8925640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 0x7f4ed419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed41a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 -- 172.21.15.60:0/3974430038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 msgr2=0x7f4ed41a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed41a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 -- 172.21.15.60:0/3974430038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 msgr2=0x7f4ed419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 0x7f4ed419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 -- 172.21.15.60:0/3974430038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41a0f70 con 0x7f4ed4103fc0 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed8925640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 0x7f4ed419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b3c0 tx=0x7f4ec000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed1ffb640 1 -- 172.21.15.60:0/3974430038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011830 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed1ffb640 1 -- 172.21.15.60:0/3974430038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec0016070 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41a1260 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4ed1ffb640 1 -- 172.21.15.60:0/3974430038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0010de0 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.693+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a79a0 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.695+0000 7f4ed1ffb640 1 -- 172.21.15.60:0/3974430038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4ec00119d0 con 0x7f4ed4103fc0 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 Processor -- start 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 -- start start 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56062caf0990 con 0x56062cb796b0 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56062cadfe90 con 0x56062cb85f60 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56062cae3ed0 con 0x56062cb79fd0 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4ed9126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4ed9126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44920/0 (socket says 172.21.15.60:44920) 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4ed9126640 1 -- 172.21.15.60:0/3832733130 learned_addr learned my addr 172.21.15.60:0/3832733130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.696+0000 7f4ed8925640 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 -- 172.21.15.60:0/3832733130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 msgr2=0x56062cb85a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb85a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 -- 172.21.15.60:0/3832733130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 msgr2=0x56062cb88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed1ffb640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 0x7f4ec40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 -- 172.21.15.60:0/3832733130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56062ca69300 con 0x56062cb796b0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed1ffb640 1 -- 172.21.15.60:0/3974430038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4ec00e36f0 con 0x7f4ed4103fc0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 0x7f4ec40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc008e60 tx=0x7f4ecc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704eb57a3d9f9eb5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ebeffd640 1 -- 172.21.15.60:0/3832733130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01f070 con 0x56062cb796b0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ebeffd640 1 -- 172.21.15.60:0/3832733130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ecc00e3b0 con 0x56062cb796b0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.697+0000 7f4ebeffd640 1 -- 172.21.15.60:0/3832733130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00b600 con 0x56062cb796b0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- 172.21.15.60:0/3832733130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 msgr2=0x56062cb79a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4ed9126640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 0x7f4ec40b6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc008fd0 tx=0x7f4ecc01f000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc008e60 tx=0x7f4ecc00d710 comp rx=0 tx=0).stop 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- 172.21.15.60:0/3832733130 shutdown_connections 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/3832733130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb79a90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- 172.21.15.60:0/3832733130 >> 172.21.15.60:0/3832733130 conn(0x56062cb774e0 msgr2=0x56062cb78680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- 172.21.15.60:0/3832733130 shutdown_connections 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- 172.21.15.60:0/3832733130 wait complete. 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 -- start start 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.698+0000 7f4edcf3a740 1 Processor -- start 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb796b0 0x56062cb82ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56062c943700 con 0x56062cb85f60 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56062ca825d0 con 0x56062cb796b0 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56062cb88be0 con 0x56062cb79fd0 2024-07-24T21:54:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4edcf3a740 10 client.0 _ll_register_callbacks cb 0x56062caa2940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44934/0 (socket says 172.21.15.60:44934) 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 -- 172.21.15.60:0/675195069 learned_addr learned my addr 172.21.15.60:0/675195069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed8925640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb796b0 0x56062cb82ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 -- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 msgr2=0x56062cb833e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb833e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 -- 172.21.15.60:0/675195069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb796b0 msgr2=0x56062cb82ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb796b0 0x56062cb82ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.699+0000 7f4ed9126640 1 -- 172.21.15.60:0/675195069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56062ca75660 con 0x56062cb85f60 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4ed9126640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fa00 tx=0x7f4ecc00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8c0999df665b7d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00ee90 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4ecc00e560 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc00e3b0 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc00e760 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4edcf3a740 10 client.17118 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56062c840650 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ecc00e3b0 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f4ecc005500 con 0x56062cb85f60 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4edcf3a740 10 client.17118 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.700+0000 7f4edcf3a740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4edcf3a740 20 client.17118 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cb958a0 0x56062cb97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4edcf3a740 4 client.17118 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56062caf28e0 con 0x56062cb958a0 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4ed8925640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cb958a0 0x56062cb97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4ed8925640 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cb958a0 0x56062cb97c90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ec761a04cd2a5e server_cookie=80171218d4f220c in_seq=0 out_seq=0 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.701+0000 7f4eb3fff640 10 client.17118 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4eb3fff640 1 -- 172.21.15.60:0/675195069 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56062caf28e0 con 0x56062cb958a0 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4eb3fff640 10 client.17118 handle_command_reply: tid=0 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 client.17118 shutdown 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 msgr2=0x56062cb83920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fa00 tx=0x7f4ecc00fa30 comp rx=0 tx=0).stop 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 shutdown_connections 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cb958a0 0x56062cb97c90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb85f60 0x56062cb83920 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb79fd0 0x56062cb833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/675195069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb796b0 0x56062cb82ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4eb3fff640 0 client.17118 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 >> 172.21.15.60:0/675195069 conn(0x56062cb774e0 msgr2=0x56062cb7d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 shutdown_connections 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 1 -- 172.21.15.60:0/675195069 wait complete. 2024-07-24T21:54:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.702+0000 7f4edcf3a740 20 client.17118 trim_cache size 0 max 16384 2024-07-24T21:54:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 Processor -- start 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 -- start start 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb79fd0 0x56062cb7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 0x56062cb9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41b2e70 con 0x56062cb796b0 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41b06b0 con 0x56062cb79fd0 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41ae520 con 0x56062cb85f60 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44940/0 (socket says 172.21.15.60:44940) 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 -- 172.21.15.60:0/1934104028 learned_addr learned my addr 172.21.15.60:0/1934104028 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed9126640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 0x56062cb9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 -- 172.21.15.60:0/1934104028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 msgr2=0x56062cb9d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb79fd0 0x56062cb7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 0x56062cb9d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 -- 172.21.15.60:0/1934104028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb79fd0 msgr2=0x56062cb7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb79fd0 0x56062cb7d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.721+0000 7f4ed8925640 1 -- 172.21.15.60:0/1934104028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56062cb7ea70 con 0x56062cb796b0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.722+0000 7f4ed9126640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 0x56062cb9d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.722+0000 7f4ed8925640 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8032fd0 tx=0x7f4ec80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4fe0f3193776dc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.722+0000 7f4ebe7fc640 1 -- 172.21.15.60:0/1934104028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8049070 con 0x56062cb796b0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.722+0000 7f4ebe7fc640 1 -- 172.21.15.60:0/1934104028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec8005020 con 0x56062cb796b0 2024-07-24T21:54:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.722+0000 7f4ebe7fc640 1 -- 172.21.15.60:0/1934104028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8035040 con 0x56062cb796b0 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1934104028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 msgr2=0x56062cb84fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8032fd0 tx=0x7f4ec80058b0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1934104028 shutdown_connections 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb85f60 0x56062cb9d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb79fd0 0x56062cb7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1934104028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb796b0 0x56062cb84fa0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1934104028 >> 172.21.15.60:0/1934104028 conn(0x56062cb774e0 msgr2=0x56062cb83630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1934104028 shutdown_connections 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1934104028 wait complete. 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.723+0000 7f4edcf3a740 1 -- start start 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 Processor -- start 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 0x56062cb8e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb802e0 0x56062cb80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41aeb70 con 0x56062cb79fd0 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41affe0 con 0x56062cb85f60 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41af240 con 0x56062cb802e0 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4edcf3a740 10 client.0 _ll_register_callbacks cb 0x7f4ecc01d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4ed8925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 0x56062cb8e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4ed3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4ed8925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 0x56062cb8e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44956/0 (socket says 172.21.15.60:44956) 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4ed8925640 1 -- 172.21.15.60:0/1414624550 learned_addr learned my addr 172.21.15.60:0/1414624550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.724+0000 7f4ed9126640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb802e0 0x56062cb80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 -- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb802e0 msgr2=0x56062cb80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb802e0 0x56062cb80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 -- 172.21.15.60:0/1414624550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 msgr2=0x56062cb8e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 0x56062cb8e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 -- 172.21.15.60:0/1414624550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56062cbff620 con 0x56062cb85f60 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb7fd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00e0ec0 tx=0x7f4ec00105b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2fc32c16b1a2b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4eb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00a9a50 con 0x56062cb85f60 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4ec00e1030 con 0x56062cb85f60 2024-07-24T21:54:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec8005020 con 0x56062cb85f60 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ec00e1230 con 0x56062cb85f60 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4edcf3a740 10 client.26546 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56062cc008e0 con 0x56062cb85f60 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.725+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec8005020 con 0x56062cb85f60 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f4ec00e3ae0 con 0x56062cb85f60 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 10 client.26546 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 20 client.26546 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cc01090 0x56062cc034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 4 client.26546 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56062caee380 con 0x56062cc01090 2024-07-24T21:54:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4ed8925640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cc01090 0x56062cc034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4ed8925640 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cc01090 0x56062cc034b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74dde7e1d5cf6d73 server_cookie=66964f8792270524 in_seq=0 out_seq=0 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.726+0000 7f4eb3fff640 10 client.26546 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4eb3fff640 1 -- 172.21.15.60:0/1414624550 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56062caee380 con 0x56062cc01090 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4eb3fff640 10 client.26546 handle_command_reply: tid=0 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 client.26546 shutdown 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 msgr2=0x56062cb7fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb7fd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00e0ec0 tx=0x7f4ec00105b0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 shutdown_connections 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56062cc01090 0x56062cc034b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56062cb802e0 0x56062cb80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56062cb85f60 0x56062cb7fd00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 --2- 172.21.15.60:0/1414624550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56062cb79fd0 0x56062cb8e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 >> 172.21.15.60:0/1414624550 conn(0x56062cb774e0 msgr2=0x56062cb75d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4eb3fff640 0 client.26546 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 shutdown_connections 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 1 -- 172.21.15.60:0/1414624550 wait complete. 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.727+0000 7f4edcf3a740 20 client.26546 trim_cache size 0 max 16384 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:54:29.734 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 msgr2=0x7f4ec40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 0x7f4ec40b6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc008fd0 tx=0x7f4ecc01f000 comp rx=0 tx=0).stop 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 msgr2=0x7f4ed419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b3c0 tx=0x7f4ec000b880 comp rx=0 tx=0).stop 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 shutdown_connections 2024-07-24T21:54:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4ec40b3d40 0x7f4ec40b6200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ed410cb00 0x7f4ed41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ed4103fc0 0x7f4ed419b400 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 --2- 172.21.15.60:0/3974430038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed41036a0 0x7f4ed419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 >> 172.21.15.60:0/3974430038 conn(0x7f4ed40fd100 msgr2=0x7f4ed410ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 shutdown_connections 2024-07-24T21:54:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:29.728+0000 7f4edab89640 1 -- 172.21.15.60:0/3974430038 wait complete. 2024-07-24T21:54:30.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:54:30.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:54:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:31.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:31 smithi060 ceph-mon[33000]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-07-24T21:54:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:31 smithi123 ceph-mon[37143]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-07-24T21:54:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:31 smithi086 ceph-mon[37128]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-07-24T21:54:32.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 -- 172.21.15.60:0/1702237405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c10cb00 msgr2=0x7f309c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 --2- 172.21.15.60:0/1702237405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c10cb00 0x7f309c10ef60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3094009ca0 tx=0x7f309402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 -- 172.21.15.60:0/1702237405 shutdown_connections 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 --2- 172.21.15.60:0/1702237405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c10cb00 0x7f309c10ef60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 --2- 172.21.15.60:0/1702237405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 0x7f309c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 --2- 172.21.15.60:0/1702237405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c1036a0 0x7f309c103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 -- 172.21.15.60:0/1702237405 >> 172.21.15.60:0/1702237405 conn(0x7f309c0fd100 msgr2=0x7f309c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 -- 172.21.15.60:0/1702237405 shutdown_connections 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.464+0000 7f30a0928640 1 -- 172.21.15.60:0/1702237405 wait complete. 2024-07-24T21:54:32.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 Processor -- start 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 -- start start 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 0x7f309c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 0x7f309c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f309c112760 con 0x7f309c1036a0 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1125e0 con 0x7f309c10cb00 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.465+0000 7f30a0928640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1128e0 con 0x7f309c103fc0 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44968/0 (socket says 172.21.15.60:44968) 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 -- 172.21.15.60:0/218736123 learned_addr learned my addr 172.21.15.60:0/218736123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f3091d9b640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 0x7f309c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309ad9d640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 0x7f309c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:32.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 -- 172.21.15.60:0/218736123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 msgr2=0x7f309c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 0x7f309c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 -- 172.21.15.60:0/218736123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 msgr2=0x7f309c19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 0x7f309c19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 -- 172.21.15.60:0/218736123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c19ed60 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309ad9d640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 0x7f309c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f309a59c640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00cd80 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c19f050 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308c0108e0 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c018980 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.466+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c1a5710 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.468+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309c103a80 con 0x7f309c1036a0 2024-07-24T21:54:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.469+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f308c010430 con 0x7f309c1036a0 2024-07-24T21:54:32.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.470+0000 7f3093fff640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 0x7f30880b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:32.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.470+0000 7f3091d9b640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 0x7f30880b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:32.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.470+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f308c0e48c0 con 0x7f309c1036a0 2024-07-24T21:54:32.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.470+0000 7f3091d9b640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 0x7f30880b6200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f309c0ff9a0 tx=0x7f3084005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:32.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.473+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f308c0abba0 con 0x7f309c1036a0 2024-07-24T21:54:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:32.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.722+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f309c02e000 con 0x7f309c1036a0 2024-07-24T21:54:32.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.722+0000 7f3093fff640 1 -- 172.21.15.60:0/218736123 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7f308c0b2e20 con 0x7f309c1036a0 2024-07-24T21:54:32.724 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:54:32.724 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"btime":"2024-07-24T21:54:06:393878+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:54:06.223693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:54:32.725 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-07-24T21:54:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 msgr2=0x7f30880b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 0x7f30880b6200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f309c0ff9a0 tx=0x7f3084005f90 comp rx=0 tx=0).stop 2024-07-24T21:54:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 msgr2=0x7f309c198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 shutdown_connections 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30880b3d40 0x7f30880b6200 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309c10cb00 0x7f309c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f309c103fc0 0x7f309c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 --2- 172.21.15.60:0/218736123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309c1036a0 0x7f309c198cb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.724+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 >> 172.21.15.60:0/218736123 conn(0x7f309c0fd100 msgr2=0x7f309c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.725+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 shutdown_connections 2024-07-24T21:54:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:32.725+0000 7f30a0928640 1 -- 172.21.15.60:0/218736123 wait complete. 2024-07-24T21:54:33.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:33 smithi060 ceph-mon[33000]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 69 KiB/s wr, 28 op/s 2024-07-24T21:54:33.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:33 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/218736123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:33 smithi123 ceph-mon[37143]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 69 KiB/s wr, 28 op/s 2024-07-24T21:54:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:33 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/218736123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:33 smithi086 ceph-mon[37128]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 69 KiB/s wr, 28 op/s 2024-07-24T21:54:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:33 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/218736123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:33.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:54:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 -- 172.21.15.60:0/1303939816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 msgr2=0x7fe1281074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 --2- 172.21.15.60:0/1303939816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe1281074a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe118009900 tx=0x7fe11802a410 comp rx=0 tx=0).stop 2024-07-24T21:54:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 -- 172.21.15.60:0/1303939816 shutdown_connections 2024-07-24T21:54:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 --2- 172.21.15.60:0/1303939816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 --2- 172.21.15.60:0/1303939816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe1281074a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 --2- 172.21.15.60:0/1303939816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 0x7fe128101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 -- 172.21.15.60:0/1303939816 >> 172.21.15.60:0/1303939816 conn(0x7fe1280fd0c0 msgr2=0x7fe1280ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 -- 172.21.15.60:0/1303939816 shutdown_connections 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.848+0000 7fe130642640 1 -- 172.21.15.60:0/1303939816 wait complete. 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 Processor -- start 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 -- start start 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 0x7fe128199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe128111660 con 0x7fe128107060 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281114e0 con 0x7fe128101650 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe130642640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281117e0 con 0x7fe1281079e0 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe12dbdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe12dbdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44984/0 (socket says 172.21.15.60:44984) 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.849+0000 7fe12dbdd640 1 -- 172.21.15.60:0/3274475284 learned_addr learned my addr 172.21.15.60:0/3274475284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12e3de640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 0x7fe128199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12ebdf640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 -- 172.21.15.60:0/3274475284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 msgr2=0x7fe12819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 -- 172.21.15.60:0/3274475284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 msgr2=0x7fe128199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 0x7fe128199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 -- 172.21.15.60:0/3274475284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1281a4f10 con 0x7fe128107060 2024-07-24T21:54:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12ebdf640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe12dbdd640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe118006010 tx=0x7fe1180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11803f030 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1281a51a0 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118003800 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118035070 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.850+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1281a5680 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.851+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe128101a30 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.853+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe118003cb0 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.853+0000 7fe11f7fe640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 0x7fe1140b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.854+0000 7fe12e3de640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 0x7fe1140b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.854+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fe118104af0 con 0x7fe128107060 2024-07-24T21:54:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.854+0000 7fe12e3de640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 0x7fe1140b6200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe1240058c0 tx=0x7fe12400a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:34.857+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe1180cbdd0 con 0x7fe128107060 2024-07-24T21:54:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.097+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe12802e000 con 0x7fe128107060 2024-07-24T21:54:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.098+0000 7fe11f7fe640 1 -- 172.21.15.60:0/3274475284 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7fe1180cb6d0 con 0x7fe128107060 2024-07-24T21:54:35.100 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:54:35.100 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"btime":"2024-07-24T21:54:06:393878+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:54:06.223693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:54:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 msgr2=0x7fe1140b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 0x7fe1140b6200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe1240058c0 tx=0x7fe12400a4c0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 msgr2=0x7fe12819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe118006010 tx=0x7fe1180042f0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 shutdown_connections 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe1140b3d40 0x7fe1140b6200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1281079e0 0x7fe12819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128107060 0x7fe12819a070 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 --2- 172.21.15.60:0/3274475284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128101650 0x7fe128199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 >> 172.21.15.60:0/3274475284 conn(0x7fe1280fd0c0 msgr2=0x7fe1280ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 shutdown_connections 2024-07-24T21:54:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:35.100+0000 7fe130642640 1 -- 172.21.15.60:0/3274475284 wait complete. 2024-07-24T21:54:35.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:35 smithi060 ceph-mon[33000]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 70 KiB/s wr, 25 op/s 2024-07-24T21:54:35.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3274475284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:35 smithi123 ceph-mon[37143]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 70 KiB/s wr, 25 op/s 2024-07-24T21:54:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3274475284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:35 smithi086 ceph-mon[37128]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 70 KiB/s wr, 25 op/s 2024-07-24T21:54:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3274475284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:54:36.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id cbe533b0 --release --await' 2024-07-24T21:54:36.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id cbe533b0 --release --await 2024-07-24T21:54:36.774 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:37.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 -- 172.21.15.60:0/649103209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 msgr2=0x7fecc4102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 --2- 172.21.15.60:0/649103209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4102b60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009900 tx=0x7fecb402a410 comp rx=0 tx=0).stop 2024-07-24T21:54:37.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 -- 172.21.15.60:0/649103209 shutdown_connections 2024-07-24T21:54:37.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 --2- 172.21.15.60:0/649103209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 0x7fecc410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 --2- 172.21.15.60:0/649103209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 0x7fecc41085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 --2- 172.21.15.60:0/649103209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4102b60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 -- 172.21.15.60:0/649103209 >> 172.21.15.60:0/649103209 conn(0x7fecc40fe1d0 msgr2=0x7fecc41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 -- 172.21.15.60:0/649103209 shutdown_connections 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.446+0000 7fecc90e9640 1 -- 172.21.15.60:0/649103209 wait complete. 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 Processor -- start 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 -- start start 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 0x7fecc40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 0x7fecc4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4112720 con 0x7fecc4102780 2024-07-24T21:54:37.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41125a0 con 0x7fecc4108190 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.447+0000 7fecc90e9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41128a0 con 0x7fecc4108b10 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60876/0 (socket says 172.21.15.60:60876) 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3476066656 learned_addr learned my addr 172.21.15.60:0/3476066656 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc359e640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 0x7fecc4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc259c640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 0x7fecc40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3476066656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 msgr2=0x7fecc4075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 0x7fecc4075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3476066656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 msgr2=0x7fecc40753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 0x7fecc40753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3476066656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc41a7070 con 0x7fecc4102780 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005940 tx=0x7fecb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecabfff640 1 -- 172.21.15.60:0/3476066656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb403f030 con 0x7fecc4102780 2024-07-24T21:54:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc41a7300 con 0x7fecc4102780 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecabfff640 1 -- 172.21.15.60:0/3476066656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4003cb0 con 0x7fecc4102780 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecabfff640 1 -- 172.21.15.60:0/3476066656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4035070 con 0x7fecc4102780 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.448+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc41a77e0 con 0x7fecc4102780 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7fecabfff640 1 -- 172.21.15.60:0/3476066656 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fecb4042030 con 0x7fecc4102780 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 Processor -- start 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 -- start start 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1b99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561bb1b019b0 con 0x561bb1b8aff0 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bb1af0eb0 con 0x561bb1b8a6d0 2024-07-24T21:54:37.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7feccb49a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561bb1af4ef0 con 0x561bb1b96f80 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60884/0 (socket says 172.21.15.60:60884) 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3902841973 learned_addr learned my addr 172.21.15.60:0/3902841973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.451+0000 7fecc359e640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecabfff640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 0x7fecac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc259c640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1b99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3902841973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 msgr2=0x561bb1b99370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1b99370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3902841973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 msgr2=0x561bb1b8aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 -- 172.21.15.60:0/3902841973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb1a7a320 con 0x561bb1b8aff0 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc359e640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:37.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc259c640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1b99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc359e640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 0x7fecac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecabfff640 1 -- 172.21.15.60:0/3476066656 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fecb4104850 con 0x7fecc4102780 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fecb4101ee0 tx=0x7fecb4094900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623b2708aa16df8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7feca97fa640 1 -- 172.21.15.60:0/3902841973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40cae30 con 0x561bb1b8aff0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7feca97fa640 1 -- 172.21.15.60:0/3902841973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40caff0 con 0x561bb1b8aff0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.452+0000 7feca97fa640 1 -- 172.21.15.60:0/3902841973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4005d50 con 0x561bb1b8aff0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- 172.21.15.60:0/3902841973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 msgr2=0x561bb1b96a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7fecc359e640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 0x7fecac0b6200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fecc4076f50 tx=0x7fecbc004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fecb4101ee0 tx=0x7fecb4094900 comp rx=0 tx=0).stop 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- 172.21.15.60:0/3902841973 shutdown_connections 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1b99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b96a40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 --2- 172.21.15.60:0/3902841973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- 172.21.15.60:0/3902841973 >> 172.21.15.60:0/3902841973 conn(0x561bb1b88500 msgr2=0x561bb1b896a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- 172.21.15.60:0/3902841973 shutdown_connections 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- 172.21.15.60:0/3902841973 wait complete. 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 -- start start 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.453+0000 7feccb49a740 1 Processor -- start 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8aff0 0x561bb1b94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b96f80 0x561bb1b949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561bb1954700 con 0x561bb1b8a6d0 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bb1a935f0 con 0x561bb1b96f80 2024-07-24T21:54:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561bb1b999c0 con 0x561bb1b8aff0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feccb49a740 10 client.0 _ll_register_callbacks cb 0x561bb1ab3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60888/0 (socket says 172.21.15.60:60888) 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 -- 172.21.15.60:0/2471505311 learned_addr learned my addr 172.21.15.60:0/2471505311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc259c640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8aff0 0x561bb1b94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc359e640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b96f80 0x561bb1b949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 -- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8aff0 msgr2=0x561bb1b94460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8aff0 0x561bb1b94460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 -- 172.21.15.60:0/2471505311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b96f80 msgr2=0x561bb1b949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b96f80 0x561bb1b949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 -- 172.21.15.60:0/2471505311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb1a86680 con 0x561bb1b8a6d0 2024-07-24T21:54:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fecb4004650 tx=0x7fecb4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd5c577c3d94602 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.454+0000 7feca6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4049040 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fecb4096b00 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb40caff0 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fecb40d3e40 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 10 client.17148 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bb1851650 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40caff0 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fecb40c6e50 con 0x561bb1b8a6d0 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 10 client.17148 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 10 client.17148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 20 client.17148 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1ba6530 0x561bb1ba8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 4 client.17148 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.455+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bb1b03900 con 0x561bb1ba6530 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.456+0000 7fecc259c640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1ba6530 0x561bb1ba8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.456+0000 7fecc259c640 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1ba6530 0x561bb1ba8920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9e87ad914144fcb server_cookie=ff24edfb2678550d in_seq=0 out_seq=0 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.456+0000 7feca6ff5640 10 client.17148 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feca6ff5640 1 -- 172.21.15.60:0/2471505311 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bb1b03900 con 0x561bb1ba6530 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feca6ff5640 10 client.17148 handle_command_reply: tid=0 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 client.17148 shutdown 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 msgr2=0x561bb1b93f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fecb4004650 tx=0x7fecb4005b50 comp rx=0 tx=0).stop 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 shutdown_connections 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1ba6530 0x561bb1ba8920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b96f80 0x561bb1b949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8aff0 0x561bb1b94460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feca6ff5640 0 client.17148 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 --2- 172.21.15.60:0/2471505311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8a6d0 0x561bb1b93f20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 >> 172.21.15.60:0/2471505311 conn(0x561bb1b88500 msgr2=0x561bb1b8e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 shutdown_connections 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 1 -- 172.21.15.60:0/2471505311 wait complete. 2024-07-24T21:54:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.457+0000 7feccb49a740 20 client.17148 trim_cache size 0 max 16384 2024-07-24T21:54:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 Processor -- start 2024-07-24T21:54:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 -- start start 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1bae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41b2d00 con 0x561bb1b8aff0 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41b0540 con 0x561bb1b8a6d0 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.471+0000 7feccb49a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ae3b0 con 0x561bb1b96f80 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60890/0 (socket says 172.21.15.60:60890) 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 -- 172.21.15.60:0/1716839226 learned_addr learned my addr 172.21.15.60:0/1716839226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc359e640 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1bae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 -- 172.21.15.60:0/1716839226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 msgr2=0x561bb1bae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1bae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 -- 172.21.15.60:0/1716839226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 msgr2=0x561bb1b8e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 -- 172.21.15.60:0/1716839226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb1b94dc0 con 0x561bb1b8aff0 2024-07-24T21:54:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7fecc259c640 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x561bb1ba90a0 tx=0x7fecb800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60afd8656e91e79d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7feca8ff9640 1 -- 172.21.15.60:0/1716839226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8003d60 con 0x561bb1b8aff0 2024-07-24T21:54:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7feca8ff9640 1 -- 172.21.15.60:0/1716839226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb8003f00 con 0x561bb1b8aff0 2024-07-24T21:54:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.472+0000 7feca8ff9640 1 -- 172.21.15.60:0/1716839226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8012a00 con 0x561bb1b8aff0 2024-07-24T21:54:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- 172.21.15.60:0/1716839226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 msgr2=0x561bb1b95550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x561bb1ba90a0 tx=0x7fecb800b390 comp rx=0 tx=0).stop 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- 172.21.15.60:0/1716839226 shutdown_connections 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b96f80 0x561bb1bae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b8aff0 0x561bb1b95550 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 --2- 172.21.15.60:0/1716839226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8a6d0 0x561bb1b8e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- 172.21.15.60:0/1716839226 >> 172.21.15.60:0/1716839226 conn(0x561bb1b88500 msgr2=0x561bb1b946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- 172.21.15.60:0/1716839226 shutdown_connections 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- 172.21.15.60:0/1716839226 wait complete. 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 -- start start 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.473+0000 7feccb49a740 1 Processor -- start 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8a6d0 0x561bb1b9f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8aff0 0x561bb1b9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41aea00 con 0x561bb1b96f80 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41afe70 con 0x561bb1b8aff0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41af0d0 con 0x561bb1b8a6d0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feccb49a740 10 client.0 _ll_register_callbacks cb 0x7fecb40c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60896/0 (socket says 172.21.15.60:60896) 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 -- 172.21.15.60:0/969228905 learned_addr learned my addr 172.21.15.60:0/969228905 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc259c640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8aff0 0x561bb1b9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8a6d0 0x561bb1b9f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 -- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8a6d0 msgr2=0x561bb1b9f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8a6d0 0x561bb1b9f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 -- 172.21.15.60:0/969228905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8aff0 msgr2=0x561bb1b9a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8aff0 0x561bb1b9a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 -- 172.21.15.60:0/969228905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb1c10100 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7fecc359e640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fecbc013ab0 tx=0x7fecbc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24cf845ad4ebccb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.474+0000 7feca6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc003b80 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb8003f00 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fecbc003d20 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fecbc020960 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 10 client.17154 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bb1c11460 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb8003f00 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fecbc031070 con 0x561bb1b96f80 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 10 client.17154 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:54:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 20 client.17154 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1c11ca0 0x561bb1c140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 4 client.17154 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.475+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bb1aff3a0 con 0x561bb1c11ca0 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1c11ca0 0x561bb1c140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feca6ff5640 10 client.17154 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7fecc2d9d640 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1c11ca0 0x561bb1c140c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1df65f88bc1b18fd server_cookie=51034bde1ac7e1de in_seq=0 out_seq=0 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feca6ff5640 1 -- 172.21.15.60:0/969228905 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bb1aff3a0 con 0x561bb1c11ca0 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feca6ff5640 10 client.17154 handle_command_reply: tid=0 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 client.17154 shutdown 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 msgr2=0x561bb1b9b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fecbc013ab0 tx=0x7fecbc004ab0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 shutdown_connections 2024-07-24T21:54:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x561bb1c11ca0 0x561bb1c140c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bb1b96f80 0x561bb1b9b140 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bb1b8aff0 0x561bb1b9a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feca6ff5640 0 client.17154 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 --2- 172.21.15.60:0/969228905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561bb1b8a6d0 0x561bb1b9f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 >> 172.21.15.60:0/969228905 conn(0x561bb1b88500 msgr2=0x561bb1b86d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 shutdown_connections 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.476+0000 7feccb49a740 1 -- 172.21.15.60:0/969228905 wait complete. 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7feccb49a740 20 client.17154 trim_cache size 0 max 16384 2024-07-24T21:54:37.483 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 msgr2=0x7fecac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 0x7fecac0b6200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fecc4076f50 tx=0x7fecbc004a40 comp rx=0 tx=0).stop 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 msgr2=0x7fecc4074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005940 tx=0x7fecb40042f0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 shutdown_connections 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fecac0b3d40 0x7fecac0b6200 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecc4108b10 0x7fecc4075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecc4108190 0x7fecc40753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 --2- 172.21.15.60:0/3476066656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc4102780 0x7fecc4074e90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 >> 172.21.15.60:0/3476066656 conn(0x7fecc40fe1d0 msgr2=0x7fecc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 shutdown_connections 2024-07-24T21:54:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:37.477+0000 7fecc90e9640 1 -- 172.21.15.60:0/3476066656 wait complete. 2024-07-24T21:54:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:37 smithi123 ceph-mon[37143]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-24T21:54:37.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:37 smithi060 ceph-mon[33000]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-24T21:54:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:37 smithi086 ceph-mon[37128]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-24T21:54:38.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:54:39.310 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:39 smithi060 ceph-mon[33000]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-07-24T21:54:39.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:39 smithi123 ceph-mon[37143]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-07-24T21:54:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:39 smithi086 ceph-mon[37128]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-07-24T21:54:39.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 -- 172.21.15.60:0/1126918078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f200010cb00 msgr2=0x7f200010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 --2- 172.21.15.60:0/1126918078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f200010cb00 0x7f200010ef60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009ca0 tx=0x7f1ff802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 -- 172.21.15.60:0/1126918078 shutdown_connections 2024-07-24T21:54:39.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 --2- 172.21.15.60:0/1126918078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f200010cb00 0x7f200010ef60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 --2- 172.21.15.60:0/1126918078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 0x7f200010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 --2- 172.21.15.60:0/1126918078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20001036a0 0x7f2000103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 -- 172.21.15.60:0/1126918078 >> 172.21.15.60:0/1126918078 conn(0x7f20000fd100 msgr2=0x7f20000ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:39.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 -- 172.21.15.60:0/1126918078 shutdown_connections 2024-07-24T21:54:39.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.665+0000 7f2004d44640 1 -- 172.21.15.60:0/1126918078 wait complete. 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 Processor -- start 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 -- start start 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 0x7f20001991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200010cb00 0x7f200019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20001127e0 con 0x7f20001036a0 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2000112660 con 0x7f200010cb00 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f2004d44640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2000112960 con 0x7f2000103fc0 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f1ffe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f1ffe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60930/0 (socket says 172.21.15.60:60930) 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f1ffe59c640 1 -- 172.21.15.60:0/1940008697 learned_addr learned my addr 172.21.15.60:0/1940008697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.666+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200010cb00 0x7f200019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 0x7f20001991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 -- 172.21.15.60:0/1940008697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 msgr2=0x7f20001991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 0x7f20001991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 -- 172.21.15.60:0/1940008697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200010cb00 msgr2=0x7f200019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200010cb00 0x7f200019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 -- 172.21.15.60:0/1940008697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200019ed60 con 0x7f20001036a0 2024-07-24T21:54:39.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b4a0 tx=0x7f1ff000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ff77fe640 1 -- 172.21.15.60:0/1940008697 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0014070 con 0x7f20001036a0 2024-07-24T21:54:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200019f050 con 0x7f20001036a0 2024-07-24T21:54:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ff77fe640 1 -- 172.21.15.60:0/1940008697 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff0010ac0 con 0x7f20001036a0 2024-07-24T21:54:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f1ff77fe640 1 -- 172.21.15.60:0/1940008697 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0017c30 con 0x7f20001036a0 2024-07-24T21:54:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.667+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a58e0 con 0x7f20001036a0 2024-07-24T21:54:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f20070f5740 1 Processor -- start 2024-07-24T21:54:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f1ff77fe640 1 -- 172.21.15.60:0/1940008697 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1ff0017070 con 0x7f20001036a0 2024-07-24T21:54:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f20070f5740 1 -- start start 2024-07-24T21:54:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f20070f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f20070f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bd31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.670+0000 7f20070f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f20070f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562c93b39dd0 con 0x562c93bd36f0 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f20070f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c93b292d0 con 0x562c93bc3350 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f20070f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c93b2d310 con 0x562c93bc2a30 2024-07-24T21:54:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ffe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bd31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60944/0 (socket says 172.21.15.60:60944) 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 -- 172.21.15.60:0/1149493146 learned_addr learned my addr 172.21.15.60:0/1149493146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 -- 172.21.15.60:0/1149493146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 msgr2=0x562c93bc2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bc2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 -- 172.21.15.60:0/1149493146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 msgr2=0x562c93bd31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bd31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff77fe640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 0x7f1fec0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 -- 172.21.15.60:0/1149493146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c93889b00 con 0x562c93bd36f0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bd31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 0x7f1fec0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff77fe640 1 -- 172.21.15.60:0/1940008697 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f1ff00e3640 con 0x7f20001036a0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800cba0 tx=0x7f1fe800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4ac052e2097516 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.671+0000 7f1ff4ff9640 1 -- 172.21.15.60:0/1149493146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8010430 con 0x562c93bd36f0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f1ff4ff9640 1 -- 172.21.15.60:0/1149493146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe8016070 con 0x562c93bd36f0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f1ff4ff9640 1 -- 172.21.15.60:0/1149493146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800f950 con 0x562c93bd36f0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 -- 172.21.15.60:0/1149493146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 msgr2=0x562c93bd5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800cba0 tx=0x7f1fe800e350 comp rx=0 tx=0).stop 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 0x7f1fec0b6200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80098d0 tx=0x7f1ff8009a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 -- 172.21.15.60:0/1149493146 shutdown_connections 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bd5ae0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bd31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 --2- 172.21.15.60:0/1149493146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bc2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 -- 172.21.15.60:0/1149493146 >> 172.21.15.60:0/1149493146 conn(0x562c93bc0860 msgr2=0x562c93bc1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 -- 172.21.15.60:0/1149493146 shutdown_connections 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.672+0000 7f20070f5740 1 -- 172.21.15.60:0/1149493146 wait complete. 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 -- start start 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 Processor -- start 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bd3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bd36f0 0x562c93bd0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562c9398cbb0 con 0x562c93bc3350 2024-07-24T21:54:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c93acb900 con 0x562c93bd36f0 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c93bd63a0 con 0x562c93bc2a30 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f20070f5740 10 client.0 _ll_register_callbacks cb 0x562c93ad5c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f1ff7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60948/0 (socket says 172.21.15.60:60948) 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f1ff7fff640 1 -- 172.21.15.60:0/2552728953 learned_addr learned my addr 172.21.15.60:0/2552728953 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bd3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.673+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bd36f0 0x562c93bd0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 -- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 msgr2=0x562c93bd3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bd3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 -- 172.21.15.60:0/2552728953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bd36f0 msgr2=0x562c93bd0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bd36f0 0x562c93bd0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 -- 172.21.15.60:0/2552728953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c93900960 con 0x562c93bc3350 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bd3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8005640 tx=0x7f1fe8005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c9aeb3cacb51b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800c740 con 0x562c93bc3350 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f1fe8004030 con 0x562c93bc3350 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe8016070 con 0x562c93bc3350 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1fe8004270 con 0x562c93bc3350 2024-07-24T21:54:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f20070f5740 10 client.17166 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562c93be1e80 con 0x562c93bc3350 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.674+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe8016070 con 0x562c93bc3350 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f1fe8004880 con 0x562c93bc3350 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 10 client.17166 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 10 client.17166 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 20 client.17166 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93be5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 4 client.17166 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c93b3bd20 con 0x562c93be3280 2024-07-24T21:54:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93be5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f1fe5ffb640 10 client.17166 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.675+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93be5670 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a0bfeb00af7517 server_cookie=1c04150eaff521ba in_seq=0 out_seq=0 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2552728953 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c93b3bd20 con 0x562c93be3280 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f1fe5ffb640 10 client.17166 handle_command_reply: tid=0 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 client.17166 shutdown 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 msgr2=0x562c93bd05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8005640 tx=0x7f1fe8005670 comp rx=0 tx=0).stop 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 shutdown_connections 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93be5670 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bd36f0 0x562c93bd0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc3350 0x562c93bd05f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f1fe5ffb640 0 client.17166 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 --2- 172.21.15.60:0/2552728953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93bd3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 >> 172.21.15.60:0/2552728953 conn(0x562c93bc0860 msgr2=0x562c93bc6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 shutdown_connections 2024-07-24T21:54:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 1 -- 172.21.15.60:0/2552728953 wait complete. 2024-07-24T21:54:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.676+0000 7f20070f5740 20 client.17166 trim_cache size 0 max 16384 2024-07-24T21:54:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 Processor -- start 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 -- start start 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bc0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 0x562c93bcee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20001ae930 con 0x562c93bc2a30 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20001ac170 con 0x562c93bc3350 2024-07-24T21:54:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f20070f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20001a9fe0 con 0x562c93bd36f0 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60952/0 (socket says 172.21.15.60:60952) 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 -- 172.21.15.60:0/3691347402 learned_addr learned my addr 172.21.15.60:0/3691347402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bc0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 -- 172.21.15.60:0/3691347402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 msgr2=0x562c93bcee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 0x562c93bcee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 0x562c93bcee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 -- 172.21.15.60:0/3691347402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 msgr2=0x562c93bc0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bc0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffe59c640 1 -- 172.21.15.60:0/3691347402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c93bbc190 con 0x562c93bc2a30 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bc0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.698+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 0x562c93bcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.699+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009420 tx=0x7f1ff00e9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98daa71812e146b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.699+0000 7f1fe7fff640 1 -- 172.21.15.60:0/3691347402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001e040 con 0x562c93bc2a30 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.699+0000 7f1fe7fff640 1 -- 172.21.15.60:0/3691347402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff0009230 con 0x562c93bc2a30 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.699+0000 7f1fe7fff640 1 -- 172.21.15.60:0/3691347402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0010c70 con 0x562c93bc2a30 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- 172.21.15.60:0/3691347402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 msgr2=0x562c93bce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009420 tx=0x7f1ff00e9550 comp rx=0 tx=0).stop 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- 172.21.15.60:0/3691347402 shutdown_connections 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bd36f0 0x562c93bcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bc0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 --2- 172.21.15.60:0/3691347402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bc2a30 0x562c93bce180 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- 172.21.15.60:0/3691347402 >> 172.21.15.60:0/3691347402 conn(0x562c93bc0860 msgr2=0x562c93bd4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- 172.21.15.60:0/3691347402 shutdown_connections 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- 172.21.15.60:0/3691347402 wait complete. 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 -- start start 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.700+0000 7f20070f5740 1 Processor -- start 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93beb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bdb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20001aa630 con 0x562c93bd36f0 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20001abaa0 con 0x562c93bc3350 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20001aad00 con 0x562c93bc2a30 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f20070f5740 10 client.0 _ll_register_callbacks cb 0x7f1fe800c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60964/0 (socket says 172.21.15.60:60964) 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 -- 172.21.15.60:0/2507943630 learned_addr learned my addr 172.21.15.60:0/2507943630 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93beb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ff7fff640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bdb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 -- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 msgr2=0x562c93beb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93beb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 -- 172.21.15.60:0/2507943630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 msgr2=0x562c93bdb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:39.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bdb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 -- 172.21.15.60:0/2507943630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c93bdc160 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1ffed9d640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8035000 tx=0x7f1ff802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb99b9c4881e12 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.701+0000 7f1fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8031e90 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f1ff8042030 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff0009230 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1ff8004250 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 10 client.17172 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562c93c4de50 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff8036050 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f1ff802fd10 con 0x562c93bd36f0 2024-07-24T21:54:39.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 10 client.17172 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 10 client.17172 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 20 client.17172 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93c50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 4 client.17172 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cbe533b0", "release": true, "await": true}] 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.702+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cbe533b0", "release": true, "await": true}) -- 0x7f1ff0009230 con 0x562c93be3280 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.703+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93c50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.703+0000 7f1fe5ffb640 10 client.17172 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:39.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:39.703+0000 7f1ffe59c640 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93c50610 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b64c8523dbcfa9 server_cookie=b0dc0795377e32ac in_seq=0 out_seq=0 2024-07-24T21:54:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/3694790827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c18108a30 msgr2=0x7f4c1810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/3694790827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c18108a30 0x7f4c1810f230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009be0 tx=0x7f4c1002a6c0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/3694790827 shutdown_connections 2024-07-24T21:54:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/3694790827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c18108a30 0x7f4c1810f230 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/3694790827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c181080b0 0x7f4c181084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/3694790827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181026b0 0x7f4c18102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/3694790827 >> 172.21.15.60:0/3694790827 conn(0x7f4c180fe0f0 msgr2=0x7f4c18100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/3694790827 shutdown_connections 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.823+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/3694790827 wait complete. 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 Processor -- start 2024-07-24T21:54:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 -- start start 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 0x7f4c18199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c18108a30 0x7f4c1819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18112660 con 0x7f4c181026b0 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181124e0 con 0x7f4c181080b0 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c1d2e9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181127e0 con 0x7f4c18108a30 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60974/0 (socket says 172.21.15.60:60974) 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c16ffd640 1 -- 172.21.15.60:0/550114419 learned_addr learned my addr 172.21.15.60:0/550114419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c177fe640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c18108a30 0x7f4c1819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.824+0000 7f4c167fc640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 0x7f4c18199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 -- 172.21.15.60:0/550114419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c18108a30 msgr2=0x7f4c1819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c18108a30 0x7f4c1819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 -- 172.21.15.60:0/550114419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 msgr2=0x7f4c18199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 0x7f4c18199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 -- 172.21.15.60:0/550114419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1819ec80 con 0x7f4c181026b0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c167fc640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 0x7f4c18199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ea60 tx=0x7f4c0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4bfffff640 1 -- 172.21.15.60:0/550114419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0800cde0 con 0x7f4c181026b0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4bfffff640 1 -- 172.21.15.60:0/550114419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c080035f0 con 0x7f4c181026b0 2024-07-24T21:54:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1819ef70 con 0x7f4c181026b0 2024-07-24T21:54:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4bfffff640 1 -- 172.21.15.60:0/550114419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08010690 con 0x7f4c181026b0 2024-07-24T21:54:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.825+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c181a5630 con 0x7f4c181026b0 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4bfffff640 1 -- 172.21.15.60:0/550114419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4c080037a0 con 0x7f4c181026b0 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 Processor -- start 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 -- start start 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf598c0 0x5607daf59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf66170 0x5607daf68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5607daecf280 con 0x5607daf5a1e0 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5607daebe780 con 0x5607daf598c0 2024-07-24T21:54:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c20043740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607daec27c0 con 0x5607daf66170 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60976/0 (socket says 172.21.15.60:60976) 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c16ffd640 1 -- 172.21.15.60:0/534593251 learned_addr learned my addr 172.21.15.60:0/534593251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.828+0000 7f4c167fc640 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf66170 0x5607daf68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c177fe640 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf598c0 0x5607daf59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 -- 172.21.15.60:0/534593251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf66170 msgr2=0x5607daf68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf66170 0x5607daf68560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 -- 172.21.15.60:0/534593251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf598c0 msgr2=0x5607daf59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf598c0 0x5607daf59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 -- 172.21.15.60:0/534593251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607dac22790 con 0x5607daf5a1e0 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4c080e1a40 tx=0x7f4c080ec880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d101b6e6cb1a7aa7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4bfffff640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 0x7f4c000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4bfd7fa640 1 -- 172.21.15.60:0/534593251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080f8070 con 0x5607daf5a1e0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4c177fe640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 0x7f4c000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4bfd7fa640 1 -- 172.21.15.60:0/534593251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0806f6d0 con 0x5607daf5a1e0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4bfd7fa640 1 -- 172.21.15.60:0/534593251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080943b0 con 0x5607daf5a1e0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.829+0000 7f4bfffff640 1 -- 172.21.15.60:0/550114419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4c080e43b0 con 0x7f4c181026b0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c177fe640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 0x7f4c000b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009370 tx=0x7f4c10004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- 172.21.15.60:0/534593251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 msgr2=0x5607daf65c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4c080e1a40 tx=0x7f4c080ec880 comp rx=0 tx=0).stop 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- 172.21.15.60:0/534593251 shutdown_connections 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf66170 0x5607daf68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf5a1e0 0x5607daf65c30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 --2- 172.21.15.60:0/534593251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf598c0 0x5607daf59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- 172.21.15.60:0/534593251 >> 172.21.15.60:0/534593251 conn(0x5607daf576f0 msgr2=0x5607daf58890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- 172.21.15.60:0/534593251 shutdown_connections 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- 172.21.15.60:0/534593251 wait complete. 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 -- start start 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.830+0000 7f4c20043740 1 Processor -- start 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf5a1e0 0x5607daf62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 0x5607daf632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5607dadf8ab0 con 0x5607daf598c0 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5607dae469e0 con 0x5607daf66170 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5607daf68e50 con 0x5607daf5a1e0 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c20043740 10 client.0 _ll_register_callbacks cb 0x5607dabcea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c167fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf5a1e0 0x5607daf62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60982/0 (socket says 172.21.15.60:60982) 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 -- 172.21.15.60:0/282422744 learned_addr learned my addr 172.21.15.60:0/282422744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c177fe640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 0x5607daf632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 -- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf5a1e0 msgr2=0x5607daf62d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf5a1e0 0x5607daf62d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 -- 172.21.15.60:0/282422744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 msgr2=0x5607daf632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 0x5607daf632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 -- 172.21.15.60:0/282422744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607dac9b620 con 0x5607daf598c0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c177fe640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 0x5607daf632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ea30 tx=0x7f4c080aa800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e620ee6adf892879 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4bfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.831+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08095440 con 0x5607daf598c0 2024-07-24T21:54:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4c080a7580 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607dae6dce0 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4c08095630 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 10 client.17184 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5607daf745e0 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5607dae6dce0 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f4c080e1d50 con 0x5607daf598c0 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 10 client.17184 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 20 client.17184 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607daf75930 0x5607daf77d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 4 client.17184 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607daed11d0 con 0x5607daf75930 2024-07-24T21:54:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.832+0000 7f4c167fc640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607daf75930 0x5607daf77d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.833+0000 7f4bfaff5640 10 client.17184 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.833+0000 7f4c167fc640 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607daf75930 0x5607daf77d20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb48417cbc9fa3a server_cookie=2fbace84e3ae75b4 in_seq=0 out_seq=0 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4bfaff5640 1 -- 172.21.15.60:0/282422744 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607daed11d0 con 0x5607daf75930 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4bfaff5640 10 client.17184 handle_command_reply: tid=0 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 client.17184 shutdown 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 msgr2=0x5607daf65890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ea30 tx=0x7f4c080aa800 comp rx=0 tx=0).stop 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 shutdown_connections 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607daf75930 0x5607daf77d20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf66170 0x5607daf632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf5a1e0 0x5607daf62d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 --2- 172.21.15.60:0/282422744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf598c0 0x5607daf65890 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4bfaff5640 0 client.17184 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 >> 172.21.15.60:0/282422744 conn(0x5607daf576f0 msgr2=0x5607daf5d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 shutdown_connections 2024-07-24T21:54:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 1 -- 172.21.15.60:0/282422744 wait complete. 2024-07-24T21:54:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.834+0000 7f4c20043740 20 client.17184 trim_cache size 0 max 16384 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 Processor -- start 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 -- start start 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf6e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181b0b40 con 0x5607daf66170 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181ae380 con 0x5607daf5a1e0 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.849+0000 7f4c20043740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181ac1f0 con 0x5607daf598c0 2024-07-24T21:54:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60990/0 (socket says 172.21.15.60:60990) 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 -- 172.21.15.60:0/4169754951 learned_addr learned my addr 172.21.15.60:0/4169754951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c167fc640 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf6e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 -- 172.21.15.60:0/4169754951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 msgr2=0x5607daf71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 -- 172.21.15.60:0/4169754951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 msgr2=0x5607daf6e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf6e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 -- 172.21.15.60:0/4169754951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607daf53210 con 0x5607daf66170 2024-07-24T21:54:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4c10033000 tx=0x7f4c1003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a28fe33aa16852 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4bfcff9640 1 -- 172.21.15.60:0/4169754951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1004f070 con 0x5607daf66170 2024-07-24T21:54:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4bfcff9640 1 -- 172.21.15.60:0/4169754951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c10030c30 con 0x5607daf66170 2024-07-24T21:54:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.850+0000 7f4bfcff9640 1 -- 172.21.15.60:0/4169754951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10036040 con 0x5607daf66170 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- 172.21.15.60:0/4169754951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 msgr2=0x5607daf6ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4c10033000 tx=0x7f4c1003f000 comp rx=0 tx=0).stop 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- 172.21.15.60:0/4169754951 shutdown_connections 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607daf6ea30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf6e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 --2- 172.21.15.60:0/4169754951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- 172.21.15.60:0/4169754951 >> 172.21.15.60:0/4169754951 conn(0x5607daf576f0 msgr2=0x5607daf58740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- 172.21.15.60:0/4169754951 shutdown_connections 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- 172.21.15.60:0/4169754951 wait complete. 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 -- start start 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.851+0000 7f4c20043740 1 Processor -- start 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607dafe22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181ac840 con 0x5607daf66170 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181adcb0 con 0x5607daf5a1e0 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181acf10 con 0x5607daf598c0 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 10 client.0 _ll_register_callbacks cb 0x7f4c080f6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607dafe22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c16ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf5f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34268/0 (socket says 172.21.15.60:34268) 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c16ffd640 1 -- 172.21.15.60:0/4108203754 learned_addr learned my addr 172.21.15.60:0/4108203754 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c167fc640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 -- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 msgr2=0x5607daf5f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf5f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 -- 172.21.15.60:0/4108203754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 msgr2=0x5607daf5f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf5f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 -- 172.21.15.60:0/4108203754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607daf58090 con 0x5607daf66170 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c177fe640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607dafe22c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4c1002a700 tx=0x7f4c10030690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f0c3028366601e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4bfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1005c070 con 0x5607daf66170 2024-07-24T21:54:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f4c10004200 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.852+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c10030c30 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4c10065050 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 10 client.17190 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5607dafe3790 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c10030c30 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f4c1005f020 con 0x5607daf66170 2024-07-24T21:54:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 10 client.17190 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 20 client.17190 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607dafe3eb0 0x5607dafe62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 4 client.17190 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607daeccc70 con 0x5607dafe3eb0 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607dafe3eb0 0x5607dafe62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4bfaff5640 10 client.17190 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.853+0000 7f4c16ffd640 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607dafe3eb0 0x5607dafe62d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17084f00e8d4b291 server_cookie=3df5b0002215e480 in_seq=0 out_seq=0 2024-07-24T21:54:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4bfaff5640 1 -- 172.21.15.60:0/4108203754 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607daeccc70 con 0x5607dafe3eb0 2024-07-24T21:54:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4bfaff5640 10 client.17190 handle_command_reply: tid=0 2024-07-24T21:54:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 client.17190 shutdown 2024-07-24T21:54:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 msgr2=0x5607dafe22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607dafe22c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4c1002a700 tx=0x7f4c10030690 comp rx=0 tx=0).stop 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 shutdown_connections 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5607dafe3eb0 0x5607dafe62d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4bfaff5640 0 client.17190 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607daf66170 0x5607dafe22c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607daf5a1e0 0x5607daf5f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 --2- 172.21.15.60:0/4108203754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5607daf598c0 0x5607daf5f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 >> 172.21.15.60:0/4108203754 conn(0x5607daf576f0 msgr2=0x5607daf61460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 shutdown_connections 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 1 -- 172.21.15.60:0/4108203754 wait complete. 2024-07-24T21:54:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.864+0000 7f4c20043740 20 client.17190 trim_cache size 0 max 16384 2024-07-24T21:54:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:54:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:54:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:54:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 msgr2=0x7f4c000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 0x7f4c000b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009370 tx=0x7f4c10004a40 comp rx=0 tx=0).stop 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 msgr2=0x7f4c18198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ea60 tx=0x7f4c0800ef20 comp rx=0 tx=0).stop 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 shutdown_connections 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c000b3d40 0x7f4c000b6200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c18108a30 0x7f4c1819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c181080b0 0x7f4c18199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 --2- 172.21.15.60:0/550114419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c181026b0 0x7f4c18198be0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 >> 172.21.15.60:0/550114419 conn(0x7f4c180fe0f0 msgr2=0x7f4c180fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 shutdown_connections 2024-07-24T21:54:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:40.865+0000 7f4c1d2e9640 1 -- 172.21.15.60:0/550114419 wait complete. 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f1fe5ffb640 1 -- 172.21.15.60:0/2507943630 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f1ff0009230 con 0x562c93be3280 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f1fe5ffb640 10 client.17172 handle_command_reply: tid=0 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 client.17172 shutdown 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 msgr2=0x562c93bdbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8035000 tx=0x7f1ff802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 shutdown_connections 2024-07-24T21:54:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562c93be3280 0x562c93c50610 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c93bd36f0 0x562c93bdbb70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c93bc3350 0x562c93bdb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 --2- 172.21.15.60:0/2507943630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c93bc2a30 0x562c93beb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 >> 172.21.15.60:0/2507943630 conn(0x562c93bc0860 msgr2=0x562c93bc18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f1fe5ffb640 0 client.17172 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 shutdown_connections 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 1 -- 172.21.15.60:0/2507943630 wait complete. 2024-07-24T21:54:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.899+0000 7f20070f5740 20 client.17172 trim_cache size 0 max 16384 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 39, 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 24, 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "cbe533b0": { 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 24, 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:54:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 150.0, 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:54:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:40.906 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 msgr2=0x7f1fec0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 0x7f1fec0b6200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80098d0 tx=0x7f1ff8009a00 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 msgr2=0x7f2000198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b4a0 tx=0x7f1ff000b960 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 shutdown_connections 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1fec0b3d40 0x7f1fec0b6200 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200010cb00 0x7f200019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2000103fc0 0x7f20001991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 --2- 172.21.15.60:0/1940008697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20001036a0 0x7f2000198cb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 >> 172.21.15.60:0/1940008697 conn(0x7f20000fd100 msgr2=0x7f20000fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:40.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 shutdown_connections 2024-07-24T21:54:40.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:40.900+0000 7f2004d44640 1 -- 172.21.15.60:0/1940008697 wait complete. 2024-07-24T21:54:41.541 INFO:tasks.workunit.client.0.smithi060.stdout: 2024-07-24T21:54:41.541 INFO:tasks.workunit.client.0.smithi060.stdout:Final score for writes: 518 2024-07-24T21:54:41.541 INFO:tasks.workunit.client.0.smithi060.stdout:Final score for reads : 629346 2024-07-24T21:54:41.541 INFO:tasks.workunit.client.0.smithi060.stdout: 2024-07-24T21:54:41.542 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-24T21:54:41.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-24T21:54:41.548 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:41 smithi060 ceph-mon[33000]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-07-24T21:54:41.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 -- 172.21.15.60:0/1331032789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36741036a0 msgr2=0x7f3674103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 --2- 172.21.15.60:0/1331032789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36741036a0 0x7f3674103a80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3670009900 tx=0x7f367002a410 comp rx=0 tx=0).stop 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 -- 172.21.15.60:0/1331032789 shutdown_connections 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 --2- 172.21.15.60:0/1331032789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 0x7f367410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 --2- 172.21.15.60:0/1331032789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3674103fc0 0x7f367410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 --2- 172.21.15.60:0/1331032789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36741036a0 0x7f3674103a80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 -- 172.21.15.60:0/1331032789 >> 172.21.15.60:0/1331032789 conn(0x7f36740fd100 msgr2=0x7f36740ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 -- 172.21.15.60:0/1331032789 shutdown_connections 2024-07-24T21:54:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.638+0000 7f367b3a4640 1 -- 172.21.15.60:0/1331032789 wait complete. 2024-07-24T21:54:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 Processor -- start 2024-07-24T21:54:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 -- start start 2024-07-24T21:54:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 0x7f3674101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 0x7f3674101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3674112760 con 0x7f3674103fc0 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36741125e0 con 0x7f36741036a0 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367b3a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36741128e0 con 0x7f367410cb00 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f3679140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 0x7f3674101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f3679140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46842/0 (socket says 172.21.15.60:46842) 2024-07-24T21:54:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f3679941640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 0x7f3674101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f3679140640 1 -- 172.21.15.60:0/3926556728 learned_addr learned my addr 172.21.15.60:0/3926556728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 -- 172.21.15.60:0/3926556728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 msgr2=0x7f3674101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 0x7f3674101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 -- 172.21.15.60:0/3926556728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 msgr2=0x7f3674101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.639+0000 7f367893f640 1 -- 172.21.15.60:0/3926556728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741a70a0 con 0x7f3674103fc0 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f3679140640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f367893f640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 0x7f3674101890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f366400e770 tx=0x7f366400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f366a7fc640 1 -- 172.21.15.60:0/3926556728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366400ccc0 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36741a7390 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f366a7fc640 1 -- 172.21.15.60:0/3926556728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f366400ee80 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f366a7fc640 1 -- 172.21.15.60:0/3926556728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664003e40 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.640+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36741a7920 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 Processor -- start 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f366a7fc640 1 -- 172.21.15.60:0/3926556728 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3664003fe0 con 0x7f3674103fc0 2024-07-24T21:54:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 -- start start 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5585d4c5b9b0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585d4c4aeb0 con 0x5585d4ce46d0 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367d755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585d4c4eef0 con 0x5585d4cf0f80 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679140640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46858/0 (socket says 172.21.15.60:46858) 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f367893f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679941640 1 -- 172.21.15.60:0/3983037913 learned_addr learned my addr 172.21.15.60:0/3983037913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679140640 1 -- 172.21.15.60:0/3983037913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 msgr2=0x5585d4cf3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679140640 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.643+0000 7f3679140640 1 -- 172.21.15.60:0/3983037913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 msgr2=0x5585d4ce4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f3679140640 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f3679140640 1 -- 172.21.15.60:0/3983037913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d4bd4320 con 0x5585d4ce4ff0 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f3679941640 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f366a7fc640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 0x7f36600b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f3679941640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 0x7f36600b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f3679140640 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf0a40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3670004770 tx=0x7f3670032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f9c221688b0371 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f366a7fc640 1 -- 172.21.15.60:0/3926556728 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f36640e4280 con 0x7f3674103fc0 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f365f7fe640 1 -- 172.21.15.60:0/3983037913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670035040 con 0x5585d4ce4ff0 2024-07-24T21:54:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f365f7fe640 1 -- 172.21.15.60:0/3983037913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36700317a0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.644+0000 7f365f7fe640 1 -- 172.21.15.60:0/3983037913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36700047a0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- 172.21.15.60:0/3983037913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 msgr2=0x5585d4cf0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f3679941640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 0x7f36600b6200 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f366c009fd0 tx=0x7f366c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf0a40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3670004770 tx=0x7f3670032ed0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- 172.21.15.60:0/3983037913 shutdown_connections 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf0a40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 --2- 172.21.15.60:0/3983037913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4ce4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- 172.21.15.60:0/3983037913 >> 172.21.15.60:0/3983037913 conn(0x5585d4ce2500 msgr2=0x5585d4ce36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- 172.21.15.60:0/3983037913 shutdown_connections 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- 172.21.15.60:0/3983037913 wait complete. 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.645+0000 7f367d755740 1 -- start start 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 Processor -- start 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce46d0 0x5585d4cedea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 0x5585d4cee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5585d4aae700 con 0x5585d4ce4ff0 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585d4bed5f0 con 0x5585d4cf0f80 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585d4cf3ba0 con 0x5585d4ce46d0 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367d755740 10 client.0 _ll_register_callbacks cb 0x5585d4c0d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367893f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367893f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32808/0 (socket says 172.21.15.60:32808) 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367893f640 1 -- 172.21.15.60:0/2792978060 learned_addr learned my addr 172.21.15.60:0/2792978060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:41.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f3679140640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce46d0 0x5585d4cedea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f3679941640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 0x5585d4cee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367893f640 1 -- 172.21.15.60:0/2792978060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce46d0 msgr2=0x5585d4cedea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.646+0000 7f367893f640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce46d0 0x5585d4cedea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367893f640 1 -- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 msgr2=0x5585d4cee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367893f640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 0x5585d4cee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367893f640 1 -- 172.21.15.60:0/2792978060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d4be0680 con 0x5585d4ce4ff0 2024-07-24T21:54:41.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f3679941640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 0x5585d4cee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367893f640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f36640e1a00 tx=0x7f36640e79c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300342040bb9d5b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36640aa7b0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f36640aa2d0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36700317a0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f366400ce60 con 0x5585d4ce4ff0 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 10 client.17202 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:41.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5585d49ab650 con 0x5585d4ce4ff0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36700317a0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f36640a9b30 con 0x5585d4ce4ff0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 10 client.17202 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 10 client.17202 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.647+0000 7f367d755740 20 client.17202 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d009f0 0x5585d4d02de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f367d755740 4 client.17202 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585d4c5d900 con 0x5585d4d009f0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f3679140640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d009f0 0x5585d4d02de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f3679140640 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d009f0 0x5585d4d02de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec7d53427dd8dd70 server_cookie=df5d8b2def463ae7 in_seq=0 out_seq=0 2024-07-24T21:54:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.648+0000 7f365cff9640 10 client.17202 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f365cff9640 1 -- 172.21.15.60:0/2792978060 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585d4c5d900 con 0x5585d4d009f0 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f365cff9640 10 client.17202 handle_command_reply: tid=0 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 client.17202 shutdown 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 msgr2=0x5585d4cee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f36640e1a00 tx=0x7f36640e79c0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 shutdown_connections 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d009f0 0x5585d4d02de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4cf0f80 0x5585d4cee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cee3e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f365cff9640 0 client.17202 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 --2- 172.21.15.60:0/2792978060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce46d0 0x5585d4cedea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 >> 172.21.15.60:0/2792978060 conn(0x5585d4ce2500 msgr2=0x5585d4ce80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 shutdown_connections 2024-07-24T21:54:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 1 -- 172.21.15.60:0/2792978060 wait complete. 2024-07-24T21:54:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.650+0000 7f367d755740 20 client.17202 trim_cache size 0 max 16384 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 Processor -- start 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 -- start start 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cefbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 0x5585d4d086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36741b2d40 con 0x5585d4cf0f80 2024-07-24T21:54:41.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36741b0580 con 0x5585d4ce46d0 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367d755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36741ae3f0 con 0x5585d4ce4ff0 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f3679941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f3679941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32822/0 (socket says 172.21.15.60:32822) 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f3679941640 1 -- 172.21.15.60:0/1905972390 learned_addr learned my addr 172.21.15.60:0/1905972390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f367893f640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 0x5585d4d086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.665+0000 7f3679140640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cefbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 -- 172.21.15.60:0/1905972390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 msgr2=0x5585d4d086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 0x5585d4d086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 -- 172.21.15.60:0/1905972390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 msgr2=0x5585d4cefbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cefbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 -- 172.21.15.60:0/1905972390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d4ce4e20 con 0x5585d4cf0f80 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f367893f640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 0x5585d4d086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679140640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cefbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f3679941640 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f366c009ed0 tx=0x7f366c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a9964ba8f2272c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f365effd640 1 -- 172.21.15.60:0/1905972390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c01c410 con 0x5585d4cf0f80 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f365effd640 1 -- 172.21.15.60:0/1905972390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f366c02c070 con 0x5585d4cf0f80 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.666+0000 7f365effd640 1 -- 172.21.15.60:0/1905972390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c027970 con 0x5585d4cf0f80 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- 172.21.15.60:0/1905972390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 msgr2=0x5585d4d08bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f366c009ed0 tx=0x7f366c016000 comp rx=0 tx=0).stop 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- 172.21.15.60:0/1905972390 shutdown_connections 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4cf0f80 0x5585d4d08bf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4ce4ff0 0x5585d4d086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 --2- 172.21.15.60:0/1905972390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cefbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- 172.21.15.60:0/1905972390 >> 172.21.15.60:0/1905972390 conn(0x5585d4ce2500 msgr2=0x5585d4cef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- 172.21.15.60:0/1905972390 shutdown_connections 2024-07-24T21:54:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- 172.21.15.60:0/1905972390 wait complete. 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 -- start start 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.667+0000 7f367d755740 1 Processor -- start 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36741aea40 con 0x5585d4ce4ff0 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36741afeb0 con 0x5585d4ce46d0 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36741af110 con 0x5585d4cf0f80 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367d755740 10 client.0 _ll_register_callbacks cb 0x7f36640e7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f367893f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46896/0 (socket says 172.21.15.60:46896) 2024-07-24T21:54:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 -- 172.21.15.60:0/2589581547 learned_addr learned my addr 172.21.15.60:0/2589581547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679941640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 -- 172.21.15.60:0/2589581547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 msgr2=0x5585d4cf5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 -- 172.21.15.60:0/2589581547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 msgr2=0x5585d4cf52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.668+0000 7f3679140640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f3679140640 1 -- 172.21.15.60:0/2589581547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d4d6a7e0 con 0x5585d4ce46d0 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f367893f640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f3679941640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f3679140640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3670005ec0 tx=0x7f3670009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc39880b9942e9db server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f365cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670035040 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f366c02c070 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f3670031980 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3670060050 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f367d755740 10 client.26615 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5585d4d6bae0 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.669+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f366c02c070 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f3670003aa0 con 0x5585d4ce46d0 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 10 client.26615 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 10 client.26615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 20 client.26615 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d6bf30 0x5585d4d6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:41.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 4 client.26615 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585d4c593a0 con 0x5585d4d6bf30 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.670+0000 7f367893f640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d6bf30 0x5585d4d6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.671+0000 7f365cff9640 10 client.26615 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.671+0000 7f367893f640 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d6bf30 0x5585d4d6e320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=722310b7efb2ba82 server_cookie=ba0ceffa668d59f1 in_seq=0 out_seq=0 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f365cff9640 1 -- 172.21.15.60:0/2589581547 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585d4c593a0 con 0x5585d4d6bf30 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f365cff9640 10 client.26615 handle_command_reply: tid=0 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 client.26615 shutdown 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 msgr2=0x5585d4cf4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3670005ec0 tx=0x7f3670009260 comp rx=0 tx=0).stop 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 shutdown_connections 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5585d4d6bf30 0x5585d4d6e320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585d4cf0f80 0x5585d4cf5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f365cff9640 0 client.26615 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585d4ce4ff0 0x5585d4cf52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 --2- 172.21.15.60:0/2589581547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585d4ce46d0 0x5585d4cf4da0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 >> 172.21.15.60:0/2589581547 conn(0x5585d4ce2500 msgr2=0x5585d4cec2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 shutdown_connections 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 1 -- 172.21.15.60:0/2589581547 wait complete. 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.672+0000 7f367d755740 20 client.26615 trim_cache size 0 max 16384 2024-07-24T21:54:41.685 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 msgr2=0x7f36600b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 0x7f36600b6200 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f366c009fd0 tx=0x7f366c009500 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 msgr2=0x7f3674101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 0x7f3674101890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f366400e770 tx=0x7f366400ec30 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 shutdown_connections 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36600b3d40 0x7f36600b6200 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367410cb00 0x7f3674101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3674103fc0 0x7f3674101890 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 --2- 172.21.15.60:0/3926556728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36741036a0 0x7f3674101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 >> 172.21.15.60:0/3926556728 conn(0x7f36740fd100 msgr2=0x7f36740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:41.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 shutdown_connections 2024-07-24T21:54:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:41.673+0000 7f367b3a4640 1 -- 172.21.15.60:0/3926556728 wait complete. 2024-07-24T21:54:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:41 smithi123 ceph-mon[37143]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-07-24T21:54:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:41 smithi086 ceph-mon[37128]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-07-24T21:54:42.189 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cbe533b0', total seconds elapsed: 40.1 2024-07-24T21:54:42.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:54:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:43.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:43 smithi060 ceph-mon[33000]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-07-24T21:54:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:43 smithi123 ceph-mon[37143]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-07-24T21:54:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:43 smithi086 ceph-mon[37128]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.068+0000 7fd674ec6640 1 -- 172.21.15.60:0/1010045966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670101670 msgr2=0x7fd670101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.068+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1010045966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670101670 0x7fd670101a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd660009900 tx=0x7fd66002a400 comp rx=0 tx=0).stop 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.069+0000 7fd674ec6640 1 -- 172.21.15.60:0/1010045966 shutdown_connections 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.069+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1010045966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.069+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1010045966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670107080 0x7fd6701074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.069+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1010045966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670101670 0x7fd670101a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.069+0000 7fd674ec6640 1 -- 172.21.15.60:0/1010045966 >> 172.21.15.60:0/1010045966 conn(0x7fd6700fd0c0 msgr2=0x7fd6700ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.070+0000 7fd674ec6640 1 -- 172.21.15.60:0/1010045966 shutdown_connections 2024-07-24T21:54:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.070+0000 7fd674ec6640 1 -- 172.21.15.60:0/1010045966 wait complete. 2024-07-24T21:54:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 Processor -- start 2024-07-24T21:54:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 -- start start 2024-07-24T21:54:45.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 0x7fd670197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670107080 0x7fd670198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd670111690 con 0x7fd670101670 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd670111510 con 0x7fd670107080 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.072+0000 7fd674ec6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd670111810 con 0x7fd670107a00 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34340/0 (socket says 172.21.15.60:34340) 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 -- 172.21.15.60:0/1655228163 learned_addr learned my addr 172.21.15.60:0/1655228163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 0x7fd670197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670107080 0x7fd670198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 -- 172.21.15.60:0/1655228163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670107080 msgr2=0x7fd670198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670107080 0x7fd670198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 -- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 msgr2=0x7fd670197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 0x7fd670197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66ed9d640 1 -- 172.21.15.60:0/1655228163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67019dcd0 con 0x7fd670107a00 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.073+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 0x7fd670197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.074+0000 7fd66ed9d640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd66800db90 tx=0x7fd66800dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.074+0000 7fd65f7fe640 1 -- 172.21.15.60:0/1655228163 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66800e980 con 0x7fd670107a00 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.074+0000 7fd65f7fe640 1 -- 172.21.15.60:0/1655228163 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd668009580 con 0x7fd670107a00 2024-07-24T21:54:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.074+0000 7fd65f7fe640 1 -- 172.21.15.60:0/1655228163 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668016650 con 0x7fd670107a00 2024-07-24T21:54:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.076+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67019df60 con 0x7fd670107a00 2024-07-24T21:54:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.076+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67006adf0 con 0x7fd670107a00 2024-07-24T21:54:45.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.081+0000 7fd677277740 1 Processor -- start 2024-07-24T21:54:45.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.081+0000 7fd65f7fe640 1 -- 172.21.15.60:0/1655228163 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd668009730 con 0x7fd670107a00 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.081+0000 7fd677277740 1 -- start start 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b87ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 0x562b50b96370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562b50afe9b0 con 0x562b50b876d0 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b50aedeb0 con 0x562b50b93f80 2024-07-24T21:54:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.082+0000 7fd677277740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b50af1ef0 con 0x562b50b87ff0 2024-07-24T21:54:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.083+0000 7fd65f7fe640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 0x7fd6580b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.083+0000 7fd66e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.083+0000 7fd66e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:34354/0 (socket says 172.21.15.60:34354) 2024-07-24T21:54:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.083+0000 7fd66e59c640 1 -- 172.21.15.60:0/543204068 learned_addr learned my addr 172.21.15.60:0/543204068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.083+0000 7fd65f7fe640 1 -- 172.21.15.60:0/1655228163 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fd6680e38c0 con 0x7fd670107a00 2024-07-24T21:54:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 0x562b50b96370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 -- 172.21.15.60:0/543204068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 msgr2=0x562b50b96370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 0x562b50b96370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 -- 172.21.15.60:0/543204068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 msgr2=0x562b50b87ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b87ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 -- 172.21.15.60:0/543204068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b50a77320 con 0x562b50b87ff0 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 0x562b50b96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66e59c640 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd660009900 tx=0x7fd660032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cb4b228c39b6bf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.084+0000 7fd66ed9d640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 0x7fd6580b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.085+0000 7fd66ed9d640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 0x7fd6580b63d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b5b0 tx=0x7fd668009e10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.086+0000 7fd65cff9640 1 -- 172.21.15.60:0/543204068 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66003f070 con 0x562b50b87ff0 2024-07-24T21:54:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.086+0000 7fd65cff9640 1 -- 172.21.15.60:0/543204068 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6600316a0 con 0x562b50b87ff0 2024-07-24T21:54:45.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.087+0000 7fd65cff9640 1 -- 172.21.15.60:0/543204068 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66003f070 con 0x562b50b87ff0 2024-07-24T21:54:45.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.087+0000 7fd677277740 1 -- 172.21.15.60:0/543204068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 msgr2=0x562b50b93a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.087+0000 7fd677277740 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd660009900 tx=0x7fd660032ce0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.088+0000 7fd677277740 1 -- 172.21.15.60:0/543204068 shutdown_connections 2024-07-24T21:54:45.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.088+0000 7fd677277740 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b93f80 0x562b50b96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.088+0000 7fd677277740 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50b93a40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.088+0000 7fd677277740 1 --2- 172.21.15.60:0/543204068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b87ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.088+0000 7fd677277740 1 -- 172.21.15.60:0/543204068 >> 172.21.15.60:0/543204068 conn(0x562b50b85500 msgr2=0x562b50b866a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.089+0000 7fd677277740 1 -- 172.21.15.60:0/543204068 shutdown_connections 2024-07-24T21:54:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.089+0000 7fd677277740 1 -- 172.21.15.60:0/543204068 wait complete. 2024-07-24T21:54:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.089+0000 7fd677277740 1 -- start start 2024-07-24T21:54:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.089+0000 7fd677277740 1 Processor -- start 2024-07-24T21:54:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b93990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b90de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b91570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562b50951700 con 0x562b50b876d0 2024-07-24T21:54:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b50a905f0 con 0x562b50b87ff0 2024-07-24T21:54:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b50b96bf0 con 0x562b50b93f80 2024-07-24T21:54:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.090+0000 7fd677277740 10 client.0 _ll_register_callbacks cb 0x562b50ab0960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.091+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b93990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.091+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b93990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32868/0 (socket says 172.21.15.60:32868) 2024-07-24T21:54:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.091+0000 7fd66dd9b640 1 -- 172.21.15.60:0/1337753313 learned_addr learned my addr 172.21.15.60:0/1337753313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.091+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b90de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 -- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 msgr2=0x562b50b91570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b91570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 -- 172.21.15.60:0/1337753313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 msgr2=0x562b50b93990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b93990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 -- 172.21.15.60:0/1337753313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b50a83680 con 0x562b50b87ff0 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b90de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd660031440 tx=0x7fd660005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e3311597cbbda server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660050060 con 0x562b50b87ff0 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.092+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fd660003800 con 0x562b50b87ff0 2024-07-24T21:54:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.093+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6600316a0 con 0x562b50b87ff0 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.093+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd66003f070 con 0x562b50b87ff0 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.093+0000 7fd677277740 10 client.26624 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.093+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562b5084e650 con 0x562b50b87ff0 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.093+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6600316a0 con 0x562b50b87ff0 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fd660004cb0 con 0x562b50b87ff0 2024-07-24T21:54:45.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 10 client.26624 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 10 client.26624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 20 client.26624 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50babf90 0x562b50bae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 4 client.26624 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b50b00900 con 0x562b50babf90 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.094+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50babf90 0x562b50bae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.095+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50babf90 0x562b50bae380 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5039bd2abca8936 server_cookie=939f0bc68f025dd8 in_seq=0 out_seq=0 2024-07-24T21:54:45.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.095+0000 7fd655ffb640 10 client.26624 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:45.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.098+0000 7fd655ffb640 1 -- 172.21.15.60:0/1337753313 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b50b00900 con 0x562b50babf90 2024-07-24T21:54:45.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.098+0000 7fd655ffb640 10 client.26624 handle_command_reply: tid=0 2024-07-24T21:54:45.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.098+0000 7fd677277740 1 client.26624 shutdown 2024-07-24T21:54:45.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 msgr2=0x562b50b90de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b90de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd660031440 tx=0x7fd660005a20 comp rx=0 tx=0).stop 2024-07-24T21:54:45.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 shutdown_connections 2024-07-24T21:54:45.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50babf90 0x562b50bae380 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b91570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b90de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 --2- 172.21.15.60:0/1337753313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b93990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 >> 172.21.15.60:0/1337753313 conn(0x562b50b85500 msgr2=0x562b50b8b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd655ffb640 0 client.26624 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 shutdown_connections 2024-07-24T21:54:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 1 -- 172.21.15.60:0/1337753313 wait complete. 2024-07-24T21:54:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.099+0000 7fd677277740 20 client.26624 trim_cache size 0 max 16384 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.124+0000 7fd677277740 1 Processor -- start 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 -- start start 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b8dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701aea60 con 0x562b50b876d0 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ac2a0 con 0x562b50b87ff0 2024-07-24T21:54:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd677277740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701aa110 con 0x562b50b93f80 2024-07-24T21:54:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.125+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32872/0 (socket says 172.21.15.60:32872) 2024-07-24T21:54:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 -- 172.21.15.60:0/4016717897 learned_addr learned my addr 172.21.15.60:0/4016717897 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66e59c640 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b8dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 -- 172.21.15.60:0/4016717897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 msgr2=0x562b50b8e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b8e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 -- 172.21.15.60:0/4016717897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 msgr2=0x562b50b8dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b8dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 -- 172.21.15.60:0/4016717897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b50b8f2e0 con 0x562b50b876d0 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd6640129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a0871a5b8a5671 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd657fff640 1 -- 172.21.15.60:0/4016717897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664004110 con 0x562b50b876d0 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.126+0000 7fd657fff640 1 -- 172.21.15.60:0/4016717897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6640042b0 con 0x562b50b876d0 2024-07-24T21:54:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd657fff640 1 -- 172.21.15.60:0/4016717897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400f7f0 con 0x562b50b876d0 2024-07-24T21:54:45.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 -- 172.21.15.60:0/4016717897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 msgr2=0x562b50b868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd6640129e0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 -- 172.21.15.60:0/4016717897 shutdown_connections 2024-07-24T21:54:45.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b93f80 0x562b50b8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b87ff0 0x562b50b8dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 --2- 172.21.15.60:0/4016717897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b876d0 0x562b50b868f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.127+0000 7fd677277740 1 -- 172.21.15.60:0/4016717897 >> 172.21.15.60:0/4016717897 conn(0x562b50b85500 msgr2=0x562b50b861e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.128+0000 7fd677277740 1 -- 172.21.15.60:0/4016717897 shutdown_connections 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.128+0000 7fd677277740 1 -- 172.21.15.60:0/4016717897 wait complete. 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.128+0000 7fd677277740 1 -- start start 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.128+0000 7fd677277740 1 Processor -- start 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b876d0 0x562b50bacfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50bad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701aa760 con 0x562b50b93f80 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701abbd0 con 0x562b50b876d0 2024-07-24T21:54:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701aae30 con 0x562b50b87ff0 2024-07-24T21:54:45.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd677277740 10 client.0 _ll_register_callbacks cb 0x7fd660032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:45.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32874/0 (socket says 172.21.15.60:32874) 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 -- 172.21.15.60:0/1373436348 learned_addr learned my addr 172.21.15.60:0/1373436348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50bad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b876d0 0x562b50bacfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 -- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 msgr2=0x562b50bad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50bad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 -- 172.21.15.60:0/1373436348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b876d0 msgr2=0x562b50bacfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.129+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b876d0 0x562b50bacfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.130+0000 7fd66dd9b640 1 -- 172.21.15.60:0/1373436348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b50c15a70 con 0x562b50b93f80 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.130+0000 7fd66dd9b640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd664012240 tx=0x7fd664003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eabbfa540b87e0c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.130+0000 7fd655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.130+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66402f040 con 0x562b50b93f80 2024-07-24T21:54:45.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.130+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6640042b0 con 0x562b50b93f80 2024-07-24T21:54:45.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.131+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fd664003a00 con 0x562b50b93f80 2024-07-24T21:54:45.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.132+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd6640122a0 con 0x562b50b93f80 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.132+0000 7fd677277740 10 client.17223 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.132+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562b50c16d50 con 0x562b50b93f80 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6640042b0 con 0x562b50b93f80 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fd66401e440 con 0x562b50b93f80 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 10 client.17223 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 20 client.17223 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50c17430 0x562b50c19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 4 client.17223 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:54:45.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.133+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b50afc3a0 con 0x562b50c17430 2024-07-24T21:54:45.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.134+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50c17430 0x562b50c19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:45.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.134+0000 7fd655ffb640 10 client.17223 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:45.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.135+0000 7fd66e59c640 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50c17430 0x562b50c19850 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70eb0fa0ecf3a39c server_cookie=5aadd3ca29a13ba6 in_seq=0 out_seq=0 2024-07-24T21:54:45.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd655ffb640 1 -- 172.21.15.60:0/1373436348 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b50afc3a0 con 0x562b50c17430 2024-07-24T21:54:45.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd655ffb640 10 client.17223 handle_command_reply: tid=0 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 client.17223 shutdown 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 msgr2=0x562b50bada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd664012240 tx=0x7fd664003e70 comp rx=0 tx=0).stop 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 shutdown_connections 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562b50c17430 0x562b50c19850 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b50b93f80 0x562b50bada40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b50b87ff0 0x562b50bad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 --2- 172.21.15.60:0/1373436348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b50b876d0 0x562b50bacfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 >> 172.21.15.60:0/1373436348 conn(0x562b50b85500 msgr2=0x562b50b92410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.136+0000 7fd655ffb640 0 client.17223 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.137+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 shutdown_connections 2024-07-24T21:54:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.137+0000 7fd677277740 1 -- 172.21.15.60:0/1373436348 wait complete. 2024-07-24T21:54:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.137+0000 7fd677277740 20 client.17223 trim_cache size 0 max 16384 2024-07-24T21:54:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:54:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.138+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 msgr2=0x7fd6580b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.138+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 0x7fd6580b63d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b5b0 tx=0x7fd668009e10 comp rx=0 tx=0).stop 2024-07-24T21:54:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.138+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 msgr2=0x7fd67019d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.138+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd66800db90 tx=0x7fd66800dc80 comp rx=0 tx=0).stop 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 shutdown_connections 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd6580b3f10 0x7fd6580b63d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd670107a00 0x7fd67019d500 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd670107080 0x7fd670198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 --2- 172.21.15.60:0/1655228163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd670101670 0x7fd670197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.139+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 >> 172.21.15.60:0/1655228163 conn(0x7fd6700fd0c0 msgr2=0x7fd6700fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:45.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.141+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 shutdown_connections 2024-07-24T21:54:45.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:45.142+0000 7fd674ec6640 1 -- 172.21.15.60:0/1655228163 wait complete. 2024-07-24T21:54:45.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:45 smithi060 ceph-mon[33000]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-07-24T21:54:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:45 smithi123 ceph-mon[37143]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-07-24T21:54:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:45 smithi086 ceph-mon[37128]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-07-24T21:54:46.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:54:46.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:54:47.174 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:47.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:47 smithi060 ceph-mon[33000]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 85 op/s 2024-07-24T21:54:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:47 smithi123 ceph-mon[37143]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 85 op/s 2024-07-24T21:54:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:47 smithi086 ceph-mon[37128]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 882 KiB/s wr, 85 op/s 2024-07-24T21:54:48.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 -- 172.21.15.60:0/1504469292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c1068c0 msgr2=0x7fde9c106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 --2- 172.21.15.60:0/1504469292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c1068c0 0x7fde9c106d10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ca0 tx=0x7fde9402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 -- 172.21.15.60:0/1504469292 shutdown_connections 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 --2- 172.21.15.60:0/1504469292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c1068c0 0x7fde9c106d10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 --2- 172.21.15.60:0/1504469292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 --2- 172.21.15.60:0/1504469292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c101480 0x7fde9c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 -- 172.21.15.60:0/1504469292 >> 172.21.15.60:0/1504469292 conn(0x7fde9c0fd0d0 msgr2=0x7fde9c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 -- 172.21.15.60:0/1504469292 shutdown_connections 2024-07-24T21:54:48.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.866+0000 7fdea1945640 1 -- 172.21.15.60:0/1504469292 wait complete. 2024-07-24T21:54:48.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 Processor -- start 2024-07-24T21:54:48.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 -- start start 2024-07-24T21:54:48.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 0x7fde9c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 0x7fde9c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c118d80 con 0x7fde9c101480 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c118c00 con 0x7fde9c101da0 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fdea1945640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c118f00 con 0x7fde9c1068c0 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 0x7fde9c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fde9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 0x7fde9c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 0x7fde9c06d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51746/0 (socket says 172.21.15.60:51746) 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3870914409 learned_addr learned my addr 172.21.15.60:0/3870914409 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.867+0000 7fde93fff640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 -- 172.21.15.60:0/3870914409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 msgr2=0x7fde9c06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 0x7fde9c06d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 -- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 msgr2=0x7fde9c0740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 0x7fde9c0740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 -- 172.21.15.60:0/3870914409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c06dc70 con 0x7fde9c101da0 2024-07-24T21:54:48.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde9affd640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 0x7fde9c0740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:48.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fde93fff640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c074620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde8400c940 tx=0x7fde8400ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fdea0943640 1 -- 172.21.15.60:0/3870914409 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84007bb0 con 0x7fde9c101da0 2024-07-24T21:54:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fdea0943640 1 -- 172.21.15.60:0/3870914409 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde84007d50 con 0x7fde9c101da0 2024-07-24T21:54:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c06df60 con 0x7fde9c101da0 2024-07-24T21:54:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fdea0943640 1 -- 172.21.15.60:0/3870914409 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84018660 con 0x7fde9c101da0 2024-07-24T21:54:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.868+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c072f20 con 0x7fde9c101da0 2024-07-24T21:54:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.871+0000 7fdea0943640 1 -- 172.21.15.60:0/3870914409 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fde8400f450 con 0x7fde9c101da0 2024-07-24T21:54:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea3cf6740 1 Processor -- start 2024-07-24T21:54:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea3cf6740 1 -- start start 2024-07-24T21:54:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b0714f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea0943640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 0x7fde880b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.872+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 0x561b0715dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561b070c69f0 con 0x561b0714f850 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b070b5ef0 con 0x561b0714ef30 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b070b9f30 con 0x561b0715b7e0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b0714f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b0714f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59704/0 (socket says 172.21.15.60:59704) 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde9b7fe640 1 -- 172.21.15.60:0/2267637065 learned_addr learned my addr 172.21.15.60:0/2267637065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fdea0943640 1 -- 172.21.15.60:0/3870914409 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fde840e3f10 con 0x7fde9c101da0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde9affd640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 0x7fde880b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde93fff640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 0x561b0715dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde93fff640 1 -- 172.21.15.60:0/2267637065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 msgr2=0x561b0715dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.873+0000 7fde93fff640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 0x561b0715dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde93fff640 1 -- 172.21.15.60:0/2267637065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 msgr2=0x561b0714f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde93fff640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b0714f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde93fff640 1 -- 172.21.15.60:0/2267637065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b06e16930 con 0x561b0714f850 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 0x561b0715dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde9affd640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 0x7fde880b6280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009fc0 tx=0x7fde8c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde93fff640 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715b2a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde840e1630 tx=0x7fde84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65347b5d1c7a4d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde91ffb640 1 -- 172.21.15.60:0/2267637065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840ab1f0 con 0x561b0714f850 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde91ffb640 1 -- 172.21.15.60:0/2267637065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde840aaaf0 con 0x561b0714f850 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.874+0000 7fde91ffb640 1 -- 172.21.15.60:0/2267637065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8400f740 con 0x561b0714f850 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- 172.21.15.60:0/2267637065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 msgr2=0x561b0715b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715b2a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde840e1630 tx=0x7fde84000bc0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- 172.21.15.60:0/2267637065 shutdown_connections 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0715b7e0 0x561b0715dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715b2a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/2267637065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b0714f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- 172.21.15.60:0/2267637065 >> 172.21.15.60:0/2267637065 conn(0x561b0714cd60 msgr2=0x561b0714df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- 172.21.15.60:0/2267637065 shutdown_connections 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- 172.21.15.60:0/2267637065 wait complete. 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.875+0000 7fdea3cf6740 1 -- start start 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 Processor -- start 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b07158850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561b06f19890 con 0x561b0715b7e0 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b07058390 con 0x561b0714ef30 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b0715e400 con 0x561b0714f850 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fdea3cf6740 10 client.0 _ll_register_callbacks cb 0x561b07062500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56104/0 (socket says 172.21.15.60:56104) 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fde93fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3884939571 learned_addr learned my addr 172.21.15.60:0/3884939571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.876+0000 7fde9affd640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b07158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 msgr2=0x561b07158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3884939571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 msgr2=0x561b07158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b07158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3884939571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b07052d60 con 0x561b0715b7e0 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde93fff640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fde940316b0 tx=0x7fde940316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c919522c1666413 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde7affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:54:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94003b70 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fde94003d10 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde840aaaf0 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fde94004b40 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fdea3cf6740 10 client.17235 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.877+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b0716a370 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.878+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde840aaaf0 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.878+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fde9402f660 con 0x561b0715b7e0 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 10 client.17235 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 20 client.17235 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b0716b840 0x561b0716dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 4 client.17235 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.880+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b070c8940 con 0x561b0716b840 2024-07-24T21:54:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.881+0000 7fde9affd640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b0716b840 0x561b0716dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.882+0000 7fde9affd640 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b0716b840 0x561b0716dc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9399b61ed2a2e4e8 server_cookie=8359ec25fbd80743 in_seq=0 out_seq=0 2024-07-24T21:54:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.882+0000 7fde7affd640 10 client.17235 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.883+0000 7fde7affd640 1 -- 172.21.15.60:0/3884939571 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b070c8940 con 0x561b0716b840 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.883+0000 7fde7affd640 10 client.17235 handle_command_reply: tid=0 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.883+0000 7fdea3cf6740 1 client.17235 shutdown 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.883+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 msgr2=0x561b071592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.883+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fde940316b0 tx=0x7fde940316e0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 shutdown_connections 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b0716b840 0x561b0716dc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0715b7e0 0x561b071592d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fde7affd640 0 client.17235 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3884939571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0714ef30 0x561b07158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.884+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 >> 172.21.15.60:0/3884939571 conn(0x561b0714cd60 msgr2=0x561b07152900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.885+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 shutdown_connections 2024-07-24T21:54:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.885+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3884939571 wait complete. 2024-07-24T21:54:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.885+0000 7fdea3cf6740 20 client.17235 trim_cache size 0 max 16384 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 Processor -- start 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 -- start start 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714ef30 0x561b07173540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b07155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1bf820 con 0x561b0714ef30 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1bd060 con 0x561b0715b7e0 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1baf20 con 0x561b0714f850 2024-07-24T21:54:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fde93fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fde93fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51776/0 (socket says 172.21.15.60:51776) 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b07155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.917+0000 7fde93fff640 1 -- 172.21.15.60:0/496643981 learned_addr learned my addr 172.21.15.60:0/496643981 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 -- 172.21.15.60:0/496643981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 msgr2=0x561b07173a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 -- 172.21.15.60:0/496643981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714ef30 msgr2=0x561b07173540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714ef30 0x561b07173540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 -- 172.21.15.60:0/496643981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0714da40 con 0x561b0715b7e0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde93fff640 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b07155a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde9402fa50 tx=0x7fde94005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d31efdefe5b304 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde917fa640 1 -- 172.21.15.60:0/496643981 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94054080 con 0x561b0715b7e0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde917fa640 1 -- 172.21.15.60:0/496643981 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94043070 con 0x561b0715b7e0 2024-07-24T21:54:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.918+0000 7fde917fa640 1 -- 172.21.15.60:0/496643981 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94004750 con 0x561b0715b7e0 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 -- 172.21.15.60:0/496643981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 msgr2=0x561b07155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b07155a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde9402fa50 tx=0x7fde94005ef0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 -- 172.21.15.60:0/496643981 shutdown_connections 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b07155a60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0714f850 0x561b07173a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/496643981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714ef30 0x561b07173540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 -- 172.21.15.60:0/496643981 >> 172.21.15.60:0/496643981 conn(0x561b0714cd60 msgr2=0x561b0714be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 -- 172.21.15.60:0/496643981 shutdown_connections 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.920+0000 7fdea3cf6740 1 -- 172.21.15.60:0/496643981 wait complete. 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.921+0000 7fdea3cf6740 1 -- start start 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.921+0000 7fdea3cf6740 1 Processor -- start 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.922+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b0715fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1bb520 con 0x561b0714f850 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1bc990 con 0x561b0715b7e0 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1bbbf0 con 0x561b0716b840 2024-07-24T21:54:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fde9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fdea3cf6740 10 client.0 _ll_register_callbacks cb 0x7fde94031e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51786/0 (socket says 172.21.15.60:51786) 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.923+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3273030600 learned_addr learned my addr 172.21.15.60:0/3273030600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3273030600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 msgr2=0x561b0715fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b0715fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3273030600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 msgr2=0x561b0715f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 -- 172.21.15.60:0/3273030600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b071d5800 con 0x561b0716b840 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde9b7fe640 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde94002230 tx=0x7fde94004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e53a22cf02b3ef3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde7affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94054070 con 0x561b0716b840 2024-07-24T21:54:48.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fde940045f0 con 0x561b0716b840 2024-07-24T21:54:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.924+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde94043070 con 0x561b0716b840 2024-07-24T21:54:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.925+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fde94067050 con 0x561b0716b840 2024-07-24T21:54:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.925+0000 7fdea3cf6740 10 client.26674 fetch_fsmap learned FSMap version 39 2024-07-24T21:54:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.925+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b071d6b40 con 0x561b0716b840 2024-07-24T21:54:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.925+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94043070 con 0x561b0716b840 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fde94030b50 con 0x561b0716b840 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 10 client.26674 fetch_fsmap finished waiting for FSMap version 39 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 20 client.26674 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b071d72a0 0x561b071d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 4 client.26674 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.926+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b070c43e0 con 0x561b071d72a0 2024-07-24T21:54:48.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.928+0000 7fde9affd640 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b071d72a0 0x561b071d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:48.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.930+0000 7fde9affd640 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b071d72a0 0x561b071d96c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0503341e16d0da8 server_cookie=86cf7e3825937a03 in_seq=0 out_seq=0 2024-07-24T21:54:48.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.930+0000 7fde7affd640 10 client.26674 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:48.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.930+0000 7fde7affd640 1 -- 172.21.15.60:0/3273030600 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b070c43e0 con 0x561b071d72a0 2024-07-24T21:54:48.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.930+0000 7fde7affd640 10 client.26674 handle_command_reply: tid=0 2024-07-24T21:54:48.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.930+0000 7fdea3cf6740 1 client.26674 shutdown 2024-07-24T21:54:48.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 msgr2=0x561b071601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde94002230 tx=0x7fde94004270 comp rx=0 tx=0).stop 2024-07-24T21:54:48.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 shutdown_connections 2024-07-24T21:54:48.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561b071d72a0 0x561b071d96c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0716b840 0x561b071601a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0715b7e0 0x561b0715fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 --2- 172.21.15.60:0/3273030600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561b0714f850 0x561b0715f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 >> 172.21.15.60:0/3273030600 conn(0x561b0714cd60 msgr2=0x561b0714b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.931+0000 7fde7affd640 0 client.26674 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:48.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.932+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 shutdown_connections 2024-07-24T21:54:48.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.936+0000 7fdea3cf6740 1 -- 172.21.15.60:0/3273030600 wait complete. 2024-07-24T21:54:48.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.936+0000 7fdea3cf6740 20 client.26674 trim_cache size 0 max 16384 2024-07-24T21:54:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:48.940 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:54:48.940 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "a945f6ce-e2d7-47bf-91c3-ec1de5505c2c", 2024-07-24T21:54:48.940 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:54:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 msgr2=0x7fde880b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 0x7fde880b6280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009fc0 tx=0x7fde8c005f90 comp rx=0 tx=0).stop 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 msgr2=0x7fde9c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c074620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde8400c940 tx=0x7fde8400ce00 comp rx=0 tx=0).stop 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 shutdown_connections 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fde880b3dc0 0x7fde880b6280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde9c1068c0 0x7fde9c06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9c101da0 0x7fde9c074620 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 --2- 172.21.15.60:0/3870914409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c101480 0x7fde9c0740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.939+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 >> 172.21.15.60:0/3870914409 conn(0x7fde9c0fd0d0 msgr2=0x7fde9c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.940+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 shutdown_connections 2024-07-24T21:54:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:48.940+0000 7fdea1945640 1 -- 172.21.15.60:0/3870914409 wait complete. 2024-07-24T21:54:49.355 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:54:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:54:49.013+0000 7f827ed99640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000b8b [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/ auth{1=1,2=1} fragtree_t(*^3) v211998 ap=2 f(v8 m2024-07-24T21:53:03.964623+0000 187=187+0) n(v18 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1) old_inodes=5 (inest sync->mix g=1,2) caps={15300=pAsLsXsFs/-@39} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x55de933fcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000b8b:[<0x10000000399/blog-17 v13270>,<0x10000000208/blogtest_in v345>,<0x10000000201/src v2099>,<0x10000000006/blogbench-1.0 v1818>,<0x10000000005/tmp v50>,<0x100000001fb/client.0 v40>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v60>,<0x10000000001/sv_1 v80>,<0x10000000000/qa v69>,<0x1/volumes v66>]//[]","memoryvalue":"(2)0x10000000b8b:[<0x10000000399/blog-17 v211998>,<0x10000000208/blogtest_in v2375>,<0x10000000201/src v2560>,<0x10000000006/blogbench-1.0 v2215>,<0x10000000005/tmp v439>,<0x100000001fb/client.0 v401>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v434>,<0x10000000001/sv_1 v439>,<0x10000000000/qa v450>,<0x1/volumes v156>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:06.935132+0000 186=186+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1)","memory_value.dirstat":"f(v8 m2024-07-24T21:53:03.964623+0000 187=187+0)","memory_value.rstat":"n(v18 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:54:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:49 smithi086 ceph-mon[37128]: pgmap v626: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-07-24T21:54:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:49 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:49 smithi086 ceph-mon[37128]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:54:49.546 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:54:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T21:54:49.265+0000 7f4d4d67a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001bd5 [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51/ auth{0=1,2=1} v185768 ap=2 f(v7 m2024-07-24T21:53:00.997649+0000 91=91+0) n(v17 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1) old_inodes=4 (inest sync->mix g=0,2) caps={15300=pAsLsXsFs/-@1} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x563853a06100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000001bd5:[<0x10000000399/blog-51 v136553>,<0x10000000208/blogtest_in v1507>,<0x10000000201/src v2196>,<0x10000000006/blogbench-1.0 v1907>,<0x10000000005/tmp v130>,<0x100000001fb/client.0 v114>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v128>,<0x10000000001/sv_1 v151>,<0x10000000000/qa v124>,<0x1/volumes v110>]//[]","memoryvalue":"(2)0x20000001bd5:[<0x10000000399/blog-51 v185768>,<0x10000000208/blogtest_in v2379>,<0x10000000201/src v2628>,<0x10000000006/blogbench-1.0 v2223>,<0x10000000005/tmp v445>,<0x100000001fb/client.0 v409>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v442>,<0x10000000001/sv_1 v447>,<0x10000000000/qa v468>,<0x1/volumes v156>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:52.799721+0000 90=90+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1)","memory_value.dirstat":"f(v7 m2024-07-24T21:53:00.997649+0000 91=91+0)","memory_value.rstat":"n(v17 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:54:49.546 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:54:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T21:54:49.351+0000 7f4d4d67a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000002d0 [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20/ auth{0=1,2=2} fragtree_t(*^3) v185770 ap=2 f(v7 m2024-07-24T21:53:02.122640+0000 218=218+0) n(v18 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1) old_inodes=5 (inest sync->mix g=0,2) caps={15300=pAsLsXsFs/-@18} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x5638535c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x200000002d0:[<0x10000000399/blog-20 v134388>,<0x10000000208/blogtest_in v1507>,<0x10000000201/src v2196>,<0x10000000006/blogbench-1.0 v1907>,<0x10000000005/tmp v130>,<0x100000001fb/client.0 v114>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v128>,<0x10000000001/sv_1 v151>,<0x10000000000/qa v124>,<0x1/volumes v110>]//[]","memoryvalue":"(2)0x200000002d0:[<0x10000000399/blog-20 v185770>,<0x10000000208/blogtest_in v2381>,<0x10000000201/src v2630>,<0x10000000006/blogbench-1.0 v2223>,<0x10000000005/tmp v445>,<0x100000001fb/client.0 v409>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v442>,<0x10000000001/sv_1 v447>,<0x10000000000/qa v468>,<0x1/volumes v156>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:26.887951+0000 216=216+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1)","memory_value.dirstat":"f(v7 m2024-07-24T21:53:02.122640+0000 218=218+0)","memory_value.rstat":"n(v18 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:54:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:49 smithi123 ceph-mon[37143]: pgmap v626: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-07-24T21:54:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:49 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:49 smithi123 ceph-mon[37143]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:54:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:49 smithi060 ceph-mon[33000]: pgmap v626: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-07-24T21:54:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:49 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:49 smithi060 ceph-mon[33000]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:54:50.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:54:50.400 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.i log and `damage ls` output for details 2024-07-24T21:54:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:50 smithi060 ceph-mon[33000]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.i log and `damage ls` output for details 2024-07-24T21:54:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:50 smithi123 ceph-mon[37143]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:54:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:54:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.i log and `damage ls` output for details 2024-07-24T21:54:50.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:50 smithi086 ceph-mon[37128]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:54:51.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:51 smithi123 ceph-mon[37143]: pgmap v627: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-07-24T21:54:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:51 smithi086 ceph-mon[37128]: pgmap v627: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-07-24T21:54:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:51 smithi060 ceph-mon[33000]: pgmap v627: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 268 op/s 2024-07-24T21:54:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.465+0000 7fa58663c640 1 -- 172.21.15.60:0/354401784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 msgr2=0x7fa5801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.465+0000 7fa58663c640 1 --2- 172.21.15.60:0/354401784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa5801038d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa570009900 tx=0x7fa57002a410 comp rx=0 tx=0).stop 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 -- 172.21.15.60:0/354401784 shutdown_connections 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 --2- 172.21.15.60:0/354401784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 --2- 172.21.15.60:0/354401784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 0x7fa58010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 --2- 172.21.15.60:0/354401784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa5801038d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 -- 172.21.15.60:0/354401784 >> 172.21.15.60:0/354401784 conn(0x7fa5800fd0c0 msgr2=0x7fa5800ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 -- 172.21.15.60:0/354401784 shutdown_connections 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 -- 172.21.15.60:0/354401784 wait complete. 2024-07-24T21:54:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.466+0000 7fa58663c640 1 Processor -- start 2024-07-24T21:54:52.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:52 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 -- start start 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 0x7fa580199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56128/0 (socket says 172.21.15.60:56128) 2024-07-24T21:54:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa580112740 con 0x7fa5801034f0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801125c0 con 0x7fa580103ef0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa58663c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801128c0 con 0x7fa58010caa0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.467+0000 7fa57ffff640 1 -- 172.21.15.60:0/1400551557 learned_addr learned my addr 172.21.15.60:0/1400551557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 0x7fa580199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa584bd9640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 -- 172.21.15.60:0/1400551557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 msgr2=0x7fa58019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 -- 172.21.15.60:0/1400551557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 msgr2=0x7fa580199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 0x7fa580199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 -- 172.21.15.60:0/1400551557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58019ecc0 con 0x7fa5801034f0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa584bd9640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.468+0000 7fa57ffff640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa570005ab0 tx=0x7fa570003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.469+0000 7fa57d7fa640 1 -- 172.21.15.60:0/1400551557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57003f070 con 0x7fa5801034f0 2024-07-24T21:54:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.469+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58019ef50 con 0x7fa5801034f0 2024-07-24T21:54:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.469+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa580101250 con 0x7fa5801034f0 2024-07-24T21:54:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.469+0000 7fa57d7fa640 1 -- 172.21.15.60:0/1400551557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570031b60 con 0x7fa5801034f0 2024-07-24T21:54:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.469+0000 7fa57d7fa640 1 -- 172.21.15.60:0/1400551557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570035030 con 0x7fa5801034f0 2024-07-24T21:54:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.472+0000 7fa57d7fa640 1 -- 172.21.15.60:0/1400551557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa570042070 con 0x7fa5801034f0 2024-07-24T21:54:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 Processor -- start 2024-07-24T21:54:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 -- start start 2024-07-24T21:54:52.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e6fd0 0x55c2088f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c20885d990 con 0x55c2088f2f60 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c20884ce90 con 0x55c2088e66b0 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa5889ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c208850ed0 con 0x55c2088e6fd0 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa584bd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa584bd9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59762/0 (socket says 172.21.15.60:59762) 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.473+0000 7fa584bd9640 1 -- 172.21.15.60:0/104793453 learned_addr learned my addr 172.21.15.60:0/104793453 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:52.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa57d7fa640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 0x7fa5680b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 -- 172.21.15.60:0/104793453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e6fd0 msgr2=0x55c2088f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e6fd0 0x55c2088f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 -- 172.21.15.60:0/104793453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 msgr2=0x55c2088f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 -- 172.21.15.60:0/104793453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2087d6300 con 0x55c2088e66b0 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa57d7fa640 1 -- 172.21.15.60:0/1400551557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fa5701046f0 con 0x7fa5801034f0 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 0x7fa5680b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa584bd9640 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa578010620 tx=0x7fa578010ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d61d602ceff71e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.474+0000 7fa5667fc640 1 -- 172.21.15.60:0/104793453 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578022070 con 0x55c2088e66b0 2024-07-24T21:54:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.475+0000 7fa5667fc640 1 -- 172.21.15.60:0/104793453 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57800ee50 con 0x55c2088e66b0 2024-07-24T21:54:52.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.475+0000 7fa5667fc640 1 -- 172.21.15.60:0/104793453 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578019070 con 0x55c2088e66b0 2024-07-24T21:54:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.475+0000 7fa584bd9640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 0x7fa5680b6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa57801f000 tx=0x7fa57801ddb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.475+0000 7fa5889ed740 1 -- 172.21.15.60:0/104793453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 msgr2=0x55c2088e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa578010620 tx=0x7fa578010ae0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 -- 172.21.15.60:0/104793453 shutdown_connections 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e6fd0 0x55c2088f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 --2- 172.21.15.60:0/104793453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e66b0 0x55c2088e6a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 -- 172.21.15.60:0/104793453 >> 172.21.15.60:0/104793453 conn(0x55c2088e44e0 msgr2=0x55c2088e5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 -- 172.21.15.60:0/104793453 shutdown_connections 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 -- 172.21.15.60:0/104793453 wait complete. 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 -- start start 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.476+0000 7fa5889ed740 1 Processor -- start 2024-07-24T21:54:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 0x55c2088efea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c2088f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088f2f60 0x55c2088f0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c2086b0700 con 0x55c2088e66b0 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2087ef5d0 con 0x55c2088e6fd0 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2088fe3f0 con 0x55c2088f2f60 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 0x55c2088efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c2088f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 0x55c2088efea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56154/0 (socket says 172.21.15.60:56154) 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57ffff640 1 -- 172.21.15.60:0/1133265584 learned_addr learned my addr 172.21.15.60:0/1133265584 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa5889ed740 10 client.0 _ll_register_callbacks cb 0x55c20880f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 -- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088f2f60 msgr2=0x55c2088f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088f2f60 0x55c2088f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 -- 172.21.15.60:0/1133265584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 msgr2=0x55c2088efea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 0x55c2088efea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.477+0000 7fa57f7fe640 1 -- 172.21.15.60:0/1133265584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2087e2660 con 0x55c2088e6fd0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c2088f03e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5740136e0 tx=0x7fa574013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2384221e61a7de66 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574004460 con 0x55c2088e6fd0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fa574011960 con 0x55c2088e6fd0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57800ee50 con 0x55c2088e6fd0 2024-07-24T21:54:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.478+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa574011b60 con 0x55c2088e6fd0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 10 client.26654 fetch_fsmap learned FSMap version 40 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c2085ad650 con 0x55c2088e6fd0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5740054d0 con 0x55c2088e6fd0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fa57401a070 con 0x55c2088e6fd0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 10 client.26654 fetch_fsmap finished waiting for FSMap version 40 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 20 client.26654 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c20890b3a0 0x55c20890d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 4 client.26654 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.479+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa57800ee50 con 0x55c20890b3a0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.480+0000 7fa57ffff640 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c20890b3a0 0x55c20890d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.481+0000 7fa57ffff640 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c20890b3a0 0x55c20890d800 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22fa05a00d27276d server_cookie=47d0ba7891c2e098 in_seq=0 out_seq=0 2024-07-24T21:54:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.481+0000 7fa55f7fe640 10 client.26654 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa55f7fe640 1 -- 172.21.15.60:0/1133265584 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa57800ee50 con 0x55c20890b3a0 2024-07-24T21:54:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa55f7fe640 10 client.26654 handle_command_reply: tid=0 2024-07-24T21:54:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 client.26654 shutdown 2024-07-24T21:54:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 msgr2=0x55c2088f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c2088f03e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5740136e0 tx=0x7fa574013ba0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 shutdown_connections 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c20890b3a0 0x55c20890d800 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088f2f60 0x55c2088f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c2088f03e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 --2- 172.21.15.60:0/1133265584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088e66b0 0x55c2088efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa55f7fe640 0 client.26654 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 >> 172.21.15.60:0/1133265584 conn(0x55c2088e44e0 msgr2=0x55c2088ea080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 shutdown_connections 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 1 -- 172.21.15.60:0/1133265584 wait complete. 2024-07-24T21:54:52.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.483+0000 7fa5889ed740 20 client.26654 trim_cache size 0 max 16384 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.509+0000 7fa5889ed740 1 Processor -- start 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 -- start start 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c208913110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ae870 con 0x55c2088f2f60 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ac080 con 0x55c2088e6fd0 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa5889ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801a9ef0 con 0x55c2088e66b0 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa584bd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59774/0 (socket says 172.21.15.60:59774) 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57f7fe640 1 -- 172.21.15.60:0/4241832474 learned_addr learned my addr 172.21.15.60:0/4241832474 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa584bd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56168/0 (socket says 172.21.15.60:56168) 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57ffff640 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c208913110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57f7fe640 1 -- 172.21.15.60:0/4241832474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 msgr2=0x55c208913110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.510+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c208913110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa57f7fe640 1 -- 172.21.15.60:0/4241832474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 msgr2=0x55c2088ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa57f7fe640 1 -- 172.21.15.60:0/4241832474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2088dfe00 con 0x55c2088e6fd0 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa57ffff640 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c208913110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa57400a010 tx=0x7fa574011990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc9dc915e39fece server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa565ffb640 1 -- 172.21.15.60:0/4241832474 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574003710 con 0x55c2088e6fd0 2024-07-24T21:54:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa565ffb640 1 -- 172.21.15.60:0/4241832474 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5740038b0 con 0x55c2088e6fd0 2024-07-24T21:54:52.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.511+0000 7fa565ffb640 1 -- 172.21.15.60:0/4241832474 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574004660 con 0x55c2088e6fd0 2024-07-24T21:54:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- 172.21.15.60:0/4241832474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 msgr2=0x55c208913650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa57400a010 tx=0x7fa574011990 comp rx=0 tx=0).stop 2024-07-24T21:54:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- 172.21.15.60:0/4241832474 shutdown_connections 2024-07-24T21:54:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208913650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 --2- 172.21.15.60:0/4241832474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c208913110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- 172.21.15.60:0/4241832474 >> 172.21.15.60:0/4241832474 conn(0x55c2088e44e0 msgr2=0x55c2088f1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- 172.21.15.60:0/4241832474 shutdown_connections 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- 172.21.15.60:0/4241832474 wait complete. 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.512+0000 7fa5889ed740 1 -- start start 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 Processor -- start 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c2089039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801aa540 con 0x55c2088f2f60 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ab9b0 con 0x55c2088e6fd0 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801aac10 con 0x55c2088e66b0 2024-07-24T21:54:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa5889ed740 10 client.0 _ll_register_callbacks cb 0x7fa574009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59780/0 (socket says 172.21.15.60:59780) 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa57f7fe640 1 -- 172.21.15.60:0/2447928239 learned_addr learned my addr 172.21.15.60:0/2447928239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa57ffff640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c2089039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.513+0000 7fa584bd9640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 -- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 msgr2=0x55c2089039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c2089039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 -- 172.21.15.60:0/2447928239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 msgr2=0x55c2088ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 -- 172.21.15.60:0/2447928239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2088ff880 con 0x55c2088e6fd0 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57ffff640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c2089039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa584bd9640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa57f7fe640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa574004e20 tx=0x7fa574003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4624f9ec329550c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.514+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574003e70 con 0x55c2088e6fd0 2024-07-24T21:54:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5740038b0 con 0x55c2088e6fd0 2024-07-24T21:54:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fa574005020 con 0x55c2088e6fd0 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa57403d050 con 0x55c2088e6fd0 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa5889ed740 10 client.26660 fetch_fsmap learned FSMap version 40 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c208975d70 con 0x55c2088e6fd0 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.515+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5740038b0 con 0x55c2088e6fd0 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fa57401a070 con 0x55c2088e6fd0 2024-07-24T21:54:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 10 client.26660 fetch_fsmap finished waiting for FSMap version 40 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 20 client.26660 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c208976520 0x55c208978940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 4 client.26660 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c20885f8e0 con 0x55c208976520 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.516+0000 7fa57ffff640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c208976520 0x55c208978940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.517+0000 7fa57ffff640 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c208976520 0x55c208978940 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18e107da58453ce server_cookie=f018ba2a0e6d584 in_seq=0 out_seq=0 2024-07-24T21:54:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.517+0000 7fa55f7fe640 10 client.26660 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.519+0000 7fa55f7fe640 1 -- 172.21.15.60:0/2447928239 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55c20885f8e0 con 0x55c208976520 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.519+0000 7fa55f7fe640 10 client.26660 handle_command_reply: tid=0 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.519+0000 7fa5889ed740 1 client.26660 shutdown 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 msgr2=0x55c208903f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa574004e20 tx=0x7fa574003780 comp rx=0 tx=0).stop 2024-07-24T21:54:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 shutdown_connections 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c208976520 0x55c208978940 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2088f2f60 0x55c2088ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2088e6fd0 0x55c208903f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 --2- 172.21.15.60:0/2447928239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2088e66b0 0x55c2089039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 >> 172.21.15.60:0/2447928239 conn(0x55c2088e44e0 msgr2=0x55c2088efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa55f7fe640 0 client.26660 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 shutdown_connections 2024-07-24T21:54:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 1 -- 172.21.15.60:0/2447928239 wait complete. 2024-07-24T21:54:52.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.520+0000 7fa5889ed740 20 client.26660 trim_cache size 0 max 16384 2024-07-24T21:54:52.526 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:54:52.526 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (15510 inodes in the stack)", 2024-07-24T21:54:52.526 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:54:52.527 INFO:teuthology.orchestra.run.smithi060.stdout: "a945f6ce-e2d7-47bf-91c3-ec1de5505c2c": { 2024-07-24T21:54:52.527 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:54:52.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "a945f6ce-e2d7-47bf-91c3-ec1de5505c2c", 2024-07-24T21:54:52.527 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:54:52.527 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:52.528 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:54:52.528 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 msgr2=0x7fa5680b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 0x7fa5680b6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa57801f000 tx=0x7fa57801ddb0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 msgr2=0x7fa580198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa570005ab0 tx=0x7fa570003c10 comp rx=0 tx=0).stop 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 shutdown_connections 2024-07-24T21:54:52.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa5680b3dc0 0x7fa5680b6280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa58010caa0 0x7fa58019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa580103ef0 0x7fa580199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 --2- 172.21.15.60:0/1400551557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5801034f0 0x7fa580198c10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 >> 172.21.15.60:0/1400551557 conn(0x7fa5800fd0c0 msgr2=0x7fa5800fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 shutdown_connections 2024-07-24T21:54:52.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:54:52.521+0000 7fa58663c640 1 -- 172.21.15.60:0/1400551557 wait complete. 2024-07-24T21:54:52.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:52 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:52.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:52 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:53 smithi123 ceph-mon[37143]: pgmap v628: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 788 op/s 2024-07-24T21:54:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:53.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:53 smithi060 ceph-mon[33000]: pgmap v628: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 788 op/s 2024-07-24T21:54:53.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:53 smithi086 ceph-mon[37128]: pgmap v628: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 788 op/s 2024-07-24T21:54:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:53.827 INFO:tasks.cephfs.filesystem:scrub status for tag:a945f6ce-e2d7-47bf-91c3-ec1de5505c2c - {'path': '/', 'tag': 'a945f6ce-e2d7-47bf-91c3-ec1de5505c2c', 'options': 'recursive,force'} 2024-07-24T21:54:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.990+0000 7f7f8559a640 1 -- 172.21.15.60:0/281503099 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c030830 con 0x7f7f88101ce0 2024-07-24T21:54:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 -- 172.21.15.60:0/281503099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f88101ce0 msgr2=0x7f7f881061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/281503099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f88101ce0 0x7f7f881061a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009890 tx=0x7f7f7c02a370 comp rx=0 tx=0).stop 2024-07-24T21:54:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 -- 172.21.15.60:0/281503099 shutdown_connections 2024-07-24T21:54:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/281503099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881067f0 0x7f7f88106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/281503099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f88101ce0 0x7f7f881061a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/281503099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f881013c0 0x7f7f881017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.991+0000 7f7f8ca65640 1 -- 172.21.15.60:0/281503099 >> 172.21.15.60:0/281503099 conn(0x7f7f880782e0 msgr2=0x7f7f880ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.992+0000 7f7f8ca65640 1 -- 172.21.15.60:0/281503099 shutdown_connections 2024-07-24T21:54:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.992+0000 7f7f8ca65640 1 -- 172.21.15.60:0/281503099 wait complete. 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 Processor -- start 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 -- start start 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 0x7f7f8819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 0x7f7f881a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88118cd0 con 0x7f7f88101ce0 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88118b50 con 0x7f7f881013c0 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8ca65640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88118e50 con 0x7f7f881067f0 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56188/0 (socket says 172.21.15.60:56188) 2024-07-24T21:54:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f85d9b640 1 -- 172.21.15.60:0/87550179 learned_addr learned my addr 172.21.15.60:0/87550179 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f8659c640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 0x7f7f8819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.993+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 0x7f7f881a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 -- 172.21.15.60:0/87550179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 msgr2=0x7f7f881a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 0x7f7f881a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 -- 172.21.15.60:0/87550179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 msgr2=0x7f7f8819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 0x7f7f8819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 -- 172.21.15.60:0/87550179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f881a53f0 con 0x7f7f88101ce0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 0x7f7f881a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f8659c640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 0x7f7f8819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02a8e0 tx=0x7f7f7c030d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f777fe640 1 -- 172.21.15.60:0/87550179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c037030 con 0x7f7f88101ce0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f881a5680 con 0x7f7f88101ce0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f777fe640 1 -- 172.21.15.60:0/87550179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c041070 con 0x7f7f88101ce0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f777fe640 1 -- 172.21.15.60:0/87550179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0046c0 con 0x7f7f88101ce0 2024-07-24T21:54:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.994+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881abda0 con 0x7f7f88101ce0 2024-07-24T21:54:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.998+0000 7f7f8f7c0740 1 Processor -- start 2024-07-24T21:54:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.998+0000 7f7f8f7c0740 1 -- start start 2024-07-24T21:54:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248f1e0 0x55f30249ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f30249d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f302404280 con 0x55f30248f1e0 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3023f3780 con 0x55f30248e8c0 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f3023f77c0 con 0x55f30249b170 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59798/0 (socket says 172.21.15.60:59798) 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 -- 172.21.15.60:0/4258835840 learned_addr learned my addr 172.21.15.60:0/4258835840 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f8659c640 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248f1e0 0x55f30249ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f777fe640 1 -- 172.21.15.60:0/87550179 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7f7c030450 con 0x7f7f88101ce0 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 -- 172.21.15.60:0/4258835840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 msgr2=0x55f30249d560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f30249d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 -- 172.21.15.60:0/4258835840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248f1e0 msgr2=0x55f30249ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248f1e0 0x55f30249ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 -- 172.21.15.60:0/4258835840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f302157790 con 0x55f30248e8c0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:53.999+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000e870 tx=0x7f7f800095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a60e2d63415b237 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.000+0000 7f7f74ff9640 1 -- 172.21.15.60:0/4258835840 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000ddd0 con 0x55f30248e8c0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.000+0000 7f7f74ff9640 1 -- 172.21.15.60:0/4258835840 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8001c070 con 0x55f30248e8c0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.000+0000 7f7f74ff9640 1 -- 172.21.15.60:0/4258835840 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800165e0 con 0x55f30248e8c0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.000+0000 7f7f777fe640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 0x7f7f700b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f777fe640 1 -- 172.21.15.60:0/87550179 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7f7c103bc0 con 0x7f7f88101ce0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/4258835840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 msgr2=0x55f30248eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000e870 tx=0x7f7f800095c0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8659c640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 0x7f7f700b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/4258835840 shutdown_connections 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f30249d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248f1e0 0x55f30249ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/4258835840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30248eca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/4258835840 >> 172.21.15.60:0/4258835840 conn(0x55f30248c6f0 msgr2=0x55f30248d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/4258835840 shutdown_connections 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/4258835840 wait complete. 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8659c640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 0x7f7f700b6230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55f302491a40 tx=0x7f7f78005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.001+0000 7f7f8f7c0740 1 -- start start 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 Processor -- start 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248f1e0 0x55f302497d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f30232dab0 con 0x55f30249b170 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f30237b9e0 con 0x55f30248e8c0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59812/0 (socket says 172.21.15.60:59812) 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8659c640 1 -- 172.21.15.60:0/30306159 learned_addr learned my addr 172.21.15.60:0/30306159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f30249def0 con 0x55f30248f1e0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f8f7c0740 10 client.0 _ll_register_callbacks cb 0x55f302103a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248f1e0 0x55f302497d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.002+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 -- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248f1e0 msgr2=0x55f302497d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248f1e0 0x55f302497d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 -- 172.21.15.60:0/30306159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 msgr2=0x55f3024982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 -- 172.21.15.60:0/30306159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3021d0620 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8659c640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800fb10 tx=0x7f7f78004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525e2b867f7fc377 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78004020 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3023a2ce0 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7f78018030 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7f780041c0 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8f7c0740 10 client.26672 fetch_fsmap learned FSMap version 40 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.003+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f3024a9730 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f3023a2ce0 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7f780036c0 con 0x55f30248e8c0 2024-07-24T21:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 10 client.26672 fetch_fsmap finished waiting for FSMap version 40 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 20 client.26672 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f3024aac60 0x55f3024ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 4 client.26672 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3024061d0 con 0x55f3024aac60 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f3024aac60 0x55f3024ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f3024aac60 0x55f3024ad070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=814f87e9138da02 server_cookie=befc9258f3e8a9d2 in_seq=0 out_seq=0 2024-07-24T21:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.004+0000 7f7f6dffb640 10 client.26672 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f6dffb640 1 -- 172.21.15.60:0/30306159 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3024061d0 con 0x55f3024aac60 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f6dffb640 10 client.26672 handle_command_reply: tid=0 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 client.26672 shutdown 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 msgr2=0x55f30249a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800fb10 tx=0x7f7f78004730 comp rx=0 tx=0).stop 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 shutdown_connections 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f3024aac60 0x55f3024ad070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248f1e0 0x55f302497d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/30306159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248e8c0 0x55f30249a890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f6dffb640 0 client.26672 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 >> 172.21.15.60:0/30306159 conn(0x55f30248c6f0 msgr2=0x55f302492290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 shutdown_connections 2024-07-24T21:54:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/30306159 wait complete. 2024-07-24T21:54:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.006+0000 7f7f8f7c0740 20 client.26672 trim_cache size 0 max 16384 2024-07-24T21:54:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.023+0000 7f7f8f7c0740 1 Processor -- start 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.023+0000 7f7f8f7c0740 1 -- start start 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248e8c0 0x55f3024b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f302494f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b7220 con 0x55f30248e8c0 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b4a30 con 0x55f30248f1e0 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b28a0 con 0x55f30249b170 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f8659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248e8c0 0x55f3024b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59828/0 (socket says 172.21.15.60:59828) 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3177493442 learned_addr learned my addr 172.21.15.60:0/3177493442 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3177493442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 msgr2=0x55f302494f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f302494f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3177493442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248e8c0 msgr2=0x55f3024b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248e8c0 0x55f3024b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3177493442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30248f010 con 0x55f30248f1e0 2024-07-24T21:54:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0c68d0 tx=0x7f7f7c0b2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4bd4e587dd3abd9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:54.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.024+0000 7f7f6ffff640 1 -- 172.21.15.60:0/3177493442 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0328f0 con 0x55f30248f1e0 2024-07-24T21:54:54.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f6ffff640 1 -- 172.21.15.60:0/3177493442 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c038040 con 0x55f30248f1e0 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f6ffff640 1 -- 172.21.15.60:0/3177493442 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0cee50 con 0x55f30248f1e0 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3177493442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 msgr2=0x55f3024b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0c68d0 tx=0x7f7f7c0b2040 comp rx=0 tx=0).stop 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3177493442 shutdown_connections 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30249b170 0x55f302494f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f3024b2e80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3177493442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30248e8c0 0x55f3024b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3177493442 >> 172.21.15.60:0/3177493442 conn(0x55f30248c6f0 msgr2=0x55f302497d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.025+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3177493442 shutdown_connections 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3177493442 wait complete. 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 -- start start 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 Processor -- start 2024-07-24T21:54:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248e8c0 0x55f3024a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b2ef0 con 0x55f30249b170 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b4360 con 0x55f30248f1e0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b35c0 con 0x55f30248e8c0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.026+0000 7f7f8f7c0740 10 client.0 _ll_register_callbacks cb 0x7f7f78009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f86d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59840/0 (socket says 172.21.15.60:59840) 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3706079755 learned_addr learned my addr 172.21.15.60:0/3706079755 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f8659c640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248e8c0 0x55f3024a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248e8c0 msgr2=0x55f3024a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248e8c0 0x55f3024a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3706079755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 msgr2=0x55f3024a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 -- 172.21.15.60:0/3706079755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f302513910 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f86d9d640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f85d9b640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009230 tx=0x7f7f7c103e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f38c74cf46e060 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0caf50 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7f7c0c6ae0 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c038040 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c0d3900 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f8f7c0740 10 client.26678 fetch_fsmap learned FSMap version 40 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.027+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f302514a70 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c038040 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7f7c101500 con 0x55f30248f1e0 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 10 client.26678 fetch_fsmap finished waiting for FSMap version 40 2024-07-24T21:54:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 10 client.26678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 20 client.26678 populate_metadata read hostname 'smithi060' 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f302515230 0x55f302517650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 4 client.26678 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f302401c70 con 0x55f302515230 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.028+0000 7f7f8659c640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f302515230 0x55f302517650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.029+0000 7f7f8659c640 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f302515230 0x55f302517650 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e28b6a64d3dc948 server_cookie=2739099b0667c356 in_seq=0 out_seq=0 2024-07-24T21:54:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.029+0000 7f7f6dffb640 10 client.26678 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:54.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:54.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f6dffb640 1 -- 172.21.15.60:0/3706079755 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f302401c70 con 0x55f302515230 2024-07-24T21:54:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f6dffb640 10 client.26678 handle_command_reply: tid=0 2024-07-24T21:54:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 client.26678 shutdown 2024-07-24T21:54:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 msgr2=0x55f30249fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009230 tx=0x7f7f7c103e80 comp rx=0 tx=0).stop 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 shutdown_connections 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f302515230 0x55f302517650 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f30249b170 0x55f3024a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f6dffb640 0 client.26678 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f30248f1e0 0x55f30249fe00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 --2- 172.21.15.60:0/3706079755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f30248e8c0 0x55f3024a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.742+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 >> 172.21.15.60:0/3706079755 conn(0x55f30248c6f0 msgr2=0x55f30248af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.743+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 shutdown_connections 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.743+0000 7f7f8f7c0740 1 -- 172.21.15.60:0/3706079755 wait complete. 2024-07-24T21:54:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.743+0000 7f7f8f7c0740 20 client.26678 trim_cache size 0 max 16384 2024-07-24T21:54:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:54:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:54:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:54:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 msgr2=0x7f7f700b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 0x7f7f700b6230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55f302491a40 tx=0x7f7f78005f90 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 msgr2=0x7f7f8819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02a8e0 tx=0x7f7f7c030d60 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 shutdown_connections 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f700b3d70 0x7f7f700b6230 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f881067f0 0x7f7f881a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88101ce0 0x7f7f8819f890 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 --2- 172.21.15.60:0/87550179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f881013c0 0x7f7f8819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 >> 172.21.15.60:0/87550179 conn(0x7f7f880782e0 msgr2=0x7f7f881040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:54:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 shutdown_connections 2024-07-24T21:54:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:54:54.744+0000 7f7f8ca65640 1 -- 172.21.15.60:0/87550179 wait complete. 2024-07-24T21:54:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:54.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:55.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:55 smithi060 ceph-mon[33000]: pgmap v629: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 1.09k op/s 2024-07-24T21:54:55.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:55 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:55.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:55 smithi123 ceph-mon[37143]: pgmap v629: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 1.09k op/s 2024-07-24T21:54:55.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:55 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:55 smithi086 ceph-mon[37128]: pgmap v629: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 1.09k op/s 2024-07-24T21:54:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:55 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:57 smithi123 ceph-mon[37143]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 2.63k op/s 2024-07-24T21:54:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:57 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:57 smithi086 ceph-mon[37128]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 2.63k op/s 2024-07-24T21:54:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:57 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:57 smithi060 ceph-mon[33000]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 2.63k op/s 2024-07-24T21:54:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:57 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.750 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.751 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.752 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.753 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.755 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:58.756 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:58.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:58.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: pgmap v631: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 3.10k op/s 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:54:59.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:54:59 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: pgmap v631: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 3.10k op/s 2024-07-24T21:54:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:54:59.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:54:59 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: pgmap v631: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 3.10k op/s 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:54:59.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:54:59 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:01 smithi123 ceph-mon[37143]: pgmap v632: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 3.37k op/s 2024-07-24T21:55:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:01 smithi086 ceph-mon[37128]: pgmap v632: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 3.37k op/s 2024-07-24T21:55:01.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:01 smithi060 ceph-mon[33000]: pgmap v632: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 3.37k op/s 2024-07-24T21:55:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:02.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:03 smithi123 ceph-mon[37143]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 3.61k op/s 2024-07-24T21:55:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:03 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:55:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:03 smithi086 ceph-mon[37128]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 3.61k op/s 2024-07-24T21:55:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:03 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:55:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:03 smithi060 ceph-mon[33000]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 3.61k op/s 2024-07-24T21:55:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:03 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:55:04.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:55:04.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:55:04.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:55:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:55:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:55:04.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:55:04.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:55:04.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:55:04.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:55:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:05 smithi086 ceph-mon[37128]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-07-24T21:55:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:05 smithi060 ceph-mon[33000]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-07-24T21:55:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:05 smithi123 ceph-mon[37143]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-07-24T21:55:07.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:06 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:55:07.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:06 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:55:07.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:06 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:55:07.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:06 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:55:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:06 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:55:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:06 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 -- 172.21.15.60:0/1520221733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4109b40 msgr2=0x7f57f4109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 --2- 172.21.15.60:0/1520221733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4109b40 0x7f57f4109f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009ca0 tx=0x7f57ec02a8d0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 -- 172.21.15.60:0/1520221733 shutdown_connections 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 --2- 172.21.15.60:0/1520221733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f4115400 0x7f57f41177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 --2- 172.21.15.60:0/1520221733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f410a460 0x7f57f4114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 --2- 172.21.15.60:0/1520221733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4109b40 0x7f57f4109f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.898+0000 7f57f999f640 1 -- 172.21.15.60:0/1520221733 >> 172.21.15.60:0/1520221733 conn(0x7f57f406d760 msgr2=0x7f57f406db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.899+0000 7f57f999f640 1 -- 172.21.15.60:0/1520221733 shutdown_connections 2024-07-24T21:55:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.901+0000 7f57f999f640 1 -- 172.21.15.60:0/1520221733 wait complete. 2024-07-24T21:55:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.902+0000 7f57f999f640 1 Processor -- start 2024-07-24T21:55:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.902+0000 7f57f999f640 1 -- start start 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.902+0000 7f57f999f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f4109b40 0x7f57f4118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.902+0000 7f57f999f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.902+0000 7f57f999f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4115400 0x7f57f41191d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f999f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f57f411d1a0 con 0x7f57f4109b40 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f999f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f411d020 con 0x7f57f410a460 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:33790/0 (socket says 172.21.15.60:33790) 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 -- 172.21.15.60:0/3325351787 learned_addr learned my addr 172.21.15.60:0/3325351787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4115400 0x7f57f41191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57f411d320 con 0x7f57f4115400 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 -- 172.21.15.60:0/3325351787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4115400 msgr2=0x7f57f41191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4115400 0x7f57f41191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 -- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f4109b40 msgr2=0x7f57f4118750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.903+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f4109b40 0x7f57f4118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.904+0000 7f57f27fc640 1 -- 172.21.15.60:0/3325351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f41b3d90 con 0x7f57f410a460 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.904+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57e800b4a0 tx=0x7f57e800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.904+0000 7f57f899d640 1 -- 172.21.15.60:0/3325351787 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8014070 con 0x7f57f410a460 2024-07-24T21:55:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.905+0000 7f57f899d640 1 -- 172.21.15.60:0/3325351787 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e8010ac0 con 0x7f57f410a460 2024-07-24T21:55:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.905+0000 7f57f899d640 1 -- 172.21.15.60:0/3325351787 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017c30 con 0x7f57f410a460 2024-07-24T21:55:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.906+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f41b4080 con 0x7f57f410a460 2024-07-24T21:55:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.906+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f41b45d0 con 0x7f57f410a460 2024-07-24T21:55:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.911+0000 7f57fc7cb740 1 Processor -- start 2024-07-24T21:55:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.913+0000 7f57fc7cb740 1 -- start start 2024-07-24T21:55:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57f899d640 1 -- 172.21.15.60:0/3325351787 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f57e8017070 con 0x7f57f410a460 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ade0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c64ad56280 con 0x55c64ade08c0 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c64ad45780 con 0x55c64ade11e0 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c64ad497c0 con 0x55c64aded170 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:33794/0 (socket says 172.21.15.60:33794) 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.914+0000 7f57f2ffd640 1 -- 172.21.15.60:0/3368696276 learned_addr learned my addr 172.21.15.60:0/3368696276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.916+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ade0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.917+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.918+0000 7f57f2ffd640 1 -- 172.21.15.60:0/3368696276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 msgr2=0x55c64adef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.918+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.918+0000 7f57f2ffd640 1 -- 172.21.15.60:0/3368696276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 msgr2=0x55c64ade0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.919+0000 7f57f899d640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 0x7f57dc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.919+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ade0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.919+0000 7f57f899d640 1 -- 172.21.15.60:0/3325351787 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8609+0+0 (secure 0 0 0) 0x7f57e80e3260 con 0x7f57f410a460 2024-07-24T21:55:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.919+0000 7f57f2ffd640 1 -- 172.21.15.60:0/3368696276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64aaa9790 con 0x55c64ade11e0 2024-07-24T21:55:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.921+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009dc0 tx=0x7f57ec009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de0334f5d12f0f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.921+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 0x7f57dc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.921+0000 7f57d9ffb640 1 -- 172.21.15.60:0/3368696276 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec031d00 con 0x55c64ade11e0 2024-07-24T21:55:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.922+0000 7f57d9ffb640 1 -- 172.21.15.60:0/3368696276 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ec03f070 con 0x55c64ade11e0 2024-07-24T21:55:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.922+0000 7f57d9ffb640 1 -- 172.21.15.60:0/3368696276 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0042a0 con 0x55c64ade11e0 2024-07-24T21:55:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.924+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3368696276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 msgr2=0x55c64adecc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.924+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009dc0 tx=0x7f57ec009830 comp rx=0 tx=0).stop 2024-07-24T21:55:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.925+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3368696276 shutdown_connections 2024-07-24T21:55:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.925+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.925+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade11e0 0x55c64adecc30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.925+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3368696276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ade0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.925+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3368696276 >> 172.21.15.60:0/3368696276 conn(0x55c64adde6f0 msgr2=0x55c64addf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.926+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3368696276 shutdown_connections 2024-07-24T21:55:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.927+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3368696276 wait complete. 2024-07-24T21:55:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.927+0000 7f57fc7cb740 1 -- start start 2024-07-24T21:55:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.927+0000 7f57fc7cb740 1 Processor -- start 2024-07-24T21:55:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64adec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c64ac7fab0 con 0x55c64ade11e0 2024-07-24T21:55:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c64accd9e0 con 0x55c64ade08c0 2024-07-24T21:55:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c64adefdf0 con 0x55c64aded170 2024-07-24T21:55:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57fc7cb740 10 client.0 _ll_register_callbacks cb 0x55c64aa55a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59708/0 (socket says 172.21.15.60:59708) 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.928+0000 7f57f2ffd640 1 -- 172.21.15.60:0/1439047574 learned_addr learned my addr 172.21.15.60:0/1439047574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f37fe640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64adec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f37fe640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f2ffd640 1 -- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 msgr2=0x55c64adea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f2ffd640 1 -- 172.21.15.60:0/1439047574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 msgr2=0x55c64adec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64adec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.929+0000 7f57f2ffd640 1 -- 172.21.15.60:0/1439047574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64ab22620 con 0x55c64ade11e0 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.930+0000 7f57f37fe640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.930+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000f80 tx=0x7f57ec031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad604667e1de5118 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.930+0000 7f57d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.931+0000 7f57f27fc640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 0x7f57dc0b63d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57e8012040 tx=0x7f57e8010f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.932+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec02f820 con 0x55c64ade11e0 2024-07-24T21:55:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.932+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8609+0+0 (secure 0 0 0) 0x7f57ec02f9c0 con 0x55c64ade11e0 2024-07-24T21:55:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.934+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c64acf4ce0 con 0x55c64ade11e0 2024-07-24T21:55:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.937+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f57ec003970 con 0x55c64ade11e0 2024-07-24T21:55:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.937+0000 7f57fc7cb740 10 client.17277 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.937+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c64adfb570 con 0x55c64ade11e0 2024-07-24T21:55:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.939+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c64acf4ce0 con 0x55c64ade11e0 2024-07-24T21:55:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f57ec030d20 con 0x55c64ade11e0 2024-07-24T21:55:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 10 client.17277 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 20 client.17277 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64adfc8c0 0x55c64adfecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 4 client.17277 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.940+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c64ad581d0 con 0x55c64adfc8c0 2024-07-24T21:55:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.941+0000 7f57f37fe640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64adfc8c0 0x55c64adfecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.943+0000 7f57d2ffd640 10 client.17277 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.943+0000 7f57f37fe640 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64adfc8c0 0x55c64adfecb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a0aef827fd270c server_cookie=cdf504ed6a5c58e7 in_seq=0 out_seq=0 2024-07-24T21:55:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.944+0000 7f57d2ffd640 1 -- 172.21.15.60:0/1439047574 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c64ad581d0 con 0x55c64adfc8c0 2024-07-24T21:55:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57d2ffd640 10 client.17277 handle_command_reply: tid=0 2024-07-24T21:55:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 client.17277 shutdown 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 msgr2=0x55c64ade9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000f80 tx=0x7f57ec031690 comp rx=0 tx=0).stop 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 shutdown_connections 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64adfc8c0 0x55c64adfecb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade11e0 0x55c64ade9d60 secure :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000f80 tx=0x7f57ec031690 comp rx=0 tx=0).stop 2024-07-24T21:55:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/1439047574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64adec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 >> 172.21.15.60:0/1439047574 conn(0x55c64adde6f0 msgr2=0x55c64ade4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.945+0000 7f57d2ffd640 0 client.17277 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.950+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 shutdown_connections 2024-07-24T21:55:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.951+0000 7f57fc7cb740 1 -- 172.21.15.60:0/1439047574 wait complete. 2024-07-24T21:55:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.951+0000 7f57fc7cb740 20 client.17277 trim_cache size 0 max 16384 2024-07-24T21:55:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.981+0000 7f57fc7cb740 1 Processor -- start 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.982+0000 7f57fc7cb740 1 -- start start 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.982+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.982+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.982+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.982+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:33820/0 (socket says 172.21.15.60:33820) 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57f37fe640 1 -- 172.21.15.60:0/2781077829 learned_addr learned my addr 172.21.15.60:0/2781077829 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64addfbf0 0x55c64ade6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bfb20 con 0x55c64addfbf0 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bd360 con 0x55c64ade08c0 2024-07-24T21:55:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bb1d0 con 0x55c64aded170 2024-07-24T21:55:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.983+0000 7f57f37fe640 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64addfbf0 0x55c64ade6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 -- 172.21.15.60:0/2781077829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 msgr2=0x55c64adea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 -- 172.21.15.60:0/2781077829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64addfbf0 msgr2=0x55c64ade6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64addfbf0 0x55c64ade6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 -- 172.21.15.60:0/2781077829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64ade1010 con 0x55c64ade08c0 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57f37fe640 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009ab0 tx=0x7f57e4009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79157a4cfe042b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57d97fa640 1 -- 172.21.15.60:0/2781077829 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4005c50 con 0x55c64ade08c0 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57d97fa640 1 -- 172.21.15.60:0/2781077829 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e401c030 con 0x55c64ade08c0 2024-07-24T21:55:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.984+0000 7f57d97fa640 1 -- 172.21.15.60:0/2781077829 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40167c0 con 0x55c64ade08c0 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.985+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 msgr2=0x55c64ade9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.985+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009ab0 tx=0x7f57e4009060 comp rx=0 tx=0).stop 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.985+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 shutdown_connections 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.985+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64addfbf0 0x55c64ade6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.986+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64adea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.986+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/2781077829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64ade08c0 0x55c64ade9d60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.986+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 >> 172.21.15.60:0/2781077829 conn(0x55c64adde6f0 msgr2=0x55c64ade85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.988+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 shutdown_connections 2024-07-24T21:55:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.988+0000 7f57fc7cb740 1 -- 172.21.15.60:0/2781077829 wait complete. 2024-07-24T21:55:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.989+0000 7f57fc7cb740 1 -- start start 2024-07-24T21:55:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.989+0000 7f57fc7cb740 1 Processor -- start 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ae01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64ae65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bb820 con 0x55c64ade08c0 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bcc90 con 0x55c64addfbf0 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:33826/0 (socket says 172.21.15.60:33826) 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57f37fe640 1 -- 172.21.15.60:0/3119293052 learned_addr learned my addr 172.21.15.60:0/3119293052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41bbef0 con 0x55c64aded170 2024-07-24T21:55:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57fc7cb740 10 client.0 _ll_register_callbacks cb 0x7f57e80e6b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ae01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.990+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64ae65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 -- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 msgr2=0x55c64ae65200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64ae65200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 -- 172.21.15.60:0/3119293052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 msgr2=0x55c64ae01470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ae01470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 -- 172.21.15.60:0/3119293052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64ae65840 con 0x55c64addfbf0 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57f37fe640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009dc0 tx=0x7f57e4009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70074d3b9ea380db server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.991+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4032020 con 0x55c64addfbf0 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.992+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e401c030 con 0x55c64addfbf0 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.992+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8609+0+0 (secure 0 0 0) 0x7f57e402ca60 con 0x55c64addfbf0 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.992+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f57e402cc80 con 0x55c64addfbf0 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.992+0000 7f57fc7cb740 10 client.26693 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.992+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c64ae66b10 con 0x55c64addfbf0 2024-07-24T21:55:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.993+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e402ce20 con 0x55c64addfbf0 2024-07-24T21:55:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.994+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f57e4015040 con 0x55c64addfbf0 2024-07-24T21:55:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.994+0000 7f57fc7cb740 10 client.26693 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.994+0000 7f57fc7cb740 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.995+0000 7f57fc7cb740 20 client.26693 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.995+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64ae671c0 0x55c64ae695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.995+0000 7f57fc7cb740 4 client.26693 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:55:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.995+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f57e401c030 con 0x55c64ae671c0 2024-07-24T21:55:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:07.997+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64ae671c0 0x55c64ae695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:08.003+0000 7f57f2ffd640 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64ae671c0 0x55c64ae695e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244573ab8644da94 server_cookie=8fcedb086be5164 in_seq=0 out_seq=0 2024-07-24T21:55:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:08.003+0000 7f57d2ffd640 10 client.26693 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:07 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:55:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:07 smithi060 ceph-mon[33000]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 3.14k op/s 2024-07-24T21:55:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:07 smithi060 ceph-mon[33000]: osdmap e93: 12 total, 12 up, 12 in 2024-07-24T21:55:08.196 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:07 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:55:08.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:07 smithi086 ceph-mon[37128]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 3.14k op/s 2024-07-24T21:55:08.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:07 smithi086 ceph-mon[37128]: osdmap e93: 12 total, 12 up, 12 in 2024-07-24T21:55:08.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:07 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:55:08.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:07 smithi123 ceph-mon[37143]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 3.14k op/s 2024-07-24T21:55:08.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:07 smithi123 ceph-mon[37143]: osdmap e93: 12 total, 12 up, 12 in 2024-07-24T21:55:10.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:09 smithi060 ceph-mon[33000]: pgmap v637: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 1.32k op/s 2024-07-24T21:55:10.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:09 smithi123 ceph-mon[37143]: pgmap v637: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 1.32k op/s 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.277+0000 7f57d2ffd640 1 -- 172.21.15.60:0/3119293052 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f57e401c030 con 0x55c64ae671c0 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.277+0000 7f57d2ffd640 10 client.26693 handle_command_reply: tid=0 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.277+0000 7f57fc7cb740 1 client.26693 shutdown 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 msgr2=0x55c64ae00f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009dc0 tx=0x7f57e4009a10 comp rx=0 tx=0).stop 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 shutdown_connections 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c64ae671c0 0x55c64ae695e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57d2ffd640 0 client.26693 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c64aded170 0x55c64ae65200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c64ade08c0 0x55c64ae01470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 --2- 172.21.15.60:0/3119293052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c64addfbf0 0x55c64ae00f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 >> 172.21.15.60:0/3119293052 conn(0x55c64adde6f0 msgr2=0x55c64ade8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 shutdown_connections 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 1 -- 172.21.15.60:0/3119293052 wait complete. 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.278+0000 7f57fc7cb740 20 client.26693 trim_cache size 0 max 16384 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:55:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:55:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.279+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 msgr2=0x7f57dc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.279+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 0x7f57dc0b63d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57e8012040 tx=0x7f57e8010f90 comp rx=0 tx=0).stop 2024-07-24T21:55:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.279+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 msgr2=0x7f57f4118c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.279+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f57e800b4a0 tx=0x7f57e800b960 comp rx=0 tx=0).stop 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 shutdown_connections 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f57dc0b3f10 0x7f57dc0b63d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f4115400 0x7f57f41191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f410a460 0x7f57f4118c90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 --2- 172.21.15.60:0/3325351787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f4109b40 0x7f57f4118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.280+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 >> 172.21.15.60:0/3325351787 conn(0x7f57f406d760 msgr2=0x7f57f40723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.281+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 shutdown_connections 2024-07-24T21:55:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:10.281+0000 7f57f999f640 1 -- 172.21.15.60:0/3325351787 wait complete. 2024-07-24T21:55:10.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:09 smithi086 ceph-mon[37128]: pgmap v637: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 1.32k op/s 2024-07-24T21:55:12.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:11 smithi060 ceph-mon[33000]: pgmap v638: 97 pgs: 12 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 1.01k op/s 2024-07-24T21:55:12.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:11 smithi123 ceph-mon[37143]: pgmap v638: 97 pgs: 12 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 1.01k op/s 2024-07-24T21:55:12.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:11 smithi086 ceph-mon[37128]: pgmap v638: 97 pgs: 12 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 1.01k op/s 2024-07-24T21:55:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:13 smithi060 ceph-mon[33000]: pgmap v639: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-07-24T21:55:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:13 smithi123 ceph-mon[37143]: pgmap v639: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-07-24T21:55:14.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:13 smithi086 ceph-mon[37128]: pgmap v639: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-07-24T21:55:16.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:15 smithi060 ceph-mon[33000]: pgmap v640: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 503 op/s 2024-07-24T21:55:16.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:15 smithi123 ceph-mon[37143]: pgmap v640: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 503 op/s 2024-07-24T21:55:16.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:15 smithi086 ceph-mon[37128]: pgmap v640: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 503 op/s 2024-07-24T21:55:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:17 smithi060 ceph-mon[33000]: pgmap v641: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 292 op/s 2024-07-24T21:55:18.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:17 smithi123 ceph-mon[37143]: pgmap v641: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 292 op/s 2024-07-24T21:55:18.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:17 smithi086 ceph-mon[37128]: pgmap v641: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 292 op/s 2024-07-24T21:55:20.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:19 smithi060 ceph-mon[33000]: pgmap v642: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-07-24T21:55:20.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:19 smithi123 ceph-mon[37143]: pgmap v642: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-07-24T21:55:20.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:19 smithi086 ceph-mon[37128]: pgmap v642: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-07-24T21:55:22.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:21 smithi123 ceph-mon[37143]: pgmap v643: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-07-24T21:55:22.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:21 smithi086 ceph-mon[37128]: pgmap v643: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-07-24T21:55:22.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:21 smithi060 ceph-mon[33000]: pgmap v643: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 -- 172.21.15.60:0/193703087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 msgr2=0x7f123810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 --2- 172.21.15.60:0/193703087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f123810ee70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1230009900 tx=0x7f123002a410 comp rx=0 tx=0).stop 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 -- 172.21.15.60:0/193703087 shutdown_connections 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 --2- 172.21.15.60:0/193703087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f123810ee70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 --2- 172.21.15.60:0/193703087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1238103ef0 0x7f123810c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 --2- 172.21.15.60:0/193703087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12381035d0 0x7f12381039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 -- 172.21.15.60:0/193703087 >> 172.21.15.60:0/193703087 conn(0x7f12380782e0 msgr2=0x7f12380ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 -- 172.21.15.60:0/193703087 shutdown_connections 2024-07-24T21:55:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.383+0000 7f123c979640 1 -- 172.21.15.60:0/193703087 wait complete. 2024-07-24T21:55:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 Processor -- start 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 -- start start 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 0x7f123819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1238103ef0 0x7f123819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1238112660 con 0x7f123810c9f0 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12381124e0 con 0x7f12381035d0 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123c979640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12381127e0 con 0x7f1238103ef0 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f1236d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f1236d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53842/0 (socket says 172.21.15.60:53842) 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f1236d9d640 1 -- 172.21.15.60:0/4142873109 learned_addr learned my addr 172.21.15.60:0/4142873109 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f1235d9b640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1238103ef0 0x7f123819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.384+0000 7f123659c640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 0x7f123819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 -- 172.21.15.60:0/4142873109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1238103ef0 msgr2=0x7f123819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1238103ef0 0x7f123819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 -- 172.21.15.60:0/4142873109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 msgr2=0x7f123819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 0x7f123819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 -- 172.21.15.60:0/4142873109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381a0e80 con 0x7f123810c9f0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f123659c640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 0x7f123819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f1236d9d640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1230009230 tx=0x7f1230004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f122f7fe640 1 -- 172.21.15.60:0/4142873109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123003f030 con 0x7f123810c9f0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f122f7fe640 1 -- 172.21.15.60:0/4142873109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1230003aa0 con 0x7f123810c9f0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381a1110 con 0x7f123810c9f0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f122f7fe640 1 -- 172.21.15.60:0/4142873109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230035070 con 0x7f123810c9f0 2024-07-24T21:55:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.385+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a7830 con 0x7f123810c9f0 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f122f7fe640 1 -- 172.21.15.60:0/4142873109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1230042030 con 0x7f123810c9f0 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 Processor -- start 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 -- start start 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffa8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55560ff1e280 con 0x55560ffb5170 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55560ff0d780 con 0x55560ffa91e0 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123f7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55560ff117c0 con 0x55560ffa88c0 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f123659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53848/0 (socket says 172.21.15.60:53848) 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f1235d9b640 1 -- 172.21.15.60:0/288231385 learned_addr learned my addr 172.21.15.60:0/288231385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.388+0000 7f1235d9b640 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffa8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f122f7fe640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 0x7f12240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 -- 172.21.15.60:0/288231385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 msgr2=0x55560ffa8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffa8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 -- 172.21.15.60:0/288231385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 msgr2=0x55560ffb4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 -- 172.21.15.60:0/288231385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560fc71790 con 0x55560ffb5170 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1236d9d640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 0x7f12240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f1235d9b640 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1220013910 tx=0x7f122000e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc3b4499fd71d4c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f122f7fe640 1 -- 172.21.15.60:0/4142873109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f1230104870 con 0x7f123810c9f0 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f122cff9640 1 -- 172.21.15.60:0/288231385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220004030 con 0x55560ffb5170 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f122cff9640 1 -- 172.21.15.60:0/288231385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122000bda0 con 0x55560ffb5170 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.389+0000 7f122cff9640 1 -- 172.21.15.60:0/288231385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220003830 con 0x55560ffb5170 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f1236d9d640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 0x7f12240b6200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12300cb150 tx=0x7f1230098ba0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 -- 172.21.15.60:0/288231385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 msgr2=0x55560ffb7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1220013910 tx=0x7f122000e430 comp rx=0 tx=0).stop 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 -- 172.21.15.60:0/288231385 shutdown_connections 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb7560 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 --2- 172.21.15.60:0/288231385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffa8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 -- 172.21.15.60:0/288231385 >> 172.21.15.60:0/288231385 conn(0x55560ffa66f0 msgr2=0x55560ffa7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 -- 172.21.15.60:0/288231385 shutdown_connections 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.390+0000 7f123f7ab740 1 -- 172.21.15.60:0/288231385 wait complete. 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 -- start start 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 Processor -- start 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffb48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55560fe47ab0 con 0x55560ffb5170 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55560fe959e0 con 0x55560ffa91e0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55560ffb7e50 con 0x55560ffa88c0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123f7ab740 10 client.0 _ll_register_callbacks cb 0x55560fc1da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53850/0 (socket says 172.21.15.60:53850) 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f1235d9b640 1 -- 172.21.15.60:0/3665274580 learned_addr learned my addr 172.21.15.60:0/3665274580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f123659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.391+0000 7f1235d9b640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffb48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 -- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 msgr2=0x55560ffb48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffb48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 -- 172.21.15.60:0/3665274580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 msgr2=0x55560ffb22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 -- 172.21.15.60:0/3665274580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560fcea620 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123659c640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb1d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1228005fd0 tx=0x7f1228012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989501ffdb1afc0e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228003d80 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55560febcce0 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f12280109b0 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1228010bb0 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123f7ab740 10 client.26699 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55560ffc3680 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.392+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55560febcce0 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f1228022070 con 0x55560ffa91e0 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 10 client.26699 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 10 client.26699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 20 client.26699 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55560ffc4b60 0x55560ffc6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 4 client.26699 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55560ff201d0 con 0x55560ffc4b60 2024-07-24T21:55:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f1235d9b640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55560ffc4b60 0x55560ffc6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f121dffb640 10 client.26699 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.393+0000 7f1235d9b640 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55560ffc4b60 0x55560ffc6f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f369e88b0ea024ff server_cookie=1e69ff0b88273e52 in_seq=0 out_seq=0 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.394+0000 7f121dffb640 1 -- 172.21.15.60:0/3665274580 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55560ff201d0 con 0x55560ffc4b60 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.394+0000 7f121dffb640 10 client.26699 handle_command_reply: tid=0 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.394+0000 7f123f7ab740 1 client.26699 shutdown 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 msgr2=0x55560ffb1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb1d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1228005fd0 tx=0x7f1228012b60 comp rx=0 tx=0).stop 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 shutdown_connections 2024-07-24T21:55:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55560ffc4b60 0x55560ffc6f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55560ffb22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffb1d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 --2- 172.21.15.60:0/3665274580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffb48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f121dffb640 0 client.26699 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 >> 172.21.15.60:0/3665274580 conn(0x55560ffa66f0 msgr2=0x55560ffac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 shutdown_connections 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 1 -- 172.21.15.60:0/3665274580 wait complete. 2024-07-24T21:55:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.395+0000 7f123f7ab740 20 client.26699 trim_cache size 0 max 16384 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 Processor -- start 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 -- start start 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffcc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffb5170 0x55560ffccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b2ca0 con 0x55560ffa88c0 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b04e0 con 0x55560ffa91e0 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.410+0000 7f123f7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ae350 con 0x55560ffb5170 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53862/0 (socket says 172.21.15.60:53862) 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 -- 172.21.15.60:0/1476071005 learned_addr learned my addr 172.21.15.60:0/1476071005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f123659c640 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffcc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 -- 172.21.15.60:0/1476071005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffb5170 msgr2=0x55560ffccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffb5170 0x55560ffccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 -- 172.21.15.60:0/1476071005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 msgr2=0x55560ffcc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffcc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 -- 172.21.15.60:0/1476071005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560ffa9010 con 0x55560ffa88c0 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f123659c640 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffcc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f1235d9b640 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f122000b6b0 tx=0x7f122000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c30a9658cedaa3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f121ffff640 1 -- 172.21.15.60:0/1476071005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220013250 con 0x55560ffa88c0 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f121ffff640 1 -- 172.21.15.60:0/1476071005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122000bda0 con 0x55560ffa88c0 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.411+0000 7f121ffff640 1 -- 172.21.15.60:0/1476071005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122002e850 con 0x55560ffa88c0 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- 172.21.15.60:0/1476071005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 msgr2=0x55560ffc0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f122000b6b0 tx=0x7f122000eb70 comp rx=0 tx=0).stop 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- 172.21.15.60:0/1476071005 shutdown_connections 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffb5170 0x55560ffccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffcc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 --2- 172.21.15.60:0/1476071005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffa88c0 0x55560ffc0f70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- 172.21.15.60:0/1476071005 >> 172.21.15.60:0/1476071005 conn(0x55560ffa66f0 msgr2=0x55560ffa57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- 172.21.15.60:0/1476071005 shutdown_connections 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- 172.21.15.60:0/1476071005 wait complete. 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 -- start start 2024-07-24T21:55:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.412+0000 7f123f7ab740 1 Processor -- start 2024-07-24T21:55:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffbad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ae9a0 con 0x55560ffb5170 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12381afe10 con 0x55560ffa91e0 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12381af070 con 0x55560ffa88c0 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123f7ab740 10 client.0 _ll_register_callbacks cb 0x7f12280099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53864/0 (socket says 172.21.15.60:53864) 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 -- 172.21.15.60:0/2061052820 learned_addr learned my addr 172.21.15.60:0/2061052820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f123659c640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffbad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 -- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 msgr2=0x55560ffba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 -- 172.21.15.60:0/2061052820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 msgr2=0x55560ffbad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffbad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 -- 172.21.15.60:0/2061052820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561002dfb0 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f1235d9b640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1220004590 tx=0x7f122002e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d33fcfbebb5d55 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.413+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220032040 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f122000b8f0 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122000bda0 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f122000baf0 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 10 client.17301 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55561002f2a0 con 0x55560ffb5170 2024-07-24T21:55:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122000bda0 con 0x55560ffb5170 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f1220038030 con 0x55560ffb5170 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 10 client.17301 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 20 client.17301 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55561002f8f0 0x555610031d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 4 client.17301 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55560ff1bc70 con 0x55561002f8f0 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.414+0000 7f123659c640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55561002f8f0 0x555610031d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.415+0000 7f121dffb640 10 client.17301 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:23.415+0000 7f123659c640 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55561002f8f0 0x555610031d10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7744389662d9b34 server_cookie=98203b79f7c306da in_seq=0 out_seq=0 2024-07-24T21:55:23.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:55:24.094 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:24.216 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:23 smithi060 ceph-mon[33000]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-07-24T21:55:24.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:23 smithi060 ceph-mon[33000]: osdmap e94: 12 total, 12 up, 12 in 2024-07-24T21:55:24.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:23 smithi123 ceph-mon[37143]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-07-24T21:55:24.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:23 smithi123 ceph-mon[37143]: osdmap e94: 12 total, 12 up, 12 in 2024-07-24T21:55:24.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:23 smithi086 ceph-mon[37128]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-07-24T21:55:24.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:23 smithi086 ceph-mon[37128]: osdmap e94: 12 total, 12 up, 12 in 2024-07-24T21:55:26.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:25 smithi060 ceph-mon[33000]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-07-24T21:55:26.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:25 smithi123 ceph-mon[37143]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-07-24T21:55:26.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:25 smithi086 ceph-mon[37128]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 288 op/s 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.304+0000 7f121dffb640 1 -- 172.21.15.60:0/2061052820 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55560ff1bc70 con 0x55561002f8f0 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.304+0000 7f121dffb640 10 client.17301 handle_command_reply: tid=0 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.304+0000 7f123f7ab740 1 client.17301 shutdown 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 msgr2=0x55561002d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1220004590 tx=0x7f122002e5d0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 shutdown_connections 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55561002f8f0 0x555610031d10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55560ffb5170 0x55561002d870 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55560ffa91e0 0x55560ffbad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 --2- 172.21.15.60:0/2061052820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55560ffa88c0 0x55560ffba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 >> 172.21.15.60:0/2061052820 conn(0x55560ffa66f0 msgr2=0x55560ffb31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f121dffb640 0 client.17301 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 shutdown_connections 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 1 -- 172.21.15.60:0/2061052820 wait complete. 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.305+0000 7f123f7ab740 20 client.17301 trim_cache size 0 max 16384 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:55:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:55:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 msgr2=0x7f12240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 0x7f12240b6200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12300cb150 tx=0x7f1230098ba0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 msgr2=0x7f12381a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1230009230 tx=0x7f1230004270 comp rx=0 tx=0).stop 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 shutdown_connections 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12240b3d40 0x7f12240b6200 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f123810c9f0 0x7f12381a06b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1238103ef0 0x7f123819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 --2- 172.21.15.60:0/4142873109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12381035d0 0x7f123819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 >> 172.21.15.60:0/4142873109 conn(0x7f12380782e0 msgr2=0x7f1238101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 shutdown_connections 2024-07-24T21:55:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:26.306+0000 7f123c979640 1 -- 172.21.15.60:0/4142873109 wait complete. 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.314+0000 7f5d3cff9640 1 -- 172.21.15.60:0/3595170577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 msgr2=0x7f5d3810e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.314+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/3595170577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3810e8e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009900 tx=0x7f5d2c02a480 comp rx=0 tx=0).stop 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 -- 172.21.15.60:0/3595170577 shutdown_connections 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/3595170577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d38111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/3595170577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3810e8e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/3595170577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d38101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 -- 172.21.15.60:0/3595170577 >> 172.21.15.60:0/3595170577 conn(0x7f5d380fd0c0 msgr2=0x7f5d380ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 -- 172.21.15.60:0/3595170577 shutdown_connections 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 -- 172.21.15.60:0/3595170577 wait complete. 2024-07-24T21:55:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.315+0000 7f5d3cff9640 1 Processor -- start 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 -- start start 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d3819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d38114930 con 0x7f5d38101b90 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381147b0 con 0x7f5d3810ee20 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3cff9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d38114ab0 con 0x7f5d38101270 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d35d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d36d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d36d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39518/0 (socket says 172.21.15.60:39518) 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d35d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39688/0 (socket says 172.21.15.60:39688) 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d36d9d640 1 -- 172.21.15.60:0/4036074826 learned_addr learned my addr 172.21.15.60:0/4036074826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d3819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 -- 172.21.15.60:0/4036074826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 msgr2=0x7f5d381a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 -- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 msgr2=0x7f5d3819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d3659c640 1 -- 172.21.15.60:0/4036074826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d381a0f10 con 0x7f5d38101270 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.316+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d3819ae60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d277fe640 1 -- 172.21.15.60:0/4036074826 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800ccc0 con 0x7f5d38101270 2024-07-24T21:55:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d381a1200 con 0x7f5d38101270 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d277fe640 1 -- 172.21.15.60:0/4036074826 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d28005020 con 0x7f5d38101270 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d277fe640 1 -- 172.21.15.60:0/4036074826 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004a00 con 0x7f5d38101270 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.317+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381a78c0 con 0x7f5d38101270 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 Processor -- start 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d277fe640 1 -- 172.21.15.60:0/4036074826 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5d28004ba0 con 0x7f5d38101270 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 -- start start 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49680 0x564437e49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e58320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564437dc0960 con 0x564437e49fa0 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564437dafe60 con 0x564437e49680 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564437db3ea0 con 0x564437e55f30 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.320+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39704/0 (socket says 172.21.15.60:39704) 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 -- 172.21.15.60:0/1445044466 learned_addr learned my addr 172.21.15.60:0/1445044466 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e58320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49680 0x564437e49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 -- 172.21.15.60:0/1445044466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 msgr2=0x564437e58320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e58320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 -- 172.21.15.60:0/1445044466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49680 msgr2=0x564437e49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49680 0x564437e49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 -- 172.21.15.60:0/1445044466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564437d392d0 con 0x564437e49fa0 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e58320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d277fe640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 0x7f5d200b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d3659c640 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5d280e1640 tx=0x7f5d280ec210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dc5bb1029f5d4b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 0x7f5d200b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d277fe640 1 -- 172.21.15.60:0/4036074826 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5d280e3fb0 con 0x7f5d38101270 2024-07-24T21:55:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d24ff9640 1 -- 172.21.15.60:0/1445044466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280a6a50 con 0x564437e49fa0 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d24ff9640 1 -- 172.21.15.60:0/1445044466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280aa6a0 con 0x564437e49fa0 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.321+0000 7f5d24ff9640 1 -- 172.21.15.60:0/1445044466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280aa300 con 0x564437e49fa0 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/1445044466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 msgr2=0x564437e559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 0x7f5d200b6200 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5d300049d0 tx=0x7f5d30009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5d280e1640 tx=0x7f5d280ec210 comp rx=0 tx=0).stop 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/1445044466 shutdown_connections 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e58320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49fa0 0x564437e559f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/1445044466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49680 0x564437e49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/1445044466 >> 172.21.15.60:0/1445044466 conn(0x564437e474b0 msgr2=0x564437e48650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/1445044466 shutdown_connections 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.322+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/1445044466 wait complete. 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 -- start start 2024-07-24T21:55:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 Processor -- start 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49fa0 0x564437e53410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e53950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564437c13700 con 0x564437e49680 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564437d525a0 con 0x564437e49fa0 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564437e58bb0 con 0x564437e55f30 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3f3aa740 10 client.0 _ll_register_callbacks cb 0x564437d72910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39708/0 (socket says 172.21.15.60:39708) 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d35d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49fa0 0x564437e53410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 -- 172.21.15.60:0/4260607363 learned_addr learned my addr 172.21.15.60:0/4260607363 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e53950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 -- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 msgr2=0x564437e53950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e53950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 -- 172.21.15.60:0/4260607363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49fa0 msgr2=0x564437e53410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.323+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49fa0 0x564437e53410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d3659c640 1 -- 172.21.15.60:0/4260607363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564437d45630 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d3659c640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5d28069440 tx=0x7f5d280096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa3bf117e1ea051 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2801c070 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5d280a7180 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d280aa6a0 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5d280aa4e0 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d3f3aa740 10 client.17313 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564437b10650 con 0x564437e49680 2024-07-24T21:55:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280aa6a0 con 0x564437e49680 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.324+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5d280920b0 con 0x564437e49680 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 10 client.17313 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 20 client.17313 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437e67ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 4 client.17313 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564437dc28b0 con 0x564437e658f0 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437e67ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437e67ce0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b1ca0fd3ccffc5 server_cookie=e4f3b77be55af92e in_seq=0 out_seq=0 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.325+0000 7f5d1dffb640 10 client.17313 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d1dffb640 1 -- 172.21.15.60:0/4260607363 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564437dc28b0 con 0x564437e658f0 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d1dffb640 10 client.17313 handle_command_reply: tid=0 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 client.17313 shutdown 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 msgr2=0x564437e52ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5d28069440 tx=0x7f5d280096e0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 shutdown_connections 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437e67ce0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d1dffb640 0 client.17313 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e55f30 0x564437e53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e49fa0 0x564437e53410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/4260607363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e52ed0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 >> 172.21.15.60:0/4260607363 conn(0x564437e474b0 msgr2=0x564437e4d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 shutdown_connections 2024-07-24T21:55:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/4260607363 wait complete. 2024-07-24T21:55:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.326+0000 7f5d3f3aa740 20 client.17313 trim_cache size 0 max 16384 2024-07-24T21:55:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 Processor -- start 2024-07-24T21:55:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 -- start start 2024-07-24T21:55:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e6b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381b2c20 con 0x564437e49680 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381b0460 con 0x564437e55f30 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381ae2d0 con 0x564437e49fa0 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39724/0 (socket says 172.21.15.60:39724) 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3659c640 1 -- 172.21.15.60:0/2341294907 learned_addr learned my addr 172.21.15.60:0/2341294907 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 -- 172.21.15.60:0/2341294907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 msgr2=0x564437e6b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e6b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 -- 172.21.15.60:0/2341294907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 msgr2=0x564437e54fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d35d9b640 1 -- 172.21.15.60:0/2341294907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564437e49dd0 con 0x564437e49fa0 2024-07-24T21:55:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.345+0000 7f5d3659c640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.346+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e6ad00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009230 tx=0x7f5d2c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a67e0c72a1821c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.346+0000 7f5d1ffff640 1 -- 172.21.15.60:0/2341294907 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c035040 con 0x564437e49fa0 2024-07-24T21:55:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.346+0000 7f5d1ffff640 1 -- 172.21.15.60:0/2341294907 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2c04b070 con 0x564437e49fa0 2024-07-24T21:55:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.346+0000 7f5d1ffff640 1 -- 172.21.15.60:0/2341294907 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0049d0 con 0x564437e49fa0 2024-07-24T21:55:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2341294907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 msgr2=0x564437e6ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e6ad00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009230 tx=0x7f5d2c005b50 comp rx=0 tx=0).stop 2024-07-24T21:55:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2341294907 shutdown_connections 2024-07-24T21:55:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e6b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e6ad00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2341294907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e54fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2341294907 >> 172.21.15.60:0/2341294907 conn(0x564437e474b0 msgr2=0x564437e53660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2341294907 shutdown_connections 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2341294907 wait complete. 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- start start 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 Processor -- start 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e68a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381ae920 con 0x564437e49680 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381afd90 con 0x564437e55f30 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.347+0000 7f5d3f3aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381aeff0 con 0x564437e49fa0 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3f3aa740 10 client.0 _ll_register_callbacks cb 0x7f5d28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d35d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e68a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39738/0 (socket says 172.21.15.60:39738) 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d35d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e68a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56716/0 (socket says 172.21.15.60:56716) 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 -- 172.21.15.60:0/2198364890 learned_addr learned my addr 172.21.15.60:0/2198364890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d36d9d640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 -- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 msgr2=0x564437e68a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e68a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 -- 172.21.15.60:0/2198364890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 msgr2=0x564437e69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 -- 172.21.15.60:0/2198364890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564437ecf230 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3659c640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5d280a6c10 tx=0x7f5d28009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10c5ec1c23faa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2801c040 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5d280aadf0 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2c04b070 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5d280aa960 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3f3aa740 10 client.17319 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.348+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564437ed0440 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2c04b070 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5d280a6500 con 0x564437e49680 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 10 client.17319 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 20 client.17319 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437ed2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 4 client.17319 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564437dbe350 con 0x564437e658f0 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437ed2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d35d9b640 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437ed2a30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130ffef6b218af5 server_cookie=4c178b948b3cab13 in_seq=0 out_seq=0 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.349+0000 7f5d1dffb640 10 client.17319 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d1dffb640 1 -- 172.21.15.60:0/2198364890 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564437dbe350 con 0x564437e658f0 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d1dffb640 10 client.17319 handle_command_reply: tid=0 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 client.17319 shutdown 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 msgr2=0x564437e6b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5d280a6c10 tx=0x7f5d28009f60 comp rx=0 tx=0).stop 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 shutdown_connections 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564437e658f0 0x564437ed2a30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564437e55f30 0x564437e69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d1dffb640 0 client.17319 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564437e49fa0 0x564437e68a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.350+0000 7f5d3f3aa740 1 --2- 172.21.15.60:0/2198364890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564437e49680 0x564437e6b530 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 >> 172.21.15.60:0/2198364890 conn(0x564437e474b0 msgr2=0x564437e51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 shutdown_connections 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3f3aa740 1 -- 172.21.15.60:0/2198364890 wait complete. 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3f3aa740 20 client.17319 trim_cache size 0 max 16384 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:55:26.357 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 msgr2=0x7f5d200b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 0x7f5d200b6200 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5d300049d0 tx=0x7f5d30009500 comp rx=0 tx=0).stop 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 msgr2=0x7f5d3819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d3819ae60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 shutdown_connections 2024-07-24T21:55:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d200b3d40 0x7f5d200b6200 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d3810ee20 0x7f5d381a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38101b90 0x7f5d3819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 --2- 172.21.15.60:0/4036074826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101270 0x7f5d3819ae60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.351+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 >> 172.21.15.60:0/4036074826 conn(0x7f5d380fd0c0 msgr2=0x7f5d381110c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.352+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 shutdown_connections 2024-07-24T21:55:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:26.352+0000 7f5d3cff9640 1 -- 172.21.15.60:0/4036074826 wait complete. 2024-07-24T21:55:27.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:55:27.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:55:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:27.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:27 smithi060 ceph-mon[33000]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-07-24T21:55:28.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:27 smithi123 ceph-mon[37143]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-07-24T21:55:28.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:27 smithi086 ceph-mon[37128]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-07-24T21:55:29.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 -- 172.21.15.60:0/3638346390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 msgr2=0x7f53e81074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:29.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/3638346390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e81074e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009900 tx=0x7f53dc02a410 comp rx=0 tx=0).stop 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 -- 172.21.15.60:0/3638346390 shutdown_connections 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/3638346390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 0x7f53e810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/3638346390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e81074e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/3638346390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e8101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 -- 172.21.15.60:0/3638346390 >> 172.21.15.60:0/3638346390 conn(0x7f53e80fd100 msgr2=0x7f53e80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 -- 172.21.15.60:0/3638346390 shutdown_connections 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.338+0000 7f53ee0a4640 1 -- 172.21.15.60:0/3638346390 wait complete. 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 Processor -- start 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 -- start start 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e80fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 0x7f53e80fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8111640 con 0x7f53e81070a0 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81114c0 con 0x7f53e8107a20 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53ee0a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81117c0 con 0x7f53e8101690 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e80fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56726/0 (socket says 172.21.15.60:56726) 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53e77fe640 1 -- 172.21.15.60:0/4163984089 learned_addr learned my addr 172.21.15.60:0/4163984089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.339+0000 7f53e7fff640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 0x7f53e80fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 -- 172.21.15.60:0/4163984089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 msgr2=0x7f53e80fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 0x7f53e80fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 -- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 msgr2=0x7f53e80fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e80fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 -- 172.21.15.60:0/4163984089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a70f0 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e77fe640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9f0 tx=0x7f53d800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d800cd80 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d80054d0 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8010690 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a73e0 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.340+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a78a0 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.341+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e8101a70 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.343+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f53d8004e30 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.343+0000 7f53e4ff9640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 0x7f53d00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.343+0000 7f53e6ffd640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 0x7f53d00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.343+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f53d80e46d0 con 0x7f53e8101690 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.344+0000 7f53e6ffd640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 0x7f53d00b6200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53dc031440 tx=0x7f53dc005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.346+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f53d80555e0 con 0x7f53e8101690 2024-07-24T21:55:29.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.653+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53e81a7c30 con 0x7f53e8101690 2024-07-24T21:55:29.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.654+0000 7f53e4ff9640 1 -- 172.21.15.60:0/4163984089 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11920 (secure 0 0 0) 0x7f53d80ab950 con 0x7f53e8101690 2024-07-24T21:55:29.656 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:55:29.656 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"btime":"2024-07-24T21:54:56:385204+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:54:55.586937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:55:29.656 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2024-07-24T21:55:29.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 msgr2=0x7f53d00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:29.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 0x7f53d00b6200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53dc031440 tx=0x7f53dc005b50 comp rx=0 tx=0).stop 2024-07-24T21:55:29.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 msgr2=0x7f53e80fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9f0 tx=0x7f53d800edf0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 shutdown_connections 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53d00b3d40 0x7f53d00b6200 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8107a20 0x7f53e80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81070a0 0x7f53e80fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 --2- 172.21.15.60:0/4163984089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53e8101690 0x7f53e80fd600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.656+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 >> 172.21.15.60:0/4163984089 conn(0x7f53e80fd100 msgr2=0x7f53e80fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.657+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 shutdown_connections 2024-07-24T21:55:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:29.657+0000 7f53ee0a4640 1 -- 172.21.15.60:0/4163984089 wait complete. 2024-07-24T21:55:29.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:29 smithi060 ceph-mon[33000]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-07-24T21:55:29.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4163984089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:55:30.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:29 smithi123 ceph-mon[37143]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-07-24T21:55:30.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4163984089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:55:30.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:29 smithi086 ceph-mon[37128]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-07-24T21:55:30.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4163984089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:55:30.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:55:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:32.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:31 smithi060 ceph-mon[33000]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 394 op/s 2024-07-24T21:55:32.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:31 smithi123 ceph-mon[37143]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 394 op/s 2024-07-24T21:55:32.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:31 smithi086 ceph-mon[37128]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 394 op/s 2024-07-24T21:55:32.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 -- 172.21.15.60:0/909397211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 msgr2=0x7f593c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 --2- 172.21.15.60:0/909397211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c102b50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f592c009900 tx=0x7f592c02a410 comp rx=0 tx=0).stop 2024-07-24T21:55:32.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 -- 172.21.15.60:0/909397211 shutdown_connections 2024-07-24T21:55:32.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 --2- 172.21.15.60:0/909397211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 0x7f593c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 --2- 172.21.15.60:0/909397211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 0x7f593c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 --2- 172.21.15.60:0/909397211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c102b50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 -- 172.21.15.60:0/909397211 >> 172.21.15.60:0/909397211 conn(0x7f593c0fe240 msgr2=0x7f593c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 -- 172.21.15.60:0/909397211 shutdown_connections 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.413+0000 7f5942930640 1 -- 172.21.15.60:0/909397211 wait complete. 2024-07-24T21:55:32.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 Processor -- start 2024-07-24T21:55:32.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 -- start start 2024-07-24T21:55:32.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 0x7f593c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 0x7f593c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f593c112710 con 0x7f593c102770 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f593c112590 con 0x7f593c108b00 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5942930640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f593c112890 con 0x7f593c108180 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39766/0 (socket says 172.21.15.60:39766) 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 -- 172.21.15.60:0/139299532 learned_addr learned my addr 172.21.15.60:0/139299532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5933fff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 0x7f593c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f5940ecd640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 0x7f593c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 -- 172.21.15.60:0/139299532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 msgr2=0x7f593c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 0x7f593c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 -- 172.21.15.60:0/139299532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 msgr2=0x7f593c19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 0x7f593c19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.414+0000 7f593bfff640 1 -- 172.21.15.60:0/139299532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19ecf0 con 0x7f593c102770 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f593bfff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f592c005940 tx=0x7f592c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f5939ffb640 1 -- 172.21.15.60:0/139299532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c03f030 con 0x7f593c102770 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f5939ffb640 1 -- 172.21.15.60:0/139299532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c003cb0 con 0x7f593c102770 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c19ef80 con 0x7f593c102770 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f5939ffb640 1 -- 172.21.15.60:0/139299532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c035070 con 0x7f593c102770 2024-07-24T21:55:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.415+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a56a0 con 0x7f593c102770 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.417+0000 7f5939ffb640 1 -- 172.21.15.60:0/139299532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f592c042030 con 0x7f593c102770 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.417+0000 7f5944ce1740 1 Processor -- start 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.417+0000 7f5944ce1740 1 -- start start 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65762a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65765370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562e656cd9b0 con 0x562e65762f80 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e656bceb0 con 0x562e65756ff0 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5944ce1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e656c0ef0 con 0x562e657566d0 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5933fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65765370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56742/0 (socket says 172.21.15.60:56742) 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 -- 172.21.15.60:0/3655298685 learned_addr learned my addr 172.21.15.60:0/3655298685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f593bfff640 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65762a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 -- 172.21.15.60:0/3655298685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 msgr2=0x562e65762a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65762a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 -- 172.21.15.60:0/3655298685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 msgr2=0x562e65765370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65765370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 -- 172.21.15.60:0/3655298685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e65646320 con 0x562e657566d0 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5939ffb640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 0x7f59280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.418+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5934009f70 tx=0x7f5934010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa9ce1bd98fae29 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f593bfff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 0x7f59280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f59327fc640 1 -- 172.21.15.60:0/3655298685 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934016070 con 0x562e657566d0 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f5939ffb640 1 -- 172.21.15.60:0/139299532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f592c104850 con 0x7f593c102770 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f59327fc640 1 -- 172.21.15.60:0/3655298685 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5934004d20 con 0x562e657566d0 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f59327fc640 1 -- 172.21.15.60:0/3655298685 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59340127d0 con 0x562e657566d0 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f5944ce1740 1 -- 172.21.15.60:0/3655298685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 msgr2=0x562e65756ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f593bfff640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 0x7f59280b6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f592c102020 tx=0x7f592c10a550 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.419+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5934009f70 tx=0x7f5934010800 comp rx=0 tx=0).stop 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 -- 172.21.15.60:0/3655298685 shutdown_connections 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65765370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65762a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3655298685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e65756ab0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 -- 172.21.15.60:0/3655298685 >> 172.21.15.60:0/3655298685 conn(0x562e65754500 msgr2=0x562e657556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 -- 172.21.15.60:0/3655298685 shutdown_connections 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 -- 172.21.15.60:0/3655298685 wait complete. 2024-07-24T21:55:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 -- start start 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.420+0000 7f5944ce1740 1 Processor -- start 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e6575fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65760960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562e65520700 con 0x562e65762f80 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e6565f5f0 con 0x562e65756ff0 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562e65765c00 con 0x562e657566d0 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5944ce1740 10 client.0 _ll_register_callbacks cb 0x562e6567f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39602/0 (socket says 172.21.15.60:39602) 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 -- 172.21.15.60:0/2334949580 learned_addr learned my addr 172.21.15.60:0/2334949580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5940ecd640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e6575fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5940ecd640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65760960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 -- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 msgr2=0x562e6575fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e6575fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 -- 172.21.15.60:0/2334949580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 msgr2=0x562e65760960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65760960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 -- 172.21.15.60:0/2334949580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e65652680 con 0x562e65756ff0 2024-07-24T21:55:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5940ecd640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65760960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f5933fff640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5924011470 tx=0x7f5924011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277c4a3203e5b441 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.421+0000 7f591b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924004020 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f59240041c0 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5934004d20 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f59240043c0 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 10 client.26741 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e6541d650 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5934004d20 con 0x562e65756ff0 2024-07-24T21:55:32.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5924003ab0 con 0x562e65756ff0 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 10 client.26741 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 10 client.26741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 20 client.26741 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e65772b10 0x562e65774f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 4 client.26741 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.422+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e656cf900 con 0x562e65772b10 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.423+0000 7f5940ecd640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e65772b10 0x562e65774f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.423+0000 7f5940ecd640 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e65772b10 0x562e65774f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c98ac08242ad56a server_cookie=ee0df1107266e697 in_seq=0 out_seq=0 2024-07-24T21:55:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.423+0000 7f591b7fe640 10 client.26741 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:55:32.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.424+0000 7f591b7fe640 1 -- 172.21.15.60:0/2334949580 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e656cf900 con 0x562e65772b10 2024-07-24T21:55:32.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.424+0000 7f591b7fe640 10 client.26741 handle_command_reply: tid=0 2024-07-24T21:55:32.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.424+0000 7f5944ce1740 1 client.26741 shutdown 2024-07-24T21:55:32.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 msgr2=0x562e65760420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5924011470 tx=0x7f5924011930 comp rx=0 tx=0).stop 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 shutdown_connections 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e65772b10 0x562e65774f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e65760960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e65756ff0 0x562e65760420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f591b7fe640 0 client.26741 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 --2- 172.21.15.60:0/2334949580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e657566d0 0x562e6575fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 >> 172.21.15.60:0/2334949580 conn(0x562e65754500 msgr2=0x562e6575a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 shutdown_connections 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 1 -- 172.21.15.60:0/2334949580 wait complete. 2024-07-24T21:55:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.425+0000 7f5944ce1740 20 client.26741 trim_cache size 0 max 16384 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 Processor -- start 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 -- start start 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e6577a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65762f80 0x562e6575e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1b0ad0 con 0x562e65756ff0 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1ae310 con 0x562e657566d0 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5944ce1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1ac180 con 0x562e65762f80 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5933fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5933fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39808/0 (socket says 172.21.15.60:39808) 2024-07-24T21:55:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.439+0000 7f5933fff640 1 -- 172.21.15.60:0/892967208 learned_addr learned my addr 172.21.15.60:0/892967208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5940ecd640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e6577a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5940ecd640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65762f80 0x562e6575e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 -- 172.21.15.60:0/892967208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65762f80 msgr2=0x562e6575e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65762f80 0x562e6575e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 -- 172.21.15.60:0/892967208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 msgr2=0x562e6577a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e6577a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 -- 172.21.15.60:0/892967208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e65775270 con 0x562e65756ff0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5940ecd640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e6577a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5933fff640 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5924003e70 tx=0x7f592401a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85a92df035a51ef server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5931ffb640 1 -- 172.21.15.60:0/892967208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924003ff0 con 0x562e65756ff0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5931ffb640 1 -- 172.21.15.60:0/892967208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5924004190 con 0x562e65756ff0 2024-07-24T21:55:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.440+0000 7f5931ffb640 1 -- 172.21.15.60:0/892967208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924005320 con 0x562e65756ff0 2024-07-24T21:55:32.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- 172.21.15.60:0/892967208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 msgr2=0x562e6577ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5924003e70 tx=0x7f592401a740 comp rx=0 tx=0).stop 2024-07-24T21:55:32.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- 172.21.15.60:0/892967208 shutdown_connections 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65762f80 0x562e6575e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65756ff0 0x562e6577ad50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 --2- 172.21.15.60:0/892967208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e6577a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- 172.21.15.60:0/892967208 >> 172.21.15.60:0/892967208 conn(0x562e65754500 msgr2=0x562e65761d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- 172.21.15.60:0/892967208 shutdown_connections 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- 172.21.15.60:0/892967208 wait complete. 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 -- start start 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.441+0000 7f5944ce1740 1 Processor -- start 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e657666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 0x562e65766c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1ac7d0 con 0x562e65762f80 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1adc40 con 0x562e657566d0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1acea0 con 0x562e65756ff0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5944ce1740 10 client.0 _ll_register_callbacks cb 0x7f592400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39818/0 (socket says 172.21.15.60:39818) 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 -- 172.21.15.60:0/3481504780 learned_addr learned my addr 172.21.15.60:0/3481504780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5933fff640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 0x562e65766c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e657666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 -- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 msgr2=0x562e65766c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 0x562e65766c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 -- 172.21.15.60:0/3481504780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 msgr2=0x562e657666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e657666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 -- 172.21.15.60:0/3481504780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6576b2d0 con 0x562e65762f80 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5933fff640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 0x562e65766c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f5940ecd640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5934009f70 tx=0x7f593402d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8251dce3908139 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.442+0000 7f591b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934016040 con 0x562e65762f80 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5924004190 con 0x562e65762f80 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5934003d20 con 0x562e65762f80 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f59340050f0 con 0x562e65762f80 2024-07-24T21:55:32.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 10 client.17343 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e657dd320 con 0x562e65762f80 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5924004190 con 0x562e65762f80 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5934028040 con 0x562e65762f80 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 10 client.17343 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 10 client.17343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 20 client.17343 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e657dd770 0x562e657dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 4 client.17343 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e656cb3a0 con 0x562e657dd770 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.443+0000 7f5933fff640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e657dd770 0x562e657dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.444+0000 7f5933fff640 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e657dd770 0x562e657dfb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=402ae6158024cb22 server_cookie=326631db5d0e72f2 in_seq=0 out_seq=0 2024-07-24T21:55:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.444+0000 7f591b7fe640 10 client.17343 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.447+0000 7f591b7fe640 1 -- 172.21.15.60:0/3481504780 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e656cb3a0 con 0x562e657dd770 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.447+0000 7f591b7fe640 10 client.17343 handle_command_reply: tid=0 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.447+0000 7f5944ce1740 1 client.17343 shutdown 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 msgr2=0x562e6576ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5934009f70 tx=0x7f593402d040 comp rx=0 tx=0).stop 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 shutdown_connections 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x562e657dd770 0x562e657dfb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e65762f80 0x562e6576ab90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e65756ff0 0x562e65766c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 --2- 172.21.15.60:0/3481504780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e657566d0 0x562e657666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 >> 172.21.15.60:0/3481504780 conn(0x562e65754500 msgr2=0x562e65759810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f591b7fe640 0 client.17343 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 shutdown_connections 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 1 -- 172.21.15.60:0/3481504780 wait complete. 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.448+0000 7f5944ce1740 20 client.17343 trim_cache size 0 max 16384 2024-07-24T21:55:32.453 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 msgr2=0x7f59280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 0x7f59280b6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f592c102020 tx=0x7f592c10a550 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 msgr2=0x7f593c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f592c005940 tx=0x7f592c0042f0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 shutdown_connections 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f59280b3d40 0x7f59280b6200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593c108b00 0x7f593c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f593c108180 0x7f593c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 --2- 172.21.15.60:0/139299532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593c102770 0x7f593c198c40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 >> 172.21.15.60:0/139299532 conn(0x7f593c0fe240 msgr2=0x7f593c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 shutdown_connections 2024-07-24T21:55:32.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:32.449+0000 7f5942930640 1 -- 172.21.15.60:0/139299532 wait complete. 2024-07-24T21:55:33.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:55:33.557 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:33.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:33 smithi060 ceph-mon[33000]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-07-24T21:55:34.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:33 smithi123 ceph-mon[37143]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-07-24T21:55:34.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:33 smithi086 ceph-mon[37128]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 439 op/s 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 -- 172.21.15.60:0/3521387348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b80109b40 msgr2=0x7f5b80110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 --2- 172.21.15.60:0/3521387348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b80109b40 0x7f5b80110ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009860 tx=0x7f5b7002a3d0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 -- 172.21.15.60:0/3521387348 shutdown_connections 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 --2- 172.21.15.60:0/3521387348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8011ba90 0x7f5b8011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 --2- 172.21.15.60:0/3521387348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b80109f20 0x7f5b8010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 --2- 172.21.15.60:0/3521387348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b80109b40 0x7f5b80110ba0 secure :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009860 tx=0x7f5b7002a3d0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.432+0000 7f5b873a4640 1 -- 172.21.15.60:0/3521387348 >> 172.21.15.60:0/3521387348 conn(0x7f5b8006d980 msgr2=0x7f5b8006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.433+0000 7f5b873a4640 1 -- 172.21.15.60:0/3521387348 shutdown_connections 2024-07-24T21:55:35.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.433+0000 7f5b873a4640 1 -- 172.21.15.60:0/3521387348 wait complete. 2024-07-24T21:55:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.433+0000 7f5b873a4640 1 Processor -- start 2024-07-24T21:55:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 -- start start 2024-07-24T21:55:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 0x7f5b801a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 0x7f5b801a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801215c0 con 0x7f5b8011ba90 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80121440 con 0x7f5b801ad5b0 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b873a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80121740 con 0x7f5b80109f20 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b85941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 0x7f5b801a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b85941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39642/0 (socket says 172.21.15.60:39642) 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 0x7f5b801a8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39828/0 (socket says 172.21.15.60:39828) 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b85941640 1 -- 172.21.15.60:0/697875852 learned_addr learned my addr 172.21.15.60:0/697875852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.434+0000 7f5b85140640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 0x7f5b801a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 -- 172.21.15.60:0/697875852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 msgr2=0x7f5b801a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 0x7f5b801a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 -- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 msgr2=0x7f5b801a8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 0x7f5b801a8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 -- 172.21.15.60:0/697875852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b801ae140 con 0x7f5b801ad5b0 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85140640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 0x7f5b801a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.435+0000 7f5b85941640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800b970 tx=0x7f5b7800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.436+0000 7f5b767fc640 1 -- 172.21.15.60:0/697875852 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78019070 con 0x7f5b801ad5b0 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.436+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801b4730 con 0x7f5b801ad5b0 2024-07-24T21:55:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.436+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801b4bb0 con 0x7f5b801ad5b0 2024-07-24T21:55:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.436+0000 7f5b767fc640 1 -- 172.21.15.60:0/697875852 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b78014440 con 0x7f5b801ad5b0 2024-07-24T21:55:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.436+0000 7f5b767fc640 1 -- 172.21.15.60:0/697875852 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780075d0 con 0x7f5b801ad5b0 2024-07-24T21:55:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.439+0000 7f5b89755740 1 Processor -- start 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.439+0000 7f5b767fc640 1 -- 172.21.15.60:0/697875852 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5b7801c070 con 0x7f5b801ad5b0 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 -- start start 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8700ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5610b86779b0 con 0x5610b870cf80 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610b8666eb0 con 0x5610b8700ff0 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b89755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5610b866aef0 con 0x5610b87006d0 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b85140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b85140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39650/0 (socket says 172.21.15.60:39650) 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b85140640 1 -- 172.21.15.60:0/4039814410 learned_addr learned my addr 172.21.15.60:0/4039814410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b8493f640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8700ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b767fc640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 0x7f5b6c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.440+0000 7f5b8493f640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 -- 172.21.15.60:0/4039814410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 msgr2=0x5610b8700ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8700ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 -- 172.21.15.60:0/4039814410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 msgr2=0x5610b870f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 -- 172.21.15.60:0/4039814410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b85f0320 con 0x5610b8700ff0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b8493f640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b767fc640 1 -- 172.21.15.60:0/697875852 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5b780e3390 con 0x7f5b801ad5b0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85941640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 0x7f5b6c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85140640 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009830 tx=0x7f5b70032fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b590c01e0e29aa1a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.441+0000 7f5b85941640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 0x7f5b6c0b6280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b780a61b0 tx=0x7f5b78004b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.442+0000 7f5b6b7fe640 1 -- 172.21.15.60:0/4039814410 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70030d00 con 0x5610b8700ff0 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.442+0000 7f5b6b7fe640 1 -- 172.21.15.60:0/4039814410 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b70005040 con 0x5610b8700ff0 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.442+0000 7f5b6b7fe640 1 -- 172.21.15.60:0/4039814410 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70004920 con 0x5610b8700ff0 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- 172.21.15.60:0/4039814410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 msgr2=0x5610b870ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009830 tx=0x7f5b70032fb0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- 172.21.15.60:0/4039814410 shutdown_connections 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b870ca40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 --2- 172.21.15.60:0/4039814410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8700ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- 172.21.15.60:0/4039814410 >> 172.21.15.60:0/4039814410 conn(0x5610b86fe500 msgr2=0x5610b86ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- 172.21.15.60:0/4039814410 shutdown_connections 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- 172.21.15.60:0/4039814410 wait complete. 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 -- start start 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.443+0000 7f5b89755740 1 Processor -- start 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.444+0000 7f5b89755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.444+0000 7f5b89755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b8709e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.444+0000 7f5b89755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b89755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5610b84ca700 con 0x5610b870cf80 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b89755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610b86095f0 con 0x5610b87006d0 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b89755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5610b870fba0 con 0x5610b8700ff0 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39664/0 (socket says 172.21.15.60:39664) 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 -- 172.21.15.60:0/3836485789 learned_addr learned my addr 172.21.15.60:0/3836485789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b85140640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b8709e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b89755740 10 client.0 _ll_register_callbacks cb 0x5610b8629960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 -- 172.21.15.60:0/3836485789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 msgr2=0x5610b8709e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b8709e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 -- 172.21.15.60:0/3836485789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 msgr2=0x5610b870a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b8493f640 1 -- 172.21.15.60:0/3836485789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b85fc680 con 0x5610b87006d0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.445+0000 7f5b85140640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b8709e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.446+0000 7f5b8493f640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c010d00 tx=0x7f5b7c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b78e453cdcb674d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.446+0000 7f5b68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.446+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c004020 con 0x5610b87006d0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.446+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b70005040 con 0x5610b87006d0 2024-07-24T21:55:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.446+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5b7c0041c0 con 0x5610b87006d0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c0043c0 con 0x5610b87006d0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 10 client.26759 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5610b83c7650 con 0x5610b87006d0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b70005040 con 0x5610b87006d0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5b7c027070 con 0x5610b87006d0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 10 client.26759 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 10 client.26759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 20 client.26759 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b871cea0 0x5610b871f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 4 client.26759 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.447+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b8679900 con 0x5610b871cea0 2024-07-24T21:55:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.448+0000 7f5b85140640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b871cea0 0x5610b871f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.449+0000 7f5b85140640 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b871cea0 0x5610b871f2e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a48cf0b3affccf9 server_cookie=66f66fe95e824635 in_seq=0 out_seq=0 2024-07-24T21:55:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.449+0000 7f5b68ff9640 10 client.26759 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:55:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.450+0000 7f5b68ff9640 1 -- 172.21.15.60:0/3836485789 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610b8679900 con 0x5610b871cea0 2024-07-24T21:55:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.450+0000 7f5b68ff9640 10 client.26759 handle_command_reply: tid=0 2024-07-24T21:55:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.450+0000 7f5b89755740 1 client.26759 shutdown 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 msgr2=0x5610b870ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c010d00 tx=0x7f5b7c009260 comp rx=0 tx=0).stop 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 shutdown_connections 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b871cea0 0x5610b871f2e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b870a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b8709e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 --2- 172.21.15.60:0/3836485789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b870ca20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.451+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 >> 172.21.15.60:0/3836485789 conn(0x5610b86fe500 msgr2=0x5610b87040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.452+0000 7f5b68ff9640 0 client.26759 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.452+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 shutdown_connections 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.452+0000 7f5b89755740 1 -- 172.21.15.60:0/3836485789 wait complete. 2024-07-24T21:55:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.452+0000 7f5b89755740 20 client.26759 trim_cache size 0 max 16384 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 Processor -- start 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 -- start start 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b8704820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b87086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5b640112a0 con 0x5610b870cf80 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400eae0 con 0x5610b87006d0 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b89755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400c950 con 0x5610b8700ff0 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39870/0 (socket says 172.21.15.60:39870) 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b8493f640 1 -- 172.21.15.60:0/1345929799 learned_addr learned my addr 172.21.15.60:0/1345929799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b85140640 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b87086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.474+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b8704820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 -- 172.21.15.60:0/1345929799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 msgr2=0x5610b87086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b87086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 -- 172.21.15.60:0/1345929799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 msgr2=0x5610b8704820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b8704820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 -- 172.21.15.60:0/1345929799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b8700e20 con 0x5610b870cf80 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005ec0 tx=0x7f5b7c012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703aedb818af9a8d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b6affd640 1 -- 172.21.15.60:0/1345929799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c02d040 con 0x5610b870cf80 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b6affd640 1 -- 172.21.15.60:0/1345929799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b7c003aa0 con 0x5610b870cf80 2024-07-24T21:55:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.475+0000 7f5b6affd640 1 -- 172.21.15.60:0/1345929799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c004220 con 0x5610b870cf80 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- 172.21.15.60:0/1345929799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 msgr2=0x5610b8708c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005ec0 tx=0x7f5b7c012040 comp rx=0 tx=0).stop 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- 172.21.15.60:0/1345929799 shutdown_connections 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8708c30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b8700ff0 0x5610b87086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 --2- 172.21.15.60:0/1345929799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b87006d0 0x5610b8704820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- 172.21.15.60:0/1345929799 >> 172.21.15.60:0/1345929799 conn(0x5610b86fe500 msgr2=0x5610b87051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- 172.21.15.60:0/1345929799 shutdown_connections 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- 172.21.15.60:0/1345929799 wait complete. 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.476+0000 7f5b89755740 1 -- start start 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 Processor -- start 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8710290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b87107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8785bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400cfa0 con 0x5610b870cf80 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400e410 con 0x5610b8700ff0 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400d670 con 0x5610b87006d0 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b89755740 10 client.0 _ll_register_callbacks cb 0x7f5b7c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8710290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b85140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b87107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b8493f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8710290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:56782/0 (socket says 172.21.15.60:56782) 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b8493f640 1 -- 172.21.15.60:0/1658797210 learned_addr learned my addr 172.21.15.60:0/1658797210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8785bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b85140640 1 -- 172.21.15.60:0/1658797210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 msgr2=0x5610b8710290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b85140640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8710290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b85140640 1 -- 172.21.15.60:0/1658797210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 msgr2=0x5610b8785bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.477+0000 7f5b85140640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8785bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b85140640 1 -- 172.21.15.60:0/1658797210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b87860f0 con 0x5610b8700ff0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8785bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b85140640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b87107d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b70005940 tx=0x7f5b7002ab70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc4bbd3dafc2985 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70005050 con 0x5610b8700ff0 2024-07-24T21:55:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610b86753a0 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f5b700051f0 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5b700053f0 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b89755740 10 client.26765 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.478+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5610b8787320 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5610b86753a0 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5b7005a440 con 0x5610b8700ff0 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 10 client.26765 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 10 client.26765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 20 client.26765 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b8787770 0x5610b8789b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 4 client.26765 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610b8672ae0 con 0x5610b8787770 2024-07-24T21:55:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b8787770 0x5610b8789b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b8493f640 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b8787770 0x5610b8789b60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c55c658754aed63 server_cookie=793c2bead5dfb788 in_seq=0 out_seq=0 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.479+0000 7f5b68ff9640 10 client.26765 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b68ff9640 1 -- 172.21.15.60:0/1658797210 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610b8672ae0 con 0x5610b8787770 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b68ff9640 10 client.26765 handle_command_reply: tid=0 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 client.26765 shutdown 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 msgr2=0x5610b87107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b87107d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b70005940 tx=0x7f5b7002ab70 comp rx=0 tx=0).stop 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 shutdown_connections 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5610b8787770 0x5610b8789b60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610b870cf80 0x5610b8785bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610b8700ff0 0x5610b87107d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b68ff9640 0 client.26765 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 --2- 172.21.15.60:0/1658797210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5610b87006d0 0x5610b8710290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 >> 172.21.15.60:0/1658797210 conn(0x5610b86fe500 msgr2=0x5610b8708670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 shutdown_connections 2024-07-24T21:55:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 1 -- 172.21.15.60:0/1658797210 wait complete. 2024-07-24T21:55:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.480+0000 7f5b89755740 20 client.26765 trim_cache size 0 max 16384 2024-07-24T21:55:35.489 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:55:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 msgr2=0x7f5b6c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 0x7f5b6c0b6280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b780a61b0 tx=0x7f5b78004b50 comp rx=0 tx=0).stop 2024-07-24T21:55:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 msgr2=0x7f5b801ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800b970 tx=0x7f5b7800be30 comp rx=0 tx=0).stop 2024-07-24T21:55:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 shutdown_connections 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5b6c0b3dc0 0x7f5b6c0b6280 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b801ad5b0 0x7f5b801ada60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b8011ba90 0x7f5b801a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 --2- 172.21.15.60:0/697875852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80109f20 0x7f5b801a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 >> 172.21.15.60:0/697875852 conn(0x7f5b8006d980 msgr2=0x7f5b8011c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 shutdown_connections 2024-07-24T21:55:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:35.481+0000 7f5b873a4640 1 -- 172.21.15.60:0/697875852 wait complete. 2024-07-24T21:55:36.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:35 smithi060 ceph-mon[33000]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 418 op/s 2024-07-24T21:55:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:35 smithi123 ceph-mon[37143]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 418 op/s 2024-07-24T21:55:36.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:55:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:35 smithi086 ceph-mon[37128]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 418 op/s 2024-07-24T21:55:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:37.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:37 smithi060 ceph-mon[33000]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-07-24T21:55:38.097 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:37 smithi086 ceph-mon[37128]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-07-24T21:55:38.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:37 smithi123 ceph-mon[37143]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 594 op/s 2024-07-24T21:55:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 -- 172.21.15.60:0/2022626002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7141079e0 msgr2=0x7fe71410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/2022626002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7141079e0 0x7fe71410e1e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009ca0 tx=0x7fe70c02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 -- 172.21.15.60:0/2022626002 shutdown_connections 2024-07-24T21:55:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/2022626002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7141079e0 0x7fe71410e1e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/2022626002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe714107060 0x7fe7141074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/2022626002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714101650 0x7fe714101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.386+0000 7fe71c3ac640 1 -- 172.21.15.60:0/2022626002 >> 172.21.15.60:0/2022626002 conn(0x7fe7140fd0c0 msgr2=0x7fe7140ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.387+0000 7fe71c3ac640 1 -- 172.21.15.60:0/2022626002 shutdown_connections 2024-07-24T21:55:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.388+0000 7fe71c3ac640 1 -- 172.21.15.60:0/2022626002 wait complete. 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 Processor -- start 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 -- start start 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714101650 0x7fe7141969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 0x7fe714196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141105e0 con 0x7fe7141079e0 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe714110460 con 0x7fe714101650 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.389+0000 7fe71c3ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe714110760 con 0x7fe714107060 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe71a949640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe71a949640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33248/0 (socket says 172.21.15.60:33248) 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe71a949640 1 -- 172.21.15.60:0/1260963944 learned_addr learned my addr 172.21.15.60:0/1260963944 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 0x7fe714196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe71a148640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714101650 0x7fe7141969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 -- 172.21.15.60:0/1260963944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714101650 msgr2=0x7fe7141969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714101650 0x7fe7141969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 -- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 msgr2=0x7fe71419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.390+0000 7fe719947640 1 -- 172.21.15.60:0/1260963944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71419ca90 con 0x7fe714107060 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe71a949640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe719947640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 0x7fe714196f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe704014fc0 tx=0x7fe704007650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe70b7fe640 1 -- 172.21.15.60:0/1260963944 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704004020 con 0x7fe714107060 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe70b7fe640 1 -- 172.21.15.60:0/1260963944 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7040041c0 con 0x7fe714107060 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71419cd80 con 0x7fe714107060 2024-07-24T21:55:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe70b7fe640 1 -- 172.21.15.60:0/1260963944 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704009800 con 0x7fe714107060 2024-07-24T21:55:38.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.391+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a3610 con 0x7fe714107060 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.395+0000 7fe71e75d740 1 Processor -- start 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.395+0000 7fe71e75d740 1 -- start start 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.395+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e4c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71e75d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55906e4179b0 con 0x55906e4a06d0 2024-07-24T21:55:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71e75d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55906e406eb0 con 0x55906e4a0ff0 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71e75d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55906e40aef0 con 0x55906e4bdf90 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe70b7fe640 1 -- 172.21.15.60:0/1260963944 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe704024020 con 0x7fe714107060 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36838/0 (socket says 172.21.15.60:36838) 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 -- 172.21.15.60:0/3090730088 learned_addr learned my addr 172.21.15.60:0/3090730088 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 -- 172.21.15.60:0/3090730088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 msgr2=0x55906e4c0380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e4c0380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 -- 172.21.15.60:0/3090730088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 msgr2=0x55906e4a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.396+0000 7fe71a148640 1 -- 172.21.15.60:0/3090730088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906e390320 con 0x55906e4a0ff0 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe71a148640 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe71000ddf0 tx=0x7fe710005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72452e46eeaf72fb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe70b7fe640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 0x7fe7000b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe708de4640 1 -- 172.21.15.60:0/3090730088 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710015620 con 0x55906e4a0ff0 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe708de4640 1 -- 172.21.15.60:0/3090730088 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71001a070 con 0x55906e4a0ff0 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe708de4640 1 -- 172.21.15.60:0/3090730088 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100149e0 con 0x55906e4a0ff0 2024-07-24T21:55:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.397+0000 7fe70b7fe640 1 -- 172.21.15.60:0/1260963944 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe7040e35c0 con 0x7fe714107060 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.398+0000 7fe71a949640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 0x7fe7000b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 -- 172.21.15.60:0/3090730088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 msgr2=0x55906e4bda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71a949640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 0x7fe7000b6300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe70c033040 tx=0x7fe70c032eb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe71000ddf0 tx=0x7fe710005f90 comp rx=0 tx=0).stop 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 -- 172.21.15.60:0/3090730088 shutdown_connections 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e4c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a0ff0 0x55906e4bda50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3090730088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.399+0000 7fe71e75d740 1 -- 172.21.15.60:0/3090730088 >> 172.21.15.60:0/3090730088 conn(0x55906e49e500 msgr2=0x55906e49f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.403+0000 7fe71e75d740 1 -- 172.21.15.60:0/3090730088 shutdown_connections 2024-07-24T21:55:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.403+0000 7fe71e75d740 1 -- 172.21.15.60:0/3090730088 wait complete. 2024-07-24T21:55:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.403+0000 7fe71e75d740 1 -- start start 2024-07-24T21:55:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.403+0000 7fe71e75d740 1 Processor -- start 2024-07-24T21:55:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 0x55906e4bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55906e26a700 con 0x55906e4bdf90 2024-07-24T21:55:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55906e3a95f0 con 0x55906e4a06d0 2024-07-24T21:55:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55906e4c0c40 con 0x55906e4a0ff0 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71e75d740 10 client.0 _ll_register_callbacks cb 0x55906e3c9960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36848/0 (socket says 172.21.15.60:36848) 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 -- 172.21.15.60:0/1967698096 learned_addr learned my addr 172.21.15.60:0/1967698096 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a949640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 0x55906e4bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 -- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 msgr2=0x55906e4bb440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4bb440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 -- 172.21.15.60:0/1967698096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 msgr2=0x55906e4bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 0x55906e4bb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.404+0000 7fe71a148640 1 -- 172.21.15.60:0/1967698096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906e39c680 con 0x55906e4a06d0 2024-07-24T21:55:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.405+0000 7fe71a949640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 0x55906e4bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:55:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.405+0000 7fe71a148640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe710017040 tx=0x7fe710014b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8350115bee30a71 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710005270 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71001a070 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe710005410 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe710005610 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe71e75d740 10 client.26774 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.406+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55906e167650 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.408+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe710023030 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe7100155c0 con 0x55906e4a06d0 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 10 client.26774 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 10 client.26774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 20 client.26774 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e4cde00 0x55906e4d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 4 client.26774 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe71001a070 con 0x55906e4cde00 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.409+0000 7fe719947640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e4cde00 0x55906e4d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.410+0000 7fe719947640 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e4cde00 0x55906e4d0220 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee51e659cc906a9c server_cookie=83d2c9a978225a42 in_seq=0 out_seq=0 2024-07-24T21:55:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.410+0000 7fe6fdffb640 10 client.26774 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:38.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.412+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1967698096 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe71001a070 con 0x55906e4cde00 2024-07-24T21:55:38.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.412+0000 7fe6fdffb640 10 client.26774 handle_command_reply: tid=0 2024-07-24T21:55:38.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 client.26774 shutdown 2024-07-24T21:55:38.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 msgr2=0x55906e4baf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe710017040 tx=0x7fe710014b90 comp rx=0 tx=0).stop 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 shutdown_connections 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e4cde00 0x55906e4d0220 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4bdf90 0x55906e4bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1967698096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4baf00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 >> 172.21.15.60:0/1967698096 conn(0x55906e49e500 msgr2=0x55906e4a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe6fdffb640 0 client.26774 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.413+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 shutdown_connections 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.414+0000 7fe71e75d740 1 -- 172.21.15.60:0/1967698096 wait complete. 2024-07-24T21:55:38.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.414+0000 7fe71e75d740 20 client.26774 trim_cache size 0 max 16384 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.435+0000 7fe71e75d740 1 Processor -- start 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 -- start start 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e49f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8011330 con 0x55906e4a0ff0 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f800eb70 con 0x55906e4a06d0 2024-07-24T21:55:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.436+0000 7fe71e75d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f800c9e0 con 0x55906e4bdf90 2024-07-24T21:55:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe71a148640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33268/0 (socket says 172.21.15.60:33268) 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 -- 172.21.15.60:0/3170874701 learned_addr learned my addr 172.21.15.60:0/3170874701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe71a949640 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e49f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 -- 172.21.15.60:0/3170874701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 msgr2=0x55906e49f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e49f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 -- 172.21.15.60:0/3170874701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 msgr2=0x55906e4b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 -- 172.21.15.60:0/3170874701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906e4a6830 con 0x55906e4a0ff0 2024-07-24T21:55:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.437+0000 7fe719947640 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe704014f90 tx=0x7fe7040b2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a94fb2e2b9a2655 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.438+0000 7fe6fffff640 1 -- 172.21.15.60:0/3170874701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b20c0 con 0x55906e4a0ff0 2024-07-24T21:55:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.439+0000 7fe6fffff640 1 -- 172.21.15.60:0/3170874701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7040ae0d0 con 0x55906e4a0ff0 2024-07-24T21:55:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.439+0000 7fe6fffff640 1 -- 172.21.15.60:0/3170874701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b6f50 con 0x55906e4a0ff0 2024-07-24T21:55:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 -- 172.21.15.60:0/3170874701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 msgr2=0x55906e4bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe704014f90 tx=0x7fe7040b2840 comp rx=0 tx=0).stop 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 -- 172.21.15.60:0/3170874701 shutdown_connections 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4bdf90 0x55906e49f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a0ff0 0x55906e4bbce0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 --2- 172.21.15.60:0/3170874701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4a06d0 0x55906e4b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 -- 172.21.15.60:0/3170874701 >> 172.21.15.60:0/3170874701 conn(0x55906e49e500 msgr2=0x55906e4bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.440+0000 7fe71e75d740 1 -- 172.21.15.60:0/3170874701 shutdown_connections 2024-07-24T21:55:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.441+0000 7fe71e75d740 1 -- 172.21.15.60:0/3170874701 wait complete. 2024-07-24T21:55:38.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.441+0000 7fe71e75d740 1 -- start start 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.441+0000 7fe71e75d740 1 Processor -- start 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.442+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.442+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.442+0000 7fe71e75d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 0x55906e4d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.442+0000 7fe71e75d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f800d030 con 0x55906e4a06d0 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe71e75d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f800e4a0 con 0x55906e4bdf90 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe719947640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe71e75d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f800d700 con 0x55906e4a0ff0 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe719947640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47220/0 (socket says 172.21.15.60:47220) 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe719947640 1 -- 172.21.15.60:0/1621575256 learned_addr learned my addr 172.21.15.60:0/1621575256 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe71a949640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 0x55906e4d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.443+0000 7fe71e75d740 10 client.0 _ll_register_callbacks cb 0x7fe71000dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 -- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 msgr2=0x55906e4c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 -- 172.21.15.60:0/1621575256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 msgr2=0x55906e4c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 -- 172.21.15.60:0/1621575256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906e4d5900 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71a949640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 0x55906e4d50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009600 tx=0x7fe70c005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ee360754822b2f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c004250 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7040ae0d0 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.444+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe70c0043f0 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe70c0045f0 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 10 client.26780 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55906e5385b0 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7040ae0d0 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe70c044040 con 0x55906e4bdf90 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 10 client.26780 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 20 client.26780 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.445+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e538dc0 0x55906e53b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.446+0000 7fe71e75d740 4 client.26780 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.446+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55906e419900 con 0x55906e538dc0 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.446+0000 7fe71a148640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e538dc0 0x55906e53b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.446+0000 7fe6fdffb640 10 client.26780 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:38.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.447+0000 7fe71a148640 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e538dc0 0x55906e53b1e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58048b3fc4936162 server_cookie=4b737043e1d293a6 in_seq=0 out_seq=0 2024-07-24T21:55:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.450+0000 7fe6fdffb640 1 -- 172.21.15.60:0/1621575256 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55906e419900 con 0x55906e538dc0 2024-07-24T21:55:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.450+0000 7fe6fdffb640 10 client.26780 handle_command_reply: tid=0 2024-07-24T21:55:38.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.450+0000 7fe71e75d740 1 client.26780 shutdown 2024-07-24T21:55:38.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 msgr2=0x55906e4d50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 0x55906e4d50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009600 tx=0x7fe70c005d80 comp rx=0 tx=0).stop 2024-07-24T21:55:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 shutdown_connections 2024-07-24T21:55:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55906e538dc0 0x55906e53b1e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe6fdffb640 0 client.26780 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55906e4bdf90 0x55906e4d50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55906e4a0ff0 0x55906e4c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 --2- 172.21.15.60:0/1621575256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55906e4a06d0 0x55906e4c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 >> 172.21.15.60:0/1621575256 conn(0x55906e49e500 msgr2=0x55906e49f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 shutdown_connections 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 1 -- 172.21.15.60:0/1621575256 wait complete. 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.451+0000 7fe71e75d740 20 client.26780 trim_cache size 0 max 16384 2024-07-24T21:55:38.454 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:55:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.452+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 msgr2=0x7fe7000b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.452+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 0x7fe7000b6300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe70c033040 tx=0x7fe70c032eb0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.452+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 msgr2=0x7fe714196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.452+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 0x7fe714196f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe704014fc0 tx=0x7fe704007650 comp rx=0 tx=0).stop 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 shutdown_connections 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7000b3e40 0x7fe7000b6300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7141079e0 0x7fe71419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe714107060 0x7fe714196f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 --2- 172.21.15.60:0/1260963944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714101650 0x7fe7141969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 >> 172.21.15.60:0/1260963944 conn(0x7fe7140fd0c0 msgr2=0x7fe714103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 shutdown_connections 2024-07-24T21:55:38.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:38.453+0000 7fe71c3ac640 1 -- 172.21.15.60:0/1260963944 wait complete. 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 -- 172.21.15.60:0/418793631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348106f90 msgr2=0x7fe3481073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 --2- 172.21.15.60:0/418793631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348106f90 0x7fe3481073d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a3f0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 -- 172.21.15.60:0/418793631 shutdown_connections 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 --2- 172.21.15.60:0/418793631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 0x7fe34810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 --2- 172.21.15.60:0/418793631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe348106f90 0x7fe3481073d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 --2- 172.21.15.60:0/418793631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3481015e0 0x7fe3481019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 -- 172.21.15.60:0/418793631 >> 172.21.15.60:0/418793631 conn(0x7fe3480782e0 msgr2=0x7fe3480ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 -- 172.21.15.60:0/418793631 shutdown_connections 2024-07-24T21:55:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.440+0000 7fe34d083640 1 -- 172.21.15.60:0/418793631 wait complete. 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 Processor -- start 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 -- start start 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 0x7fe3480fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 0x7fe3480fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481115b0 con 0x7fe348106f90 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe348111430 con 0x7fe3481015e0 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.441+0000 7fe34d083640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe348111730 con 0x7fe348107910 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe34659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 0x7fe3480fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe34659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 0x7fe3480fdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33282/0 (socket says 172.21.15.60:33282) 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36886/0 (socket says 172.21.15.60:36886) 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe34659c640 1 -- 172.21.15.60:0/3713809984 learned_addr learned my addr 172.21.15.60:0/3713809984 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe34759e640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 0x7fe3480fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 -- 172.21.15.60:0/3713809984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 msgr2=0x7fe3480fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 0x7fe3480fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 -- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 msgr2=0x7fe3480fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 0x7fe3480fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 -- 172.21.15.60:0/3713809984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3481a7090 con 0x7fe3481015e0 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe346d9d640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3f0 tx=0x7fe33800b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe33ffff640 1 -- 172.21.15.60:0/3713809984 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338015020 con 0x7fe3481015e0 2024-07-24T21:55:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.442+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3481a7380 con 0x7fe3481015e0 2024-07-24T21:55:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.443+0000 7fe33ffff640 1 -- 172.21.15.60:0/3713809984 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338011ce0 con 0x7fe3481015e0 2024-07-24T21:55:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.443+0000 7fe33ffff640 1 -- 172.21.15.60:0/3713809984 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338010d30 con 0x7fe3481015e0 2024-07-24T21:55:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.443+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481a7970 con 0x7fe3481015e0 2024-07-24T21:55:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.445+0000 7fe33ffff640 1 -- 172.21.15.60:0/3713809984 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe338018020 con 0x7fe3481015e0 2024-07-24T21:55:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.445+0000 7fe34fdd6740 1 Processor -- start 2024-07-24T21:55:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 -- start start 2024-07-24T21:55:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a24694a280 con 0x55a2469d48c0 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a246939780 con 0x55a2469d51e0 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a24693d7c0 con 0x55a2469e1170 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469d4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33296/0 (socket says 172.21.15.60:33296) 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe34759e640 1 -- 172.21.15.60:0/1400685833 learned_addr learned my addr 172.21.15.60:0/1400685833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.446+0000 7fe346d9d640 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 -- 172.21.15.60:0/1400685833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 msgr2=0x55a2469e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 -- 172.21.15.60:0/1400685833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 msgr2=0x55a2469d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 -- 172.21.15.60:0/1400685833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24669d790 con 0x55a2469e1170 2024-07-24T21:55:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe33ffff640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 0x7fe3340b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469e3560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe330004770 tx=0x7fe330031c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1e1240900b03d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe34759e640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 0x7fe3340b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe33ffff640 1 -- 172.21.15.60:0/3713809984 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe3380e3670 con 0x7fe3481015e0 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe33d7fa640 1 -- 172.21.15.60:0/1400685833 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330040070 con 0x55a2469e1170 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe33d7fa640 1 -- 172.21.15.60:0/1400685833 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3300316a0 con 0x55a2469e1170 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.447+0000 7fe33d7fa640 1 -- 172.21.15.60:0/1400685833 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035040 con 0x55a2469e1170 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400685833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 msgr2=0x55a2469e3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34759e640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 0x7fe3340b6200 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe340012680 tx=0x7fe340009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469e3560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe330004770 tx=0x7fe330031c40 comp rx=0 tx=0).stop 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400685833 shutdown_connections 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469e3560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400685833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400685833 >> 172.21.15.60:0/1400685833 conn(0x55a2469d26f0 msgr2=0x55a2469d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400685833 shutdown_connections 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400685833 wait complete. 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.448+0000 7fe34fdd6740 1 -- start start 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 Processor -- start 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a246873ab0 con 0x55a2469d48c0 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2468c19e0 con 0x55a2469d51e0 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2469e3df0 con 0x55a2469e1170 2024-07-24T21:55:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34fdd6740 10 client.0 _ll_register_callbacks cb 0x55a246649a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33300/0 (socket says 172.21.15.60:33300) 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 -- 172.21.15.60:0/863504548 learned_addr learned my addr 172.21.15.60:0/863504548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34759e640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34659c640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 -- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 msgr2=0x55a2469de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 -- 172.21.15.60:0/863504548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 msgr2=0x55a2469ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 -- 172.21.15.60:0/863504548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a246716620 con 0x55a2469d48c0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe34759e640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe346d9d640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe33800bcf0 tx=0x7fe3380aa200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece63885497c2a94 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe32e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.449+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380a9e70 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe338011830 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2468e8ce0 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe3380aaa60 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 10 client.17376 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a2469ef5c0 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a2468e8ce0 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe3380b1ce0 con 0x55a2469d48c0 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 10 client.17376 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 20 client.17376 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a2469f0a80 0x55a2469f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 4 client.17376 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24694c1d0 con 0x55a2469f0a80 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.450+0000 7fe34659c640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a2469f0a80 0x55a2469f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe32e7fc640 10 client.17376 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe34659c640 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a2469f0a80 0x55a2469f2e70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a745ac7681673686 server_cookie=d379af8766027d14 in_seq=0 out_seq=0 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe32e7fc640 1 -- 172.21.15.60:0/863504548 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a24694c1d0 con 0x55a2469f0a80 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe32e7fc640 10 client.17376 handle_command_reply: tid=0 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe34fdd6740 1 client.17376 shutdown 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 msgr2=0x55a2469e0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.451+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe33800bcf0 tx=0x7fe3380aa200 comp rx=0 tx=0).stop 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 shutdown_connections 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a2469f0a80 0x55a2469f2e70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/863504548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e0900 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe32e7fc640 0 client.17376 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 >> 172.21.15.60:0/863504548 conn(0x55a2469d26f0 msgr2=0x55a2469d8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 shutdown_connections 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 1 -- 172.21.15.60:0/863504548 wait complete. 2024-07-24T21:55:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.452+0000 7fe34fdd6740 20 client.17376 trim_cache size 0 max 16384 2024-07-24T21:55:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.468+0000 7fe34fdd6740 1 Processor -- start 2024-07-24T21:55:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.469+0000 7fe34fdd6740 1 -- start start 2024-07-24T21:55:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.469+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33304/0 (socket says 172.21.15.60:33304) 2024-07-24T21:55:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe346d9d640 1 -- 172.21.15.60:0/1400939557 learned_addr learned my addr 172.21.15.60:0/1400939557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ae9e0 con 0x55a2469d48c0 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ac220 con 0x55a2469d51e0 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481aa0b0 con 0x55a2469e1170 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34759e640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 -- 172.21.15.60:0/1400939557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 msgr2=0x55a2469f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 -- 172.21.15.60:0/1400939557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 msgr2=0x55a2469ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.470+0000 7fe34659c640 1 -- 172.21.15.60:0/1400939557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2469f31e0 con 0x55a2469d51e0 2024-07-24T21:55:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.471+0000 7fe346d9d640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.471+0000 7fe34659c640 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469dfa10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe330006010 tx=0x7fe330031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ce43963837f54a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.471+0000 7fe33cff9640 1 -- 172.21.15.60:0/1400939557 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330040070 con 0x55a2469d51e0 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.471+0000 7fe33cff9640 1 -- 172.21.15.60:0/1400939557 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3300316a0 con 0x55a2469d51e0 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.471+0000 7fe33cff9640 1 -- 172.21.15.60:0/1400939557 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035040 con 0x55a2469d51e0 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.472+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 msgr2=0x55a2469dfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.472+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469dfa10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe330006010 tx=0x7fe330031400 comp rx=0 tx=0).stop 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 shutdown_connections 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a2469f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469dfa10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1400939557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 >> 172.21.15.60:0/1400939557 conn(0x55a2469d26f0 msgr2=0x55a2469df300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 shutdown_connections 2024-07-24T21:55:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1400939557 wait complete. 2024-07-24T21:55:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.473+0000 7fe34fdd6740 1 -- start start 2024-07-24T21:55:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 Processor -- start 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a246a59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481aa700 con 0x55a2469d48c0 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe346d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33318/0 (socket says 172.21.15.60:33318) 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe346d9d640 1 -- 172.21.15.60:0/1511567716 learned_addr learned my addr 172.21.15.60:0/1511567716 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481abb50 con 0x55a2469d51e0 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481aadb0 con 0x55a2469e1170 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.474+0000 7fe34659c640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe34fdd6740 10 client.0 _ll_register_callbacks cb 0x7fe338010650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 -- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 msgr2=0x55a246a59750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a246a59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 -- 172.21.15.60:0/1511567716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 msgr2=0x55a2469e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a246a59c90 con 0x55a2469d48c0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe346d9d640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe3380a63e0 tx=0x7fe3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c1123ac45dcde server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe32e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380a9ad0 con 0x55a2469d48c0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fe3380a5ee0 con 0x55a2469d48c0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.475+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3300316a0 con 0x55a2469d48c0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.476+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe3380a9e70 con 0x55a2469d48c0 2024-07-24T21:55:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.476+0000 7fe34fdd6740 10 client.17382 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.476+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a246a5adc0 con 0x55a2469d48c0 2024-07-24T21:55:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.476+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3300316a0 con 0x55a2469d48c0 2024-07-24T21:55:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe3380b1ce0 con 0x55a2469d48c0 2024-07-24T21:55:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 10 client.17382 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 20 client.17382 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a246a5b5a0 0x55a246a5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 4 client.17382 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a246947c70 con 0x55a246a5b5a0 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34659c640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a246a5b5a0 0x55a246a5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe34659c640 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a246a5b5a0 0x55a246a5d9e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cebd7a5ab8da44 server_cookie=11526d4dafc29452 in_seq=0 out_seq=0 2024-07-24T21:55:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.477+0000 7fe32e7fc640 10 client.17382 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe32e7fc640 1 -- 172.21.15.60:0/1511567716 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a246947c70 con 0x55a246a5b5a0 2024-07-24T21:55:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe32e7fc640 10 client.17382 handle_command_reply: tid=0 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 client.17382 shutdown 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 msgr2=0x55a2469e61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe3380a63e0 tx=0x7fe3380046c0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 shutdown_connections 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a246a5b5a0 0x55a246a5d9e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe32e7fc640 0 client.17382 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2469e1170 0x55a246a59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2469d51e0 0x55a2469e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 --2- 172.21.15.60:0/1511567716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a2469d48c0 0x55a2469e61e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 >> 172.21.15.60:0/1511567716 conn(0x55a2469d26f0 msgr2=0x55a2469d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 shutdown_connections 2024-07-24T21:55:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 1 -- 172.21.15.60:0/1511567716 wait complete. 2024-07-24T21:55:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.573+0000 7fe34fdd6740 20 client.17382 trim_cache size 0 max 16384 2024-07-24T21:55:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:55:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:55:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:55:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 msgr2=0x7fe3340b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 0x7fe3340b6200 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe340012680 tx=0x7fe340009500 comp rx=0 tx=0).stop 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 msgr2=0x7fe3480fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3f0 tx=0x7fe33800b7f0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 shutdown_connections 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3340b3d40 0x7fe3340b6200 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348107910 0x7fe3480fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe348106f90 0x7fe3480fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 --2- 172.21.15.60:0/3713809984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3481015e0 0x7fe3480fd780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 >> 172.21.15.60:0/3713809984 conn(0x7fe3480782e0 msgr2=0x7fe3480698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 shutdown_connections 2024-07-24T21:55:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:39.574+0000 7fe34d083640 1 -- 172.21.15.60:0/3713809984 wait complete. 2024-07-24T21:55:40.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:39 smithi123 ceph-mon[37143]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-07-24T21:55:40.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:39 smithi086 ceph-mon[37128]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-07-24T21:55:40.298 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:55:40.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:55:40.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:39 smithi060 ceph-mon[33000]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-07-24T21:55:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:41.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:41 smithi060 ceph-mon[33000]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 691 op/s 2024-07-24T21:55:42.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:41 smithi123 ceph-mon[37143]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 691 op/s 2024-07-24T21:55:42.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:41 smithi086 ceph-mon[37128]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 691 op/s 2024-07-24T21:55:42.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f21f9d9b640 1 -- 172.21.15.60:0/729760531 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4036070 con 0x7f21fc107a20 2024-07-24T21:55:42.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 -- 172.21.15.60:0/729760531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc107a20 msgr2=0x7f21fc10e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 --2- 172.21.15.60:0/729760531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc107a20 0x7f21fc10e220 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009be0 tx=0x7f21f402a6c0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 -- 172.21.15.60:0/729760531 shutdown_connections 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 --2- 172.21.15.60:0/729760531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc107a20 0x7f21fc10e220 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 --2- 172.21.15.60:0/729760531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1070a0 0x7f21fc1074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 --2- 172.21.15.60:0/729760531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc101690 0x7f21fc101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.332+0000 7f22009e6640 1 -- 172.21.15.60:0/729760531 >> 172.21.15.60:0/729760531 conn(0x7f21fc0fd120 msgr2=0x7f21fc0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.333+0000 7f22009e6640 1 -- 172.21.15.60:0/729760531 shutdown_connections 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.333+0000 7f22009e6640 1 -- 172.21.15.60:0/729760531 wait complete. 2024-07-24T21:55:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 Processor -- start 2024-07-24T21:55:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 -- start start 2024-07-24T21:55:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc101690 0x7f21fc10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc1070a0 0x7f21fc10b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc110620 con 0x7f21fc1070a0 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1104a0 con 0x7f21fc101690 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f22009e6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1107a0 con 0x7f21fc107a20 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.334+0000 7f21fad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47300/0 (socket says 172.21.15.60:47300) 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 -- 172.21.15.60:0/850120516 learned_addr learned my addr 172.21.15.60:0/850120516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21f3fff640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc1070a0 0x7f21fc10b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fa59c640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc101690 0x7f21fc10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 -- 172.21.15.60:0/850120516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc101690 msgr2=0x7f21fc10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc101690 0x7f21fc10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 -- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc1070a0 msgr2=0x7f21fc10b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc1070a0 0x7f21fc10b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.335+0000 7f21fad9d640 1 -- 172.21.15.60:0/850120516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc10bdb0 con 0x7f21fc107a20 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.336+0000 7f21fad9d640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21f4030bc0 tx=0x7f21f40318f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.336+0000 7f21f37fe640 1 -- 172.21.15.60:0/850120516 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4041070 con 0x7f21fc107a20 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.336+0000 7f21f37fe640 1 -- 172.21.15.60:0/850120516 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f4003e20 con 0x7f21fc107a20 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.336+0000 7f21f37fe640 1 -- 172.21.15.60:0/850120516 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4036070 con 0x7f21fc107a20 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.337+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1a0f10 con 0x7f21fc107a20 2024-07-24T21:55:42.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.337+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a13a0 con 0x7f21fc107a20 2024-07-24T21:55:42.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.342+0000 7f2202d97740 1 Processor -- start 2024-07-24T21:55:42.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.343+0000 7f21f37fe640 1 -- 172.21.15.60:0/850120516 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f21f4044070 con 0x7f21fc107a20 2024-07-24T21:55:42.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.343+0000 7f2202d97740 1 -- start start 2024-07-24T21:55:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.345+0000 7f21f37fe640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 0x7f21e80b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.345+0000 7f21f37fe640 1 -- 172.21.15.60:0/850120516 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f21f4104530 con 0x7f21fc107a20 2024-07-24T21:55:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.345+0000 7f21f3fff640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 0x7f21e80b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 0x5616b5fc6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5616b5f2f9f0 con 0x5616b5fb8840 2024-07-24T21:55:42.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f21fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f21fa59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36952/0 (socket says 172.21.15.60:36952) 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f21fa59c640 1 -- 172.21.15.60:0/104123282 learned_addr learned my addr 172.21.15.60:0/104123282 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b5f1eef0 con 0x5616b5fb7f20 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5616b5f22f30 con 0x5616b5fc47d0 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.346+0000 7f21fad9d640 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 0x5616b5fc6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21f3fff640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 0x7f21e80b6350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f21fc10dbd0 tx=0x7f21e40113e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 -- 172.21.15.60:0/104123282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 msgr2=0x5616b5fc6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 0x5616b5fc6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 -- 172.21.15.60:0/104123282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 msgr2=0x5616b5fc4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 -- 172.21.15.60:0/104123282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b5c7f930 con 0x5616b5fb7f20 2024-07-24T21:55:42.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fad9d640 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 0x5616b5fc6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:42.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.347+0000 7f21fa59c640 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009d20 tx=0x7f21ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc1114fb9aec99a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.348+0000 7f21f0ff9640 1 -- 172.21.15.60:0/104123282 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec01e070 con 0x5616b5fb7f20 2024-07-24T21:55:42.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.348+0000 7f21f0ff9640 1 -- 172.21.15.60:0/104123282 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ec019ae0 con 0x5616b5fb7f20 2024-07-24T21:55:42.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 msgr2=0x5616b5fb8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009d20 tx=0x7f21ec011810 comp rx=0 tx=0).stop 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 shutdown_connections 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fc47d0 0x5616b5fc6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 --2- 172.21.15.60:0/104123282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb7f20 0x5616b5fb8300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.349+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 >> 172.21.15.60:0/104123282 conn(0x5616b5fb5d50 msgr2=0x5616b5fb6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.350+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 shutdown_connections 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.351+0000 7f2202d97740 1 -- 172.21.15.60:0/104123282 wait complete. 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.351+0000 7f2202d97740 1 -- start start 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.351+0000 7f2202d97740 1 Processor -- start 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fc1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5616b5d82890 con 0x5616b5fb8840 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b5ec1390 con 0x5616b5fc47d0 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5616b5fc74a0 con 0x5616b5fb7f20 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f2202d97740 10 client.0 _ll_register_callbacks cb 0x5616b5ecb500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47328/0 (socket says 172.21.15.60:47328) 2024-07-24T21:55:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 -- 172.21.15.60:0/2621530178 learned_addr learned my addr 172.21.15.60:0/2621530178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fad9d640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fc1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 -- 172.21.15.60:0/2621530178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 msgr2=0x5616b5fc1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fc1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 -- 172.21.15.60:0/2621530178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 msgr2=0x5616b5fc16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.352+0000 7f21fa59c640 1 -- 172.21.15.60:0/2621530178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b5ebbd60 con 0x5616b5fb7f20 2024-07-24T21:55:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.353+0000 7f21fad9d640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fc1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:42.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.356+0000 7f21fa59c640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21ec019a90 tx=0x7f21ec009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33460f5aac1c778d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.356+0000 7f21e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:55:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.356+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec01e040 con 0x5616b5fb7f20 2024-07-24T21:55:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.357+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec019ae0 con 0x5616b5fb7f20 2024-07-24T21:55:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.357+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f21ec005070 con 0x5616b5fb7f20 2024-07-24T21:55:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.357+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f21ec02f050 con 0x5616b5fb7f20 2024-07-24T21:55:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.359+0000 7f2202d97740 10 client.26824 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.359+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5616b5fd3360 con 0x5616b5fb7f20 2024-07-24T21:55:42.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.360+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ec019ae0 con 0x5616b5fb7f20 2024-07-24T21:55:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f21ec02a030 con 0x5616b5fb7f20 2024-07-24T21:55:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 10 client.26824 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 20 client.26824 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b5fd4890 0x5616b5fd6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 4 client.26824 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b5f31940 con 0x5616b5fd4890 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.361+0000 7f21f3fff640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b5fd4890 0x5616b5fd6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.362+0000 7f21f3fff640 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b5fd4890 0x5616b5fd6d10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6919270a40411af0 server_cookie=e526c80995966c6f in_seq=0 out_seq=0 2024-07-24T21:55:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.362+0000 7f21e1ffb640 10 client.26824 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.362+0000 7f21e1ffb640 1 -- 172.21.15.60:0/2621530178 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616b5f31940 con 0x5616b5fd4890 2024-07-24T21:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f21e1ffb640 10 client.26824 handle_command_reply: tid=0 2024-07-24T21:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 client.26824 shutdown 2024-07-24T21:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 msgr2=0x5616b5fc4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21ec019a90 tx=0x7f21ec009f50 comp rx=0 tx=0).stop 2024-07-24T21:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 shutdown_connections 2024-07-24T21:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b5fd4890 0x5616b5fd6d10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fc1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 --2- 172.21.15.60:0/2621530178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc4270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 >> 172.21.15.60:0/2621530178 conn(0x5616b5fb5d50 msgr2=0x5616b5fbb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f21e1ffb640 0 client.26824 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 shutdown_connections 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 1 -- 172.21.15.60:0/2621530178 wait complete. 2024-07-24T21:55:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.363+0000 7f2202d97740 20 client.26824 trim_cache size 0 max 16384 2024-07-24T21:55:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.382+0000 7f2202d97740 1 Processor -- start 2024-07-24T21:55:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.382+0000 7f2202d97740 1 -- start start 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.382+0000 7f2202d97740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fd6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.382+0000 7f2202d97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f2202d97740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fdc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f2202d97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bf970 con 0x5616b5fb8840 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f2202d97740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bd1b0 con 0x5616b5fc47d0 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f2202d97740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bb020 con 0x5616b5fb7f20 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33360/0 (socket says 172.21.15.60:33360) 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 -- 172.21.15.60:0/2701665168 learned_addr learned my addr 172.21.15.60:0/2701665168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21fa59c640 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fd6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21fad9d640 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fdc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 -- 172.21.15.60:0/2701665168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 msgr2=0x5616b5fd6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fd6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 -- 172.21.15.60:0/2701665168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 msgr2=0x5616b5fdc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fdc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 -- 172.21.15.60:0/2701665168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b5fc45e0 con 0x5616b5fb8840 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21f3fff640 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a635481acf4f1ed6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.383+0000 7f21e3fff640 1 -- 172.21.15.60:0/2701665168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e402e070 con 0x5616b5fb8840 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f21e3fff640 1 -- 172.21.15.60:0/2701665168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e4029a50 con 0x5616b5fb8840 2024-07-24T21:55:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f21e3fff640 1 -- 172.21.15.60:0/2701665168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4021d20 con 0x5616b5fb8840 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 -- 172.21.15.60:0/2701665168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 msgr2=0x5616b5fc3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e4016000 comp rx=0 tx=0).stop 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 -- 172.21.15.60:0/2701665168 shutdown_connections 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fc47d0 0x5616b5fdc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fb8840 0x5616b5fc3400 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 --2- 172.21.15.60:0/2701665168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fd6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.384+0000 7f2202d97740 1 -- 172.21.15.60:0/2701665168 >> 172.21.15.60:0/2701665168 conn(0x5616b5fb5d50 msgr2=0x5616b5fc2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.385+0000 7f2202d97740 1 -- 172.21.15.60:0/2701665168 shutdown_connections 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.385+0000 7f2202d97740 1 -- 172.21.15.60:0/2701665168 wait complete. 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.385+0000 7f2202d97740 1 -- start start 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.385+0000 7f2202d97740 1 Processor -- start 2024-07-24T21:55:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb8840 0x5616b5fc9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bb670 con 0x5616b5fc47d0 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bcae0 con 0x5616b5fb8840 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1bbd40 con 0x5616b5fb7f20 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f2202d97740 10 client.0 _ll_register_callbacks cb 0x7f21ec009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fa59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33372/0 (socket says 172.21.15.60:33372) 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 -- 172.21.15.60:0/849670683 learned_addr learned my addr 172.21.15.60:0/849670683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21f3fff640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb8840 0x5616b5fc9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 -- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 msgr2=0x5616b5fc8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 -- 172.21.15.60:0/849670683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb8840 msgr2=0x5616b5fc9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb8840 0x5616b5fc9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 -- 172.21.15.60:0/849670683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b603ec60 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fa59c640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21fad9d640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f21f4101cb0 tx=0x7f21f4102150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5a4cc11362b889 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.386+0000 7f21e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f40e4020 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f21f40cb400 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4029a50 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f21f40ec050 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 10 client.17397 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5616b603ffe0 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f40cb8a0 con 0x5616b5fc47d0 2024-07-24T21:55:42.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f21f40ec1f0 con 0x5616b5fc47d0 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 10 client.17397 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 20 client.17397 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b6040840 0x5616b6042c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 4 client.17397 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f21e4029a50 con 0x5616b6040840 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.387+0000 7f21fa59c640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b6040840 0x5616b6042c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f21fa59c640 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b6040840 0x5616b6042c30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73be515c58a79935 server_cookie=5c2891a3664228e4 in_seq=0 out_seq=0 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f21e1ffb640 10 client.17397 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f21e1ffb640 1 -- 172.21.15.60:0/849670683 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f21e4029a50 con 0x5616b6040840 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f21e1ffb640 10 client.17397 handle_command_reply: tid=0 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f2202d97740 1 client.17397 shutdown 2024-07-24T21:55:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 msgr2=0x5616b603e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.388+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f21f4101cb0 tx=0x7f21f4102150 comp rx=0 tx=0).stop 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 shutdown_connections 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5616b6040840 0x5616b6042c30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616b5fc47d0 0x5616b603e5f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5fb8840 0x5616b5fc9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 --2- 172.21.15.60:0/849670683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616b5fb7f20 0x5616b5fc8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 >> 172.21.15.60:0/849670683 conn(0x5616b5fb5d50 msgr2=0x5616b5fbfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f21e1ffb640 0 client.17397 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 shutdown_connections 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 1 -- 172.21.15.60:0/849670683 wait complete. 2024-07-24T21:55:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f2202d97740 20 client.17397 trim_cache size 0 max 16384 2024-07-24T21:55:42.397 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:55:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:55:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "14e9bceb-ff2b-4130-8b5e-831898a64114", 2024-07-24T21:55:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:55:42.397 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 msgr2=0x7f21e80b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.389+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 0x7f21e80b6350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f21fc10dbd0 tx=0x7f21e40113e0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 msgr2=0x7f21fc10b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21f4030bc0 tx=0x7f21f40318f0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 shutdown_connections 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f21e80b3e90 0x7f21e80b6350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21fc107a20 0x7f21fc10b610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21fc1070a0 0x7f21fc10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 --2- 172.21.15.60:0/850120516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc101690 0x7f21fc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 >> 172.21.15.60:0/850120516 conn(0x7f21fc0fd120 msgr2=0x7f21fc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:42.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 shutdown_connections 2024-07-24T21:55:42.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:42.390+0000 7f22009e6640 1 -- 172.21.15.60:0/850120516 wait complete. 2024-07-24T21:55:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:55:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:55:43.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T21:55:42.743+0000 7f4d4d67a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001bd5 [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51/ auth{2=1} v185768 ap=2 f(v7 m2024-07-24T21:53:00.997649+0000 91=91+0) n(v17 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1) old_inodes=4 (inest sync->mix g=2) caps={15300=pAsLsXsFs/-@1} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x563853a06100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000001bd5:[<0x10000000399/blog-51 v136553>,<0x10000000208/blogtest_in v1507>,<0x10000000201/src v2196>,<0x10000000006/blogbench-1.0 v1907>,<0x10000000005/tmp v130>,<0x100000001fb/client.0 v114>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v128>,<0x10000000001/sv_1 v151>,<0x10000000000/qa v124>,<0x1/volumes v110>]//[]","memoryvalue":"(2)0x20000001bd5:[<0x10000000399/blog-51 v185768>,<0x10000000208/blogtest_in v2518>,<0x10000000201/src v2676>,<0x10000000006/blogbench-1.0 v2258>,<0x10000000005/tmp v473>,<0x100000001fb/client.0 v435>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v468>,<0x10000000001/sv_1 v473>,<0x10000000000/qa v500>,<0x1/volumes v166>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:52.799721+0000 90=90+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1)","memory_value.dirstat":"f(v7 m2024-07-24T21:53:00.997649+0000 91=91+0)","memory_value.rstat":"n(v17 rc2024-07-24T21:53:00.997649+0000 b1763378 92=91+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:55:43.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T21:55:42.781+0000 7f4d4d67a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000002d0 [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20/ auth{2=2} fragtree_t(*^3) v185770 ap=2 f(v7 m2024-07-24T21:53:02.122640+0000 218=218+0) n(v18 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1) old_inodes=5 (inest sync->mix g=2) caps={15300=pAsLsXsFs/-@18} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 dirwaiter=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x5638535c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x200000002d0:[<0x10000000399/blog-20 v134388>,<0x10000000208/blogtest_in v1507>,<0x10000000201/src v2196>,<0x10000000006/blogbench-1.0 v1907>,<0x10000000005/tmp v130>,<0x100000001fb/client.0 v114>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v128>,<0x10000000001/sv_1 v151>,<0x10000000000/qa v124>,<0x1/volumes v110>]//[]","memoryvalue":"(2)0x200000002d0:[<0x10000000399/blog-20 v185770>,<0x10000000208/blogtest_in v2518>,<0x10000000201/src v2676>,<0x10000000006/blogbench-1.0 v2258>,<0x10000000005/tmp v473>,<0x100000001fb/client.0 v435>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v468>,<0x10000000001/sv_1 v473>,<0x10000000000/qa v500>,<0x1/volumes v166>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:26.887951+0000 216=216+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1)","memory_value.dirstat":"f(v7 m2024-07-24T21:53:02.122640+0000 218=218+0)","memory_value.rstat":"n(v18 rc2024-07-24T21:53:02.122640+0000 b3497783 219=218+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:55:43.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 21:55:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T21:55:42.792+0000 7f4d4d67a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000b8b [...a,head] /volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17/ auth{0=1,2=1} fragtree_t(*^3) v211998 ap=2 f(v8 m2024-07-24T21:53:03.964623+0000 187=187+0) n(v18 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1) old_inodes=5 (inest sync->mix g=0,2) caps={15300=pAsLsXsFs/-@9} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x5638530ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000b8b:[<0x10000000399/blog-17 v13270>,<0x10000000208/blogtest_in v345>,<0x10000000201/src v2099>,<0x10000000006/blogbench-1.0 v1818>,<0x10000000005/tmp v50>,<0x100000001fb/client.0 v40>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v60>,<0x10000000001/sv_1 v80>,<0x10000000000/qa v69>,<0x1/volumes v66>]//[]","memoryvalue":"(2)0x10000000b8b:[<0x10000000399/blog-17 v211998>,<0x10000000208/blogtest_in v2518>,<0x10000000201/src v2676>,<0x10000000006/blogbench-1.0 v2258>,<0x10000000005/tmp v473>,<0x100000001fb/client.0 v435>,<0x100000001fa/5bf008ce-3167-4364-b97f-6c159317768b v468>,<0x10000000001/sv_1 v473>,<0x10000000000/qa v500>,<0x1/volumes v166>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-07-24T21:52:06.935132+0000 186=186+0)","ondisk_value.rstat":"n(v0 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1)","memory_value.dirstat":"f(v8 m2024-07-24T21:53:03.964623+0000 187=187+0)","memory_value.rstat":"n(v18 rc2024-07-24T21:53:03.964623+0000 b3511583 188=187+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-07-24T21:55:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:42 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:55:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:42 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:55:43.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:55:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:42 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:55:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:42 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:55:43.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:43.626 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 824 op/s 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:43 smithi060 ceph-mon[33000]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 824 op/s 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:43 smithi123 ceph-mon[37143]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 824 op/s 2024-07-24T21:55:44.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:55:44.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:55:44.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: Scrub error on inode 0x20000001bd5 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-51) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: Scrub error on inode 0x200000002d0 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-20) see mds.b log and `damage ls` output for details 2024-07-24T21:55:44.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:43 smithi086 ceph-mon[37128]: Scrub error on inode 0x10000000b8b (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-17) see mds.b log and `damage ls` output for details 2024-07-24T21:55:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 -- 172.21.15.60:0/265587519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c101650 msgr2=0x7f540c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 --2- 172.21.15.60:0/265587519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c101650 0x7f540c101a30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009900 tx=0x7f53fc02a410 comp rx=0 tx=0).stop 2024-07-24T21:55:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 -- 172.21.15.60:0/265587519 shutdown_connections 2024-07-24T21:55:45.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 --2- 172.21.15.60:0/265587519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c1079e0 0x7f540c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 --2- 172.21.15.60:0/265587519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c107060 0x7f540c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 --2- 172.21.15.60:0/265587519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c101650 0x7f540c101a30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 -- 172.21.15.60:0/265587519 >> 172.21.15.60:0/265587519 conn(0x7f540c0fd0c0 msgr2=0x7f540c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 -- 172.21.15.60:0/265587519 shutdown_connections 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.533+0000 7f5410b2b640 1 -- 172.21.15.60:0/265587519 wait complete. 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 Processor -- start 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 -- start start 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 0x7f540c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f540c110560 con 0x7f540c1079e0 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1103e0 con 0x7f540c101650 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.534+0000 7f5410b2b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1106e0 con 0x7f540c107060 2024-07-24T21:55:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f5403fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 0x7f540c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37008/0 (socket says 172.21.15.60:37008) 2024-07-24T21:55:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f5403fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47362/0 (socket says 172.21.15.60:47362) 2024-07-24T21:55:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 -- 172.21.15.60:0/2237904145 learned_addr learned my addr 172.21.15.60:0/2237904145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 -- 172.21.15.60:0/2237904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 msgr2=0x7f540c196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 -- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 msgr2=0x7f540c19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 0x7f540c19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 -- 172.21.15.60:0/2237904145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19cb20 con 0x7f540c101650 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f5403fff640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:55:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540ad9d640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 0x7f540c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.535+0000 7f540a59c640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005940 tx=0x7f53fc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.536+0000 7f54037fe640 1 -- 172.21.15.60:0/2237904145 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc03f030 con 0x7f540c101650 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.536+0000 7f54037fe640 1 -- 172.21.15.60:0/2237904145 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc0044c0 con 0x7f540c101650 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.536+0000 7f54037fe640 1 -- 172.21.15.60:0/2237904145 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc035070 con 0x7f540c101650 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.536+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19cdb0 con 0x7f540c101650 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.536+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a34d0 con 0x7f540c101650 2024-07-24T21:55:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.539+0000 7f5412edc740 1 Processor -- start 2024-07-24T21:55:45.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.539+0000 7f5412edc740 1 -- start start 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb362e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562cdb2c9990 con 0x562cdb3526b0 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562cdb2b8e90 con 0x562cdb363370 2024-07-24T21:55:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5412edc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562cdb2bced0 con 0x562cdb352fd0 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f54037fe640 1 -- 172.21.15.60:0/2237904145 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f53fc042030 con 0x7f540c101650 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37018/0 (socket says 172.21.15.60:37018) 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33414/0 (socket says 172.21.15.60:33414) 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 -- 172.21.15.60:0/2044002568 learned_addr learned my addr 172.21.15.60:0/2044002568 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f540a59c640 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb362e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 -- 172.21.15.60:0/2044002568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 msgr2=0x562cdb362e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb362e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 -- 172.21.15.60:0/2044002568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 msgr2=0x562cdb352a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f5403fff640 1 -- 172.21.15.60:0/2044002568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdb242300 con 0x562cdb363370 2024-07-24T21:55:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.540+0000 7f540ad9d640 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f5403fff640 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009cf0 tx=0x7f53f40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d7d244b4ff32c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f5400ff9640 1 -- 172.21.15.60:0/2044002568 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4012450 con 0x562cdb363370 2024-07-24T21:55:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f54037fe640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 0x7f53f80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f5400ff9640 1 -- 172.21.15.60:0/2044002568 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f4012aa0 con 0x562cdb363370 2024-07-24T21:55:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f5400ff9640 1 -- 172.21.15.60:0/2044002568 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400fbb0 con 0x562cdb363370 2024-07-24T21:55:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.541+0000 7f540ad9d640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 0x7f53f80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f54037fe640 1 -- 172.21.15.60:0/2237904145 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f53fc1043f0 con 0x7f540c101650 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 -- 172.21.15.60:0/2044002568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 msgr2=0x562cdb365760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009cf0 tx=0x7f53f40117b0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f540ad9d640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 0x7f53f80b6300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f5404009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 -- 172.21.15.60:0/2044002568 shutdown_connections 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb365760 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb362e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 --2- 172.21.15.60:0/2044002568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb352a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 -- 172.21.15.60:0/2044002568 >> 172.21.15.60:0/2044002568 conn(0x562cdb3504e0 msgr2=0x562cdb351680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 -- 172.21.15.60:0/2044002568 shutdown_connections 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.542+0000 7f5412edc740 1 -- 172.21.15.60:0/2044002568 wait complete. 2024-07-24T21:55:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.543+0000 7f5412edc740 1 -- start start 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.543+0000 7f5412edc740 1 Processor -- start 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.543+0000 7f5412edc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb362e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.543+0000 7f5412edc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.543+0000 7f5412edc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb360a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5412edc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562cdb11c700 con 0x562cdb3526b0 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5412edc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562cdb25b5d0 con 0x562cdb363370 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5412edc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562cdb366020 con 0x562cdb352fd0 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb360a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb360a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37026/0 (socket says 172.21.15.60:37026) 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47376/0 (socket says 172.21.15.60:47376) 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f540ad9d640 1 -- 172.21.15.60:0/821714867 learned_addr learned my addr 172.21.15.60:0/821714867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5412edc740 10 client.0 _ll_register_callbacks cb 0x562cdb27b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f540a59c640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb362e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 -- 172.21.15.60:0/821714867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 msgr2=0x562cdb360a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb360a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 -- 172.21.15.60:0/821714867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 msgr2=0x562cdb362e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb362e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 -- 172.21.15.60:0/821714867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdb24e660 con 0x562cdb352fd0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f5403fff640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53f4011cb0 tx=0x7f53f400f4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f15367b7741f66 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.544+0000 7f53f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.545+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4029020 con 0x562cdb352fd0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.545+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f4012aa0 con 0x562cdb352fd0 2024-07-24T21:55:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.545+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f53f4012660 con 0x562cdb352fd0 2024-07-24T21:55:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f53f40128a0 con 0x562cdb352fd0 2024-07-24T21:55:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 10 client.26842 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562cdb019650 con 0x562cdb352fd0 2024-07-24T21:55:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f4009930 con 0x562cdb352fd0 2024-07-24T21:55:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f53f40046c0 con 0x562cdb352fd0 2024-07-24T21:55:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 10 client.26842 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 10 client.26842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 20 client.26842 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb372cf0 0x562cdb3750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 4 client.26842 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f53f4012aa0 con 0x562cdb372cf0 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.546+0000 7f540a59c640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb372cf0 0x562cdb3750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.547+0000 7f540a59c640 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb372cf0 0x562cdb3750e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9641465ad619760 server_cookie=532fd2f684c87aae in_seq=0 out_seq=0 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.547+0000 7f53f1ffb640 10 client.26842 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.547+0000 7f53f1ffb640 1 -- 172.21.15.60:0/821714867 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f53f4012aa0 con 0x562cdb372cf0 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.547+0000 7f53f1ffb640 10 client.26842 handle_command_reply: tid=0 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.547+0000 7f5412edc740 1 client.26842 shutdown 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 msgr2=0x562cdb360270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53f4011cb0 tx=0x7f53f400f4b0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 shutdown_connections 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb372cf0 0x562cdb3750e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb360a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb352fd0 0x562cdb360270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 --2- 172.21.15.60:0/821714867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb3526b0 0x562cdb362e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 >> 172.21.15.60:0/821714867 conn(0x562cdb3504e0 msgr2=0x562cdb356080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f53f1ffb640 0 client.26842 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 shutdown_connections 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 1 -- 172.21.15.60:0/821714867 wait complete. 2024-07-24T21:55:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.548+0000 7f5412edc740 20 client.26842 trim_cache size 0 max 16384 2024-07-24T21:55:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.571+0000 7f5412edc740 1 Processor -- start 2024-07-24T21:55:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.571+0000 7f5412edc740 1 -- start start 2024-07-24T21:55:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb35de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb35eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ae920 con 0x562cdb352fd0 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ac160 con 0x562cdb363370 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5412edc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a9fd0 con 0x562cdb3526b0 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33420/0 (socket says 172.21.15.60:33420) 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 -- 172.21.15.60:0/164021702 learned_addr learned my addr 172.21.15.60:0/164021702 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f540a59c640 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb35de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f540ad9d640 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb35eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 -- 172.21.15.60:0/164021702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 msgr2=0x562cdb35de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb35de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 -- 172.21.15.60:0/164021702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 msgr2=0x562cdb35eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb35eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 -- 172.21.15.60:0/164021702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdb360d70 con 0x562cdb352fd0 2024-07-24T21:55:45.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.572+0000 7f5403fff640 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f53f4030040 tx=0x7f53f4009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f157395365901654 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.573+0000 7f53f3fff640 1 -- 172.21.15.60:0/164021702 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f403a030 con 0x562cdb352fd0 2024-07-24T21:55:45.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.573+0000 7f53f3fff640 1 -- 172.21.15.60:0/164021702 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f4012660 con 0x562cdb352fd0 2024-07-24T21:55:45.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.573+0000 7f53f3fff640 1 -- 172.21.15.60:0/164021702 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4029040 con 0x562cdb352fd0 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.573+0000 7f5412edc740 1 -- 172.21.15.60:0/164021702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 msgr2=0x562cdb363850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.573+0000 7f5412edc740 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f53f4030040 tx=0x7f53f4009930 comp rx=0 tx=0).stop 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- 172.21.15.60:0/164021702 shutdown_connections 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb363370 0x562cdb35eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb352fd0 0x562cdb363850 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- 172.21.15.60:0/164021702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb35de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- 172.21.15.60:0/164021702 >> 172.21.15.60:0/164021702 conn(0x562cdb3504e0 msgr2=0x562cdb364420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- 172.21.15.60:0/164021702 shutdown_connections 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- 172.21.15.60:0/164021702 wait complete. 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- start start 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 Processor -- start 2024-07-24T21:55:45.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb36b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb363370 0x562cdb37a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aa620 con 0x562cdb363370 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.574+0000 7f5412edc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aba90 con 0x562cdb352fd0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5412edc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aacf0 con 0x562cdb3526b0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5412edc740 10 client.0 _ll_register_callbacks cb 0x7f5404009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb363370 0x562cdb37a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37036/0 (socket says 172.21.15.60:37036) 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 -- 172.21.15.60:0/972929920 learned_addr learned my addr 172.21.15.60:0/972929920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f540a59c640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb36b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 -- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 msgr2=0x562cdb36b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb36b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 -- 172.21.15.60:0/972929920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb363370 msgr2=0x562cdb37a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb363370 0x562cdb37a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 -- 172.21.15.60:0/972929920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdb3dc6b0 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f5403fff640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53f40095f0 tx=0x7f53f4009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be898ab175c425b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.575+0000 7f53f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4029030 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f53f4003690 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cdb363f30 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f53f403a070 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f5412edc740 10 client.26834 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.576+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562cdb3dd810 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f4003d40 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f53f4005430 con 0x562cdb352fd0 2024-07-24T21:55:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 10 client.26834 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 20 client.26834 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb3ddf20 0x562cdb3e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 4 client.26834 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cdb363f30 con 0x562cdb3ddf20 2024-07-24T21:55:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.577+0000 7f540a59c640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb3ddf20 0x562cdb3e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f53f1ffb640 10 client.26834 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f540a59c640 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb3ddf20 0x562cdb3e0340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12f75a680eef891 server_cookie=63c78790f20e1182 in_seq=0 out_seq=0 2024-07-24T21:55:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f53f1ffb640 1 -- 172.21.15.60:0/972929920 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562cdb363f30 con 0x562cdb3ddf20 2024-07-24T21:55:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f53f1ffb640 10 client.26834 handle_command_reply: tid=0 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 client.26834 shutdown 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 msgr2=0x562cdb37a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53f40095f0 tx=0x7f53f4009b70 comp rx=0 tx=0).stop 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 shutdown_connections 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cdb3ddf20 0x562cdb3e0340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cdb363370 0x562cdb37a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cdb352fd0 0x562cdb37a1c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 --2- 172.21.15.60:0/972929920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cdb3526b0 0x562cdb36b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 >> 172.21.15.60:0/972929920 conn(0x562cdb3504e0 msgr2=0x562cdb365300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f53f1ffb640 0 client.26834 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 shutdown_connections 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 1 -- 172.21.15.60:0/972929920 wait complete. 2024-07-24T21:55:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.578+0000 7f5412edc740 20 client.26834 trim_cache size 0 max 16384 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "14e9bceb-ff2b-4130-8b5e-831898a64114": { 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "14e9bceb-ff2b-4130-8b5e-831898a64114", 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:55:45.585 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 msgr2=0x7f53f80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 0x7f53f80b6300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f5404009500 comp rx=0 tx=0).stop 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 msgr2=0x7f540c196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005940 tx=0x7f53fc0040c0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 shutdown_connections 2024-07-24T21:55:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53f80b3e40 0x7f53f80b6300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c1079e0 0x7f540c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f540c107060 0x7f540c196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 --2- 172.21.15.60:0/2237904145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f540c101650 0x7f540c196a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 >> 172.21.15.60:0/2237904145 conn(0x7f540c0fd0c0 msgr2=0x7f540c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 shutdown_connections 2024-07-24T21:55:45.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:55:45.579+0000 7f5410b2b640 1 -- 172.21.15.60:0/2237904145 wait complete. 2024-07-24T21:55:46.172 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:45 smithi060 ceph-mon[33000]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 767 op/s 2024-07-24T21:55:46.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:45 smithi123 ceph-mon[37143]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 767 op/s 2024-07-24T21:55:46.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:45 smithi086 ceph-mon[37128]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 767 op/s 2024-07-24T21:55:46.632 INFO:tasks.cephfs.filesystem:scrub status for tag:14e9bceb-ff2b-4130-8b5e-831898a64114 - {'path': '/', 'tag': '14e9bceb-ff2b-4130-8b5e-831898a64114', 'options': 'recursive,force'} 2024-07-24T21:55:48.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:47 smithi123 ceph-mon[37143]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.61k op/s 2024-07-24T21:55:48.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:47 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:48.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:47 smithi086 ceph-mon[37128]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.61k op/s 2024-07-24T21:55:48.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:47 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:48.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:47 smithi060 ceph-mon[33000]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 1.61k op/s 2024-07-24T21:55:48.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:47 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:50.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:49 smithi123 ceph-mon[37143]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 2.19k op/s 2024-07-24T21:55:50.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:49 smithi086 ceph-mon[37128]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 2.19k op/s 2024-07-24T21:55:50.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:49 smithi060 ceph-mon[33000]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 2.19k op/s 2024-07-24T21:55:52.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:51 smithi123 ceph-mon[37143]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 2.63k op/s 2024-07-24T21:55:52.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:51 smithi086 ceph-mon[37128]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 2.63k op/s 2024-07-24T21:55:52.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:51 smithi060 ceph-mon[33000]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.5 MiB/s wr, 2.63k op/s 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- 172.21.15.60:0/844203449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c0103440 msgr2=0x7f71c0103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 --2- 172.21.15.60:0/844203449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c0103440 0x7f71c0103820 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02a410 comp rx=0 tx=0).stop 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- 172.21.15.60:0/844203449 shutdown_connections 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 --2- 172.21.15.60:0/844203449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c010c9f0 0x7f71c010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 --2- 172.21.15.60:0/844203449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103e40 0x7f71c010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 --2- 172.21.15.60:0/844203449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c0103440 0x7f71c0103820 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- 172.21.15.60:0/844203449 >> 172.21.15.60:0/844203449 conn(0x7f71c00fd010 msgr2=0x7f71c00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- 172.21.15.60:0/844203449 shutdown_connections 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- 172.21.15.60:0/844203449 wait complete. 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 Processor -- start 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.676+0000 7f71c81bb640 1 -- start start 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c0103e40 0x7f71c0101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112680 con 0x7f71c010c9f0 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112500 con 0x7f71c0103440 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112800 con 0x7f71c0103e40 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c5f57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60230/0 (socket says 172.21.15.60:60230) 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c5f57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:56280/0 (socket says 172.21.15.60:56280) 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 -- 172.21.15.60:0/883863483 learned_addr learned my addr 172.21.15.60:0/883863483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c5756640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c0103e40 0x7f71c0101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 -- 172.21.15.60:0/883863483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c0103e40 msgr2=0x7f71c0101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c0103e40 0x7f71c0101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 -- 172.21.15.60:0/883863483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 msgr2=0x7f71c01012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 -- 172.21.15.60:0/883863483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01a6ff0 con 0x7f71c010c9f0 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c5f57640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c6758640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71b800eb10 tx=0x7f71b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71b6ffd640 1 -- 172.21.15.60:0/883863483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8004b20 con 0x7f71c010c9f0 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71b6ffd640 1 -- 172.21.15.60:0/883863483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8004cc0 con 0x7f71c010c9f0 2024-07-24T21:55:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01a72e0 con 0x7f71c010c9f0 2024-07-24T21:55:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71b6ffd640 1 -- 172.21.15.60:0/883863483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8010670 con 0x7f71c010c9f0 2024-07-24T21:55:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.677+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a7830 con 0x7f71c010c9f0 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.680+0000 7f71cafec740 1 Processor -- start 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.680+0000 7f71b6ffd640 1 -- 172.21.15.60:0/883863483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f71b8005020 con 0x7f71c010c9f0 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.680+0000 7f71cafec740 1 -- start start 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.680+0000 7f71cafec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.680+0000 7f71cafec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 0x562cc23b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71cafec740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71cafec740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562cc231e280 con 0x562cc23b5170 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71cafec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562cc230d780 con 0x562cc23a91e0 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71cafec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562cc23117c0 con 0x562cc23a88c0 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5756640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5756640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60246/0 (socket says 172.21.15.60:60246) 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 0x562cc23b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5756640 1 -- 172.21.15.60:0/2836228697 learned_addr learned my addr 172.21.15.60:0/2836228697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5756640 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 -- 172.21.15.60:0/2836228697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 msgr2=0x562cc23a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 -- 172.21.15.60:0/2836228697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 msgr2=0x562cc23b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 -- 172.21.15.60:0/2836228697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc2071790 con 0x562cc23a91e0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71b6ffd640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 0x7f71ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5756640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 0x7f71ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71b6ffd640 1 -- 172.21.15.60:0/883863483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f71b80e45c0 con 0x7f71c010c9f0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.681+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 0x562cc23b4c30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009230 tx=0x7f71bc005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e4bd720d12deab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71abfff640 1 -- 172.21.15.60:0/2836228697 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc040070 con 0x562cc23a91e0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71c5756640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 0x7f71ac0b6200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71abfff640 1 -- 172.21.15.60:0/2836228697 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc031760 con 0x562cc23a91e0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71abfff640 1 -- 172.21.15.60:0/2836228697 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc035040 con 0x562cc23a91e0 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 -- 172.21.15.60:0/2836228697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 msgr2=0x562cc23b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 0x562cc23b4c30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009230 tx=0x7f71bc005d90 comp rx=0 tx=0).stop 2024-07-24T21:55:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 -- 172.21.15.60:0/2836228697 shutdown_connections 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a91e0 0x562cc23b4c30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 --2- 172.21.15.60:0/2836228697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.682+0000 7f71cafec740 1 -- 172.21.15.60:0/2836228697 >> 172.21.15.60:0/2836228697 conn(0x562cc23a66f0 msgr2=0x562cc23a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- 172.21.15.60:0/2836228697 shutdown_connections 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- 172.21.15.60:0/2836228697 wait complete. 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- start start 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 Processor -- start 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562cc2247ab0 con 0x562cc23b5170 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562cc22959e0 con 0x562cc23a88c0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562cc23b7df0 con 0x562cc23a91e0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71cafec740 10 client.0 _ll_register_callbacks cb 0x562cc201da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60256/0 (socket says 172.21.15.60:60256) 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.683+0000 7f71c6758640 1 -- 172.21.15.60:0/2943430636 learned_addr learned my addr 172.21.15.60:0/2943430636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c5756640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 -- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 msgr2=0x562cc23b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 -- 172.21.15.60:0/2943430636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 msgr2=0x562cc23b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 -- 172.21.15.60:0/2943430636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc20ea620 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71c6758640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f71b80a7220 tx=0x7f71b80aac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55852f1d5f9d1559 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80ab8a0 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cc22bcce0 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f71b80a7790 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f71b80cc050 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71cafec740 10 client.17427 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.684+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562cc23c3530 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562cc22bcce0 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f71b80c5ce0 con 0x562cc23b5170 2024-07-24T21:55:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 10 client.17427 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 20 client.17427 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc23c49f0 0x562cc23c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 4 client.17427 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cc23201d0 con 0x562cc23c49f0 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc23c49f0 0x562cc23c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc23c49f0 0x562cc23c6e10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54322bd302ef00f4 server_cookie=e3a18ad4f6c0dc33 in_seq=0 out_seq=0 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.685+0000 7f71a97fa640 10 client.17427 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71a97fa640 1 -- 172.21.15.60:0/2943430636 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cc23201d0 con 0x562cc23c49f0 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71a97fa640 10 client.17427 handle_command_reply: tid=0 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 client.17427 shutdown 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 msgr2=0x562cc23b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f71b80a7220 tx=0x7f71b80aac20 comp rx=0 tx=0).stop 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 shutdown_connections 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc23c49f0 0x562cc23c6e10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23b2290 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71a97fa640 0 client.17427 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 --2- 172.21.15.60:0/2943430636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 >> 172.21.15.60:0/2943430636 conn(0x562cc23a66f0 msgr2=0x562cc23ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 shutdown_connections 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 1 -- 172.21.15.60:0/2943430636 wait complete. 2024-07-24T21:55:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.686+0000 7f71cafec740 20 client.17427 trim_cache size 0 max 16384 2024-07-24T21:55:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 Processor -- start 2024-07-24T21:55:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 -- start start 2024-07-24T21:55:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23b5170 0x562cc23ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b2c70 con 0x562cc23a91e0 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b04b0 con 0x562cc23b5170 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71cafec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01ae320 con 0x562cc23a88c0 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60262/0 (socket says 172.21.15.60:60262) 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 -- 172.21.15.60:0/2112751139 learned_addr learned my addr 172.21.15.60:0/2112751139 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c6758640 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23b5170 0x562cc23ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5f57640 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 -- 172.21.15.60:0/2112751139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 msgr2=0x562cc23c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 -- 172.21.15.60:0/2112751139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23b5170 msgr2=0x562cc23ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23b5170 0x562cc23ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.702+0000 7f71c5756640 1 -- 172.21.15.60:0/2112751139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc23a9010 con 0x562cc23a91e0 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71c5756640 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003620 tx=0x7f71b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bfc6a40217ab1e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71ab7fe640 1 -- 172.21.15.60:0/2112751139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b001e4c0 con 0x562cc23a91e0 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71ab7fe640 1 -- 172.21.15.60:0/2112751139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b002e070 con 0x562cc23a91e0 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71ab7fe640 1 -- 172.21.15.60:0/2112751139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0029920 con 0x562cc23a91e0 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 -- 172.21.15.60:0/2112751139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 msgr2=0x562cc23cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003620 tx=0x7f71b00046c0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 -- 172.21.15.60:0/2112751139 shutdown_connections 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23b5170 0x562cc23ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23a91e0 0x562cc23cc720 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 --2- 172.21.15.60:0/2112751139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a88c0 0x562cc23c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 -- 172.21.15.60:0/2112751139 >> 172.21.15.60:0/2112751139 conn(0x562cc23a66f0 msgr2=0x562cc23b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 -- 172.21.15.60:0/2112751139 shutdown_connections 2024-07-24T21:55:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.703+0000 7f71cafec740 1 -- 172.21.15.60:0/2112751139 wait complete. 2024-07-24T21:55:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 -- start start 2024-07-24T21:55:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 Processor -- start 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01ae970 con 0x562cc23b5170 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01afde0 con 0x562cc23a88c0 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01af040 con 0x562cc23a91e0 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71cafec740 10 client.0 _ll_register_callbacks cb 0x7f71b80aafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71c6758640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60268/0 (socket says 172.21.15.60:60268) 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.704+0000 7f71c6758640 1 -- 172.21.15.60:0/1220600715 learned_addr learned my addr 172.21.15.60:0/1220600715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c5f57640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c5756640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 -- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 msgr2=0x562cc23ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23ba470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 -- 172.21.15.60:0/1220600715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 msgr2=0x562cc23b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 -- 172.21.15.60:0/1220600715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc242d7a0 con 0x562cc23b5170 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71c6758640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f71b80049d0 tx=0x7f71b8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00fc38fb5a45be3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b808b060 con 0x562cc23b5170 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f71b80a7790 con 0x562cc23b5170 2024-07-24T21:55:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b002e070 con 0x562cc23b5170 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f71b80aa730 con 0x562cc23b5170 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71cafec740 10 client.17433 fetch_fsmap learned FSMap version 41 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562cc242eab0 con 0x562cc23b5170 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.705+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b002e070 con 0x562cc23b5170 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f71b80c5ce0 con 0x562cc23b5170 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 10 client.17433 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 20 client.17433 populate_metadata read hostname 'smithi060' 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc242f2b0 0x562cc24316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 4 client.17433 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cc231bc70 con 0x562cc242f2b0 2024-07-24T21:55:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71c5f57640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc242f2b0 0x562cc24316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:55:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71c5f57640 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc242f2b0 0x562cc24316d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f31c40dfbbd65e server_cookie=29637068594463d0 in_seq=0 out_seq=0 2024-07-24T21:55:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.706+0000 7f71a97fa640 10 client.17433 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71a97fa640 1 -- 172.21.15.60:0/1220600715 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cc231bc70 con 0x562cc242f2b0 2024-07-24T21:55:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71a97fa640 10 client.17433 handle_command_reply: tid=0 2024-07-24T21:55:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 client.17433 shutdown 2024-07-24T21:55:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 msgr2=0x562cc23ba9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f71b80049d0 tx=0x7f71b8000f40 comp rx=0 tx=0).stop 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 shutdown_connections 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562cc242f2b0 0x562cc24316d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562cc23b5170 0x562cc23ba9b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562cc23a91e0 0x562cc23ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71a97fa640 0 client.17433 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 --2- 172.21.15.60:0/1220600715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562cc23a88c0 0x562cc23b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 >> 172.21.15.60:0/1220600715 conn(0x562cc23a66f0 msgr2=0x562cc23b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 shutdown_connections 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 1 -- 172.21.15.60:0/1220600715 wait complete. 2024-07-24T21:55:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.745+0000 7f71cafec740 20 client.17433 trim_cache size 0 max 16384 2024-07-24T21:55:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:55:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:55:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:55:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:55:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 msgr2=0x7f71ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 0x7f71ac0b6200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0005850 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 msgr2=0x7f71c0101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71b800eb10 tx=0x7f71b800c2d0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 shutdown_connections 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71ac0b3d40 0x7f71ac0b6200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71c010c9f0 0x7f71c0101d40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c0103e40 0x7f71c0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 --2- 172.21.15.60:0/883863483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c0103440 0x7f71c01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 >> 172.21.15.60:0/883863483 conn(0x7f71c00fd010 msgr2=0x7f71c010ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.746+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 shutdown_connections 2024-07-24T21:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:55:52.747+0000 7f71c81bb640 1 -- 172.21.15.60:0/883863483 wait complete. 2024-07-24T21:55:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:53 smithi123 ceph-mon[37143]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 2.80k op/s 2024-07-24T21:55:54.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:53 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:53 smithi086 ceph-mon[37128]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 2.80k op/s 2024-07-24T21:55:54.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:53 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:54.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:53 smithi060 ceph-mon[33000]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 2.80k op/s 2024-07-24T21:55:54.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:53 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:55:56.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:55 smithi123 ceph-mon[37143]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 2.74k op/s 2024-07-24T21:55:56.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:55 smithi086 ceph-mon[37128]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 2.74k op/s 2024-07-24T21:55:56.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:55 smithi060 ceph-mon[33000]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 2.74k op/s 2024-07-24T21:55:58.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:57 smithi123 ceph-mon[37143]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 3.12k op/s 2024-07-24T21:55:58.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:57 smithi086 ceph-mon[37128]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 3.12k op/s 2024-07-24T21:55:58.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:57 smithi060 ceph-mon[33000]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 3.12k op/s 2024-07-24T21:56:00.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:55:59 smithi123 ceph-mon[37143]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 2.45k op/s 2024-07-24T21:56:00.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:55:59 smithi086 ceph-mon[37128]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 2.45k op/s 2024-07-24T21:56:00.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:55:59 smithi060 ceph-mon[33000]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 2.45k op/s 2024-07-24T21:56:02.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:01 smithi123 ceph-mon[37143]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 1.75k op/s 2024-07-24T21:56:02.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:02.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:02.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:01 smithi086 ceph-mon[37128]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 1.75k op/s 2024-07-24T21:56:02.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:02.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:02.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:01 smithi060 ceph-mon[33000]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 1.75k op/s 2024-07-24T21:56:02.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:02.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:03.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:03.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:03.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:04.144 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:03 smithi123 ceph-mon[37143]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-07-24T21:56:04.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:03 smithi086 ceph-mon[37128]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-07-24T21:56:04.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:03 smithi060 ceph-mon[33000]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-07-24T21:56:04.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:04 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:56:05.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:04 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:56:05.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:04 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:56:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- 172.21.15.60:0/2523841734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c102740 msgr2=0x7fa29c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 --2- 172.21.15.60:0/2523841734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c102740 0x7fa29c102b20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009900 tx=0x7fa28c02a410 comp rx=0 tx=0).stop 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- 172.21.15.60:0/2523841734 shutdown_connections 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 --2- 172.21.15.60:0/2523841734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108ac0 0x7fa29c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 --2- 172.21.15.60:0/2523841734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c108140 0x7fa29c108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 --2- 172.21.15.60:0/2523841734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c102740 0x7fa29c102b20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- 172.21.15.60:0/2523841734 >> 172.21.15.60:0/2523841734 conn(0x7fa29c0fe180 msgr2=0x7fa29c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- 172.21.15.60:0/2523841734 shutdown_connections 2024-07-24T21:56:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- 172.21.15.60:0/2523841734 wait complete. 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 Processor -- start 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.842+0000 7fa2a2367640 1 -- start start 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 0x7fa29c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c108ac0 0x7fa29c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1126f0 con 0x7fa29c108ac0 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c112570 con 0x7fa29c102740 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a2367640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c112870 con 0x7fa29c108140 2024-07-24T21:56:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c108ac0 0x7fa29c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 0x7fa29c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58174/0 (socket says 172.21.15.60:58174) 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 -- 172.21.15.60:0/690470498 learned_addr learned my addr 172.21.15.60:0/690470498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 0x7fa29c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40996/0 (socket says 172.21.15.60:40996) 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 -- 172.21.15.60:0/690470498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 msgr2=0x7fa29c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 0x7fa29c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 -- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c108ac0 msgr2=0x7fa29c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c108ac0 0x7fa29c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 -- 172.21.15.60:0/690470498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c19ed10 con 0x7fa29c108140 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.843+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e9f0 tx=0x7fa29000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.844+0000 7fa2997fa640 1 -- 172.21.15.60:0/690470498 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000cd80 con 0x7fa29c108140 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.844+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c19f000 con 0x7fa29c108140 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.844+0000 7fa2997fa640 1 -- 172.21.15.60:0/690470498 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2900108e0 con 0x7fa29c108140 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.844+0000 7fa2997fa640 1 -- 172.21.15.60:0/690470498 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290018980 con 0x7fa29c108140 2024-07-24T21:56:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.844+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c1a56c0 con 0x7fa29c108140 2024-07-24T21:56:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.846+0000 7fa2997fa640 1 -- 172.21.15.60:0/690470498 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa290010430 con 0x7fa29c108140 2024-07-24T21:56:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 Processor -- start 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 -- start start 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8673c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55eac85dd280 con 0x55eac86678c0 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eac85cc780 con 0x55eac8674170 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a5197740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eac85d07c0 con 0x55eac86681e0 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2997fa640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8673c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41012/0 (socket says 172.21.15.60:41012) 2024-07-24T21:56:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2909390668 learned_addr learned my addr 172.21.15.60:0/2909390668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41378/0 (socket says 172.21.15.60:41378) 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2a0904640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.847+0000 7fa2997fa640 1 -- 172.21.15.60:0/690470498 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fa2900e4360 con 0x7fa29c108140 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2909390668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 msgr2=0x55eac8673c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8673c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2909390668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 msgr2=0x55eac8667ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2909390668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac8330790 con 0x55eac8674170 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa2a0904640 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29bfff640 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8673c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2900aaf60 tx=0x7fa2900787e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d53f1702ed5bbf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa2a0904640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 0x7fa2840b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa294005c40 tx=0x7fa294009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa2827fc640 1 -- 172.21.15.60:0/2909390668 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900aafd0 con 0x55eac8674170 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa2827fc640 1 -- 172.21.15.60:0/2909390668 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290084e50 con 0x55eac8674170 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.848+0000 7fa2827fc640 1 -- 172.21.15.60:0/2909390668 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900a34f0 con 0x55eac8674170 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- 172.21.15.60:0/2909390668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 msgr2=0x55eac8676560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2900aaf60 tx=0x7fa2900787e0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- 172.21.15.60:0/2909390668 shutdown_connections 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac8676560 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8673c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2909390668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86678c0 0x55eac8667ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- 172.21.15.60:0/2909390668 >> 172.21.15.60:0/2909390668 conn(0x55eac86656f0 msgr2=0x55eac8666890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- 172.21.15.60:0/2909390668 shutdown_connections 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- 172.21.15.60:0/2909390668 wait complete. 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 -- start start 2024-07-24T21:56:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.849+0000 7fa2a5197740 1 Processor -- start 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55eac8506ab0 con 0x55eac8674170 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eac85549e0 con 0x55eac86678c0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eac8676e50 con 0x55eac86681e0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a5197740 10 client.0 _ll_register_callbacks cb 0x55eac82dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41028/0 (socket says 172.21.15.60:41028) 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58204/0 (socket says 172.21.15.60:58204) 2024-07-24T21:56:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 -- 172.21.15.60:0/2282649840 learned_addr learned my addr 172.21.15.60:0/2282649840 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41384/0 (socket says 172.21.15.60:41384) 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 -- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 msgr2=0x55eac8670d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 -- 172.21.15.60:0/2282649840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 msgr2=0x55eac8671280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 -- 172.21.15.60:0/2282649840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac83a9620 con 0x55eac86678c0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa2a0904640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa29bfff640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a930 tx=0x7fa28c032f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967c1e3a835b1250 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.850+0000 7fa27b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c035040 con 0x55eac86678c0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eac857bce0 con 0x55eac86678c0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fa28c031720 con 0x55eac86678c0 2024-07-24T21:56:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa28c031920 con 0x55eac86678c0 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 10 client.26864 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eac86825d0 con 0x55eac86678c0 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eac857bce0 con 0x55eac86678c0 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fa28c04d070 con 0x55eac86678c0 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 10 client.26864 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 20 client.26864 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac8685f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 4 client.26864 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.851+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eac85df1d0 con 0x55eac8683b30 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.852+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac8685f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.852+0000 7fa27b7fe640 10 client.26864 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.852+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac8685f40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ffd88dcfc86117 server_cookie=3f25cd5c9904b7f5 in_seq=0 out_seq=0 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa27b7fe640 1 -- 172.21.15.60:0/2282649840 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eac85df1d0 con 0x55eac8683b30 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa27b7fe640 10 client.26864 handle_command_reply: tid=0 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 client.26864 shutdown 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 msgr2=0x55eac8673860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa28c02a930 tx=0x7fa28c032f00 comp rx=0 tx=0).stop 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 shutdown_connections 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac8685f40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac8671280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa27b7fe640 0 client.26864 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac8670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 --2- 172.21.15.60:0/2282649840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac8673860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 >> 172.21.15.60:0/2282649840 conn(0x55eac86656f0 msgr2=0x55eac866b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 shutdown_connections 2024-07-24T21:56:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 1 -- 172.21.15.60:0/2282649840 wait complete. 2024-07-24T21:56:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.853+0000 7fa2a5197740 20 client.26864 trim_cache size 0 max 16384 2024-07-24T21:56:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.870+0000 7fa2a5197740 1 Processor -- start 2024-07-24T21:56:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.870+0000 7fa2a5197740 1 -- start start 2024-07-24T21:56:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86678c0 0x55eac8666740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac868b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b0ab0 con 0x55eac86681e0 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ae2f0 con 0x55eac8674170 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a5197740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ac160 con 0x55eac86678c0 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41400/0 (socket says 172.21.15.60:41400) 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 -- 172.21.15.60:0/806789150 learned_addr learned my addr 172.21.15.60:0/806789150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29bfff640 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86678c0 0x55eac8666740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa2a0904640 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac868b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 -- 172.21.15.60:0/806789150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86678c0 msgr2=0x55eac8666740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86678c0 0x55eac8666740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 -- 172.21.15.60:0/806789150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 msgr2=0x55eac868b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac868b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 -- 172.21.15.60:0/806789150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac866fc10 con 0x55eac86681e0 2024-07-24T21:56:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.871+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa2900aacd0 tx=0x7fa290010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f7bc144adb9408 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa281ffb640 1 -- 172.21.15.60:0/806789150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900c4030 con 0x55eac86681e0 2024-07-24T21:56:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa281ffb640 1 -- 172.21.15.60:0/806789150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2900a7260 con 0x55eac86681e0 2024-07-24T21:56:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa281ffb640 1 -- 172.21.15.60:0/806789150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290018d80 con 0x55eac86681e0 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 -- 172.21.15.60:0/806789150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 msgr2=0x55eac8666c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa2900aacd0 tx=0x7fa290010ca0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 -- 172.21.15.60:0/806789150 shutdown_connections 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac8674170 0x55eac868b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac86681e0 0x55eac8666c80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 --2- 172.21.15.60:0/806789150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86678c0 0x55eac8666740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 -- 172.21.15.60:0/806789150 >> 172.21.15.60:0/806789150 conn(0x55eac86656f0 msgr2=0x55eac8670d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 -- 172.21.15.60:0/806789150 shutdown_connections 2024-07-24T21:56:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.872+0000 7fa2a5197740 1 -- 172.21.15.60:0/806789150 wait complete. 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 -- start start 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 Processor -- start 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac867c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac867b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ac7b0 con 0x55eac8674170 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1adc20 con 0x55eac86678c0 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ace80 con 0x55eac86681e0 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a5197740 10 client.0 _ll_register_callbacks cb 0x7fa28c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa29bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac867c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a0904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41408/0 (socket says 172.21.15.60:41408) 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa2a0904640 1 -- 172.21.15.60:0/186498162 learned_addr learned my addr 172.21.15.60:0/186498162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.873+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac867b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 -- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 msgr2=0x55eac867b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac867b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 -- 172.21.15.60:0/186498162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 msgr2=0x55eac867c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac867c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 -- 172.21.15.60:0/186498162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac867bdc0 con 0x55eac8674170 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa29bfff640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac867c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a0904640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa294003e90 tx=0x7fa294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40fd384a5d3be48 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa27b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29402f060 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7fa294020860 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2900a7260 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa294020a60 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a5197740 10 client.17451 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.874+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eac86ed7c0 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2900a7260 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fa29401d040 con 0x55eac8674170 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 10 client.17451 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 20 client.17451 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac86f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 4 client.17451 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:56:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eac85dac70 con 0x55eac8683b30 2024-07-24T21:56:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa29bfff640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac86f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa29bfff640 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac86f0010 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb0d9180a39de45 server_cookie=6aef340350c447cc in_seq=0 out_seq=0 2024-07-24T21:56:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.875+0000 7fa27b7fe640 10 client.17451 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.883+0000 7fa27b7fe640 1 -- 172.21.15.60:0/186498162 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eac85dac70 con 0x55eac8683b30 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.883+0000 7fa27b7fe640 10 client.17451 handle_command_reply: tid=0 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.883+0000 7fa2a5197740 1 client.17451 shutdown 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 msgr2=0x55eac867b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa294003e90 tx=0x7fa294004a40 comp rx=0 tx=0).stop 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 shutdown_connections 2024-07-24T21:56:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55eac8683b30 0x55eac86f0010 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eac8674170 0x55eac867b6b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eac86681e0 0x55eac867b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa27b7fe640 0 client.17451 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 --2- 172.21.15.60:0/186498162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eac86678c0 0x55eac867c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 >> 172.21.15.60:0/186498162 conn(0x55eac86656f0 msgr2=0x55eac86647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 shutdown_connections 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 1 -- 172.21.15.60:0/186498162 wait complete. 2024-07-24T21:56:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a5197740 20 client.17451 trim_cache size 0 max 16384 2024-07-24T21:56:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:56:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:56:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:56:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.884+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 msgr2=0x7fa2840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 0x7fa2840b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa294005c40 tx=0x7fa294009500 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 msgr2=0x7fa29c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e9f0 tx=0x7fa29000eeb0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 shutdown_connections 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c108ac0 0x7fa29c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c108140 0x7fa29c1991b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 --2- 172.21.15.60:0/690470498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa29c102740 0x7fa29c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 >> 172.21.15.60:0/690470498 conn(0x7fa29c0fe180 msgr2=0x7fa29c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 shutdown_connections 2024-07-24T21:56:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:05.885+0000 7fa2a2367640 1 -- 172.21.15.60:0/690470498 wait complete. 2024-07-24T21:56:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:05 smithi123 ceph-mon[37143]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 951 op/s 2024-07-24T21:56:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:05 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:05 smithi086 ceph-mon[37128]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 951 op/s 2024-07-24T21:56:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:05 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:05 smithi060 ceph-mon[33000]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 951 op/s 2024-07-24T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:06.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:05 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 945 op/s 2024-07-24T21:56:07.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:56:07.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:56:07.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 945 op/s 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 945 op/s 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:56:07.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:56:08.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:08 smithi086 ceph-mon[37128]: osdmap e95: 12 total, 12 up, 12 in 2024-07-24T21:56:08.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:08 smithi060 ceph-mon[33000]: osdmap e95: 12 total, 12 up, 12 in 2024-07-24T21:56:08.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:08 smithi123 ceph-mon[37143]: osdmap e95: 12 total, 12 up, 12 in 2024-07-24T21:56:09.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:09 smithi086 ceph-mon[37128]: pgmap v669: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-07-24T21:56:09.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:09 smithi060 ceph-mon[33000]: pgmap v669: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-07-24T21:56:09.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:09 smithi123 ceph-mon[37143]: pgmap v669: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-07-24T21:56:11.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:11 smithi086 ceph-mon[37128]: pgmap v670: 97 pgs: 14 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 58 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-07-24T21:56:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:11 smithi123 ceph-mon[37143]: pgmap v670: 97 pgs: 14 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 58 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-07-24T21:56:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:11 smithi060 ceph-mon[33000]: pgmap v670: 97 pgs: 14 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 58 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-07-24T21:56:13.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:13 smithi123 ceph-mon[37143]: pgmap v671: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 502 op/s 2024-07-24T21:56:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:13 smithi086 ceph-mon[37128]: pgmap v671: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 502 op/s 2024-07-24T21:56:13.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:13 smithi060 ceph-mon[33000]: pgmap v671: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 502 op/s 2024-07-24T21:56:15.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:15 smithi123 ceph-mon[37143]: pgmap v672: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 488 op/s 2024-07-24T21:56:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:15 smithi086 ceph-mon[37128]: pgmap v672: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 488 op/s 2024-07-24T21:56:15.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:15 smithi060 ceph-mon[33000]: pgmap v672: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 488 op/s 2024-07-24T21:56:16.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:56:17.048 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:17.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:17 smithi060 ceph-mon[33000]: pgmap v673: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-07-24T21:56:17.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:17 smithi123 ceph-mon[37143]: pgmap v673: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-07-24T21:56:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:17 smithi086 ceph-mon[37128]: pgmap v673: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-07-24T21:56:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.051+0000 7fc39dca3640 1 -- 172.21.15.60:0/524208689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109b40 msgr2=0x7fc398110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.051+0000 7fc39dca3640 1 --2- 172.21.15.60:0/524208689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109b40 0x7fc398110ba0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ca0 tx=0x7fc38802a8f0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.056+0000 7fc39dca3640 1 -- 172.21.15.60:0/524208689 shutdown_connections 2024-07-24T21:56:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.056+0000 7fc39dca3640 1 --2- 172.21.15.60:0/524208689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc39811ba90 0x7fc39811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.056+0000 7fc39dca3640 1 --2- 172.21.15.60:0/524208689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398109f20 0x7fc39810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.056+0000 7fc39dca3640 1 --2- 172.21.15.60:0/524208689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109b40 0x7fc398110ba0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.056+0000 7fc39dca3640 1 -- 172.21.15.60:0/524208689 >> 172.21.15.60:0/524208689 conn(0x7fc39806d760 msgr2=0x7fc39806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.059+0000 7fc39dca3640 1 -- 172.21.15.60:0/524208689 shutdown_connections 2024-07-24T21:56:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.063+0000 7fc39dca3640 1 -- 172.21.15.60:0/524208689 wait complete. 2024-07-24T21:56:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.063+0000 7fc39dca3640 1 Processor -- start 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 -- start start 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 0x7fc3981a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109f20 0x7fc3981a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 0x7fc3981ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc398121560 con 0x7fc398109f20 2024-07-24T21:56:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981213e0 con 0x7fc398109b40 2024-07-24T21:56:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.065+0000 7fc39dca3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981216e0 con 0x7fc39811ba90 2024-07-24T21:56:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.067+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 0x7fc3981a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.067+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 0x7fc3981a7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37688/0 (socket says 172.21.15.60:37688) 2024-07-24T21:56:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.067+0000 7fc3977fe640 1 -- 172.21.15.60:0/1703443625 learned_addr learned my addr 172.21.15.60:0/1703443625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.069+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 0x7fc3981ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.070+0000 7fc397fff640 1 -- 172.21.15.60:0/1703443625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 msgr2=0x7fc3981a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.070+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 0x7fc3981a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.070+0000 7fc397fff640 1 -- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109f20 msgr2=0x7fc3981a7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.070+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109f20 0x7fc3981a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.070+0000 7fc397fff640 1 -- 172.21.15.60:0/1703443625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981adac0 con 0x7fc39811ba90 2024-07-24T21:56:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.071+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 0x7fc3981ad2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc398074d40 tx=0x7fc39000dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.073+0000 7fc394ff9640 1 -- 172.21.15.60:0/1703443625 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390017070 con 0x7fc39811ba90 2024-07-24T21:56:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.073+0000 7fc394ff9640 1 -- 172.21.15.60:0/1703443625 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3900047a0 con 0x7fc39811ba90 2024-07-24T21:56:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.073+0000 7fc394ff9640 1 -- 172.21.15.60:0/1703443625 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390009ba0 con 0x7fc39811ba90 2024-07-24T21:56:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.075+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981addb0 con 0x7fc39811ba90 2024-07-24T21:56:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.075+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc398111270 con 0x7fc39811ba90 2024-07-24T21:56:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.089+0000 7fc3a09f5740 1 Processor -- start 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 -- start start 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f6207ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f6213c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f6216560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c5f617d280 con 0x55c5f62078c0 2024-07-24T21:56:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5f616c780 con 0x55c5f6214170 2024-07-24T21:56:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.090+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c5f61707c0 con 0x55c5f62081e0 2024-07-24T21:56:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.091+0000 7fc394ff9640 1 -- 172.21.15.60:0/1703443625 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc390005220 con 0x7fc39811ba90 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.091+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f6216560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.091+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f6216560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37704/0 (socket says 172.21.15.60:37704) 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.091+0000 7fc396ffd640 1 -- 172.21.15.60:0/839813659 learned_addr learned my addr 172.21.15.60:0/839813659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.092+0000 7fc394ff9640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 0x7fc3840b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.093+0000 7fc394ff9640 1 -- 172.21.15.60:0/1703443625 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc3900e3c30 con 0x7fc39811ba90 2024-07-24T21:56:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.093+0000 7fc396ffd640 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f6207ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc3977fe640 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f6213c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc396ffd640 1 -- 172.21.15.60:0/839813659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 msgr2=0x55c5f6213c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc396ffd640 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f6213c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc396ffd640 1 -- 172.21.15.60:0/839813659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 msgr2=0x55c5f6216560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc396ffd640 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f6216560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc396ffd640 1 -- 172.21.15.60:0/839813659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f5ed0790 con 0x55c5f62078c0 2024-07-24T21:56:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.094+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 0x7fc3840b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.095+0000 7fc396ffd640 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f6207ca0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3800121e0 tx=0x7fc380006920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee84a34551e78963 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.096+0000 7fc397fff640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 0x7fc3840b6410 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc3900a7a70 tx=0x7fc3900a6040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.096+0000 7fc38dffb640 1 -- 172.21.15.60:0/839813659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004020 con 0x55c5f62078c0 2024-07-24T21:56:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.096+0000 7fc38dffb640 1 -- 172.21.15.60:0/839813659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3800060e0 con 0x55c5f62078c0 2024-07-24T21:56:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.096+0000 7fc38dffb640 1 -- 172.21.15.60:0/839813659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004b60 con 0x55c5f62078c0 2024-07-24T21:56:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.097+0000 7fc3a09f5740 1 -- 172.21.15.60:0/839813659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 msgr2=0x55c5f6207ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.097+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f6207ca0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc3800121e0 tx=0x7fc380006920 comp rx=0 tx=0).stop 2024-07-24T21:56:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.100+0000 7fc3a09f5740 1 -- 172.21.15.60:0/839813659 shutdown_connections 2024-07-24T21:56:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.100+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f6216560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.100+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f6213c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.100+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/839813659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f6207ca0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.100+0000 7fc3a09f5740 1 -- 172.21.15.60:0/839813659 >> 172.21.15.60:0/839813659 conn(0x55c5f62056f0 msgr2=0x55c5f6206890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.102+0000 7fc3a09f5740 1 -- 172.21.15.60:0/839813659 shutdown_connections 2024-07-24T21:56:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.106+0000 7fc3a09f5740 1 -- 172.21.15.60:0/839813659 wait complete. 2024-07-24T21:56:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.106+0000 7fc3a09f5740 1 -- start start 2024-07-24T21:56:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.106+0000 7fc3a09f5740 1 Processor -- start 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 0x55c5f62139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 0x55c5f6210e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c5f60a6ab0 con 0x55c5f62081e0 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5f60f49e0 con 0x55c5f6214170 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c5f6216e50 con 0x55c5f62078c0 2024-07-24T21:56:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.112+0000 7fc3a09f5740 10 client.0 _ll_register_callbacks cb 0x55c5f5e7ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 0x55c5f62139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 0x55c5f62139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37642/0 (socket says 172.21.15.60:37642) 2024-07-24T21:56:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc396ffd640 1 -- 172.21.15.60:0/2406172622 learned_addr learned my addr 172.21.15.60:0/2406172622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 0x55c5f6210e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.115+0000 7fc3977fe640 1 -- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 msgr2=0x55c5f62139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 0x55c5f62139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc3977fe640 1 -- 172.21.15.60:0/2406172622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 msgr2=0x55c5f62113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc3977fe640 1 -- 172.21.15.60:0/2406172622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f5f49620 con 0x55c5f62081e0 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 0x55c5f6210e80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc388006010 tx=0x7fc388031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d253ef77bff46c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.116+0000 7fc376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.117+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388003970 con 0x55c5f62081e0 2024-07-24T21:56:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.117+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5f611bce0 con 0x55c5f62081e0 2024-07-24T21:56:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.118+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc388003b10 con 0x55c5f62081e0 2024-07-24T21:56:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.121+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc388003d10 con 0x55c5f62081e0 2024-07-24T21:56:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.121+0000 7fc3a09f5740 10 client.17460 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.121+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c5f6222710 con 0x55c5f62081e0 2024-07-24T21:56:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.122+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5f611bce0 con 0x55c5f62081e0 2024-07-24T21:56:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc38802f850 con 0x55c5f62081e0 2024-07-24T21:56:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 10 client.17460 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 10 client.17460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 20 client.17460 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f6223ff0 0x55c5f6226410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 4 client.17460 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.123+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5f617f1d0 con 0x55c5f6223ff0 2024-07-24T21:56:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.124+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f6223ff0 0x55c5f6226410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.129+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f6223ff0 0x55c5f6226410 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea05e296a8de958 server_cookie=d2d24cbb8e118970 in_seq=0 out_seq=0 2024-07-24T21:56:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.129+0000 7fc376ffd640 10 client.17460 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.130+0000 7fc376ffd640 1 -- 172.21.15.60:0/2406172622 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5f617f1d0 con 0x55c5f6223ff0 2024-07-24T21:56:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.130+0000 7fc376ffd640 10 client.17460 handle_command_reply: tid=0 2024-07-24T21:56:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.130+0000 7fc3a09f5740 1 client.17460 shutdown 2024-07-24T21:56:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.131+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 msgr2=0x55c5f6210e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.131+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 0x55c5f6210e80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc388006010 tx=0x7fc388031480 comp rx=0 tx=0).stop 2024-07-24T21:56:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 shutdown_connections 2024-07-24T21:56:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f6223ff0 0x55c5f6226410 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62081e0 0x55c5f6210e80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2406172622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62078c0 0x55c5f62139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.134+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 >> 172.21.15.60:0/2406172622 conn(0x55c5f62056f0 msgr2=0x55c5f620b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.135+0000 7fc376ffd640 0 client.17460 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.138+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 shutdown_connections 2024-07-24T21:56:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.141+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2406172622 wait complete. 2024-07-24T21:56:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.143+0000 7fc3a09f5740 20 client.17460 trim_cache size 0 max 16384 2024-07-24T21:56:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.174+0000 7fc3a09f5740 1 Processor -- start 2024-07-24T21:56:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc3a09f5740 1 -- start start 2024-07-24T21:56:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f621ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f6214170 0x55c5f622bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.175+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37734/0 (socket says 172.21.15.60:37734) 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 -- 172.21.15.60:0/2177420955 learned_addr learned my addr 172.21.15.60:0/2177420955 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981c80c0 con 0x55c5f6214170 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981c5900 con 0x55c5f62078c0 2024-07-24T21:56:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810a360 con 0x55c5f62081e0 2024-07-24T21:56:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f6214170 0x55c5f622bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 -- 172.21.15.60:0/2177420955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 msgr2=0x55c5f621ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:56:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f621ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 -- 172.21.15.60:0/2177420955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f6214170 msgr2=0x55c5f622bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f6214170 0x55c5f622bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.176+0000 7fc396ffd640 1 -- 172.21.15.60:0/2177420955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f62085c0 con 0x55c5f62078c0 2024-07-24T21:56:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.177+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bbe0 tx=0x7fc380009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39532df622912adb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.180+0000 7fc38d7fa640 1 -- 172.21.15.60:0/2177420955 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004220 con 0x55c5f62078c0 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.180+0000 7fc38d7fa640 1 -- 172.21.15.60:0/2177420955 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3800043c0 con 0x55c5f62078c0 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.180+0000 7fc38d7fa640 1 -- 172.21.15.60:0/2177420955 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380003980 con 0x55c5f62078c0 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 msgr2=0x55c5f6211360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc38000bbe0 tx=0x7fc380009040 comp rx=0 tx=0).stop 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 shutdown_connections 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f6214170 0x55c5f622bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f621ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2177420955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f62078c0 0x55c5f6211360 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 >> 172.21.15.60:0/2177420955 conn(0x55c5f62056f0 msgr2=0x55c5f620c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 shutdown_connections 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2177420955 wait complete. 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.181+0000 7fc3a09f5740 1 -- start start 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.182+0000 7fc3a09f5740 1 Processor -- start 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.182+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.182+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f62170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.182+0000 7fc3a09f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc398118460 con 0x55c5f62078c0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc398119850 con 0x55c5f6214170 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3a09f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc398118ab0 con 0x55c5f62081e0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37670/0 (socket says 172.21.15.60:37670) 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f62170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc396ffd640 1 -- 172.21.15.60:0/2030088035 learned_addr learned my addr 172.21.15.60:0/2030088035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3a09f5740 10 client.0 _ll_register_callbacks cb 0x7fc38802f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 -- 172.21.15.60:0/2030088035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 msgr2=0x55c5f62175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 -- 172.21.15.60:0/2030088035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 msgr2=0x55c5f621c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 -- 172.21.15.60:0/2030088035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f6217d70 con 0x55c5f62081e0 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc3977fe640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f62170a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc38802fca0 tx=0x7fc388009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d60873e431e2e7b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.183+0000 7fc376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880305f0 con 0x55c5f62081e0 2024-07-24T21:56:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3800043c0 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc38802f850 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc38802fe90 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 10 client.26893 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c5f628dc60 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3800043c0 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc388004120 con 0x55c5f62081e0 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 10 client.26893 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 20 client.26893 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.184+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f628e3e0 0x55c5f6290800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.185+0000 7fc3a09f5740 4 client.26893 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.185+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5f617ac70 con 0x55c5f628e3e0 2024-07-24T21:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.185+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f628e3e0 0x55c5f6290800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.186+0000 7fc396ffd640 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f628e3e0 0x55c5f6290800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2227292e696517a0 server_cookie=ecbc2410b4daa33c in_seq=0 out_seq=0 2024-07-24T21:56:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.186+0000 7fc376ffd640 10 client.26893 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.217+0000 7fc376ffd640 1 -- 172.21.15.60:0/2030088035 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5f617ac70 con 0x55c5f628e3e0 2024-07-24T21:56:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.217+0000 7fc376ffd640 10 client.26893 handle_command_reply: tid=0 2024-07-24T21:56:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.217+0000 7fc3a09f5740 1 client.26893 shutdown 2024-07-24T21:56:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 msgr2=0x55c5f62170a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f62170a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc38802fca0 tx=0x7fc388009360 comp rx=0 tx=0).stop 2024-07-24T21:56:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 shutdown_connections 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5f628e3e0 0x55c5f6290800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc376ffd640 0 client.26893 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5f6214170 0x55c5f62175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5f62081e0 0x55c5f62170a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 --2- 172.21.15.60:0/2030088035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5f62078c0 0x55c5f621c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 >> 172.21.15.60:0/2030088035 conn(0x55c5f62056f0 msgr2=0x55c5f620aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 shutdown_connections 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 1 -- 172.21.15.60:0/2030088035 wait complete. 2024-07-24T21:56:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.218+0000 7fc3a09f5740 20 client.26893 trim_cache size 0 max 16384 2024-07-24T21:56:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:56:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:56:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:56:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:56:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 msgr2=0x7fc3840b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 0x7fc3840b6410 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc3900a7a70 tx=0x7fc3900a6040 comp rx=0 tx=0).stop 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 msgr2=0x7fc3981ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 0x7fc3981ad2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc398074d40 tx=0x7fc39000dae0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 shutdown_connections 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3840b3f50 0x7fc3840b6410 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc39811ba90 0x7fc3981ad2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398109f20 0x7fc3981a7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 --2- 172.21.15.60:0/1703443625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc398109b40 0x7fc3981a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 >> 172.21.15.60:0/1703443625 conn(0x7fc39806d760 msgr2=0x7fc39811bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 shutdown_connections 2024-07-24T21:56:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:19.219+0000 7fc39dca3640 1 -- 172.21.15.60:0/1703443625 wait complete. 2024-07-24T21:56:19.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:19 smithi060 ceph-mon[33000]: pgmap v674: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 397 op/s 2024-07-24T21:56:19.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:19 smithi123 ceph-mon[37143]: pgmap v674: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 397 op/s 2024-07-24T21:56:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:19 smithi086 ceph-mon[37128]: pgmap v674: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 397 op/s 2024-07-24T21:56:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.737+0000 7fc8f88e4640 1 -- 172.21.15.60:0/803789828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 msgr2=0x7fc8f4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.737+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/803789828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4101a30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009900 tx=0x7fc8e402a410 comp rx=0 tx=0).stop 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 -- 172.21.15.60:0/803789828 shutdown_connections 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/803789828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f41079e0 0x7fc8f410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/803789828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f4107060 0x7fc8f41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/803789828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4101a30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 -- 172.21.15.60:0/803789828 >> 172.21.15.60:0/803789828 conn(0x7fc8f40fd0c0 msgr2=0x7fc8f40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 -- 172.21.15.60:0/803789828 shutdown_connections 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 -- 172.21.15.60:0/803789828 wait complete. 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.738+0000 7fc8f88e4640 1 Processor -- start 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 -- start start 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 0x7fc8f4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f41079e0 0x7fc8f419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41115f0 con 0x7fc8f4101650 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111470 con 0x7fc8f41079e0 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f88e4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111770 con 0x7fc8f4107060 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37684/0 (socket says 172.21.15.60:37684) 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 0x7fc8f4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 -- 172.21.15.60:0/922126067 learned_addr learned my addr 172.21.15.60:0/922126067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 -- 172.21.15.60:0/922126067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 msgr2=0x7fc8f4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 0x7fc8f4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 -- 172.21.15.60:0/922126067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f41079e0 msgr2=0x7fc8f419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f41079e0 0x7fc8f419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f259c640 1 -- 172.21.15.60:0/922126067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419ece0 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.739+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 0x7fc8f4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8f259c640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005940 tx=0x7fc8e40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8e37fe640 1 -- 172.21.15.60:0/922126067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e403f030 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419ef70 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8e37fe640 1 -- 172.21.15.60:0/922126067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40044c0 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8e37fe640 1 -- 172.21.15.60:0/922126067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4035070 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.740+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a5690 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.742+0000 7fc8e37fe640 1 -- 172.21.15.60:0/922126067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc8e4042030 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.743+0000 7fc8e37fe640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 0x7fc8dc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.743+0000 7fc8fac95740 1 Processor -- start 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8e37fe640 1 -- 172.21.15.60:0/922126067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc8e4104490 con 0x7fc8f4101650 2024-07-24T21:56:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 0x7fc8dc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 -- start start 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 0x562f9305c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562f92fc4960 con 0x562f9304dfa0 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f92fb3e60 con 0x562f9304d680 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8fac95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f92fb7ea0 con 0x562f93059f30 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37768/0 (socket says 172.21.15.60:37768) 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 -- 172.21.15.60:0/2283113089 learned_addr learned my addr 172.21.15.60:0/2283113089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 0x7fc8dc0b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4106870 tx=0x7fc8e8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 0x562f9305c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 -- 172.21.15.60:0/2283113089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 msgr2=0x562f9305c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 0x562f9305c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 -- 172.21.15.60:0/2283113089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 msgr2=0x562f930599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 -- 172.21.15.60:0/2283113089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f92f3d2d0 con 0x562f9304d680 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 0x562f9305c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.745+0000 7fc8f259c640 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4032af0 tx=0x7fc8e40caaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7001aa3f3ad6ee7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.746+0000 7fc8e0ff9640 1 -- 172.21.15.60:0/2283113089 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4049040 con 0x562f9304d680 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.746+0000 7fc8e0ff9640 1 -- 172.21.15.60:0/2283113089 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40cf5e0 con 0x562f9304d680 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.746+0000 7fc8e0ff9640 1 -- 172.21.15.60:0/2283113089 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40cb4a0 con 0x562f9304d680 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 -- 172.21.15.60:0/2283113089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 msgr2=0x562f9304da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4032af0 tx=0x7fc8e40caaf0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 -- 172.21.15.60:0/2283113089 shutdown_connections 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f93059f30 0x562f9305c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2283113089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f9304da60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.747+0000 7fc8fac95740 1 -- 172.21.15.60:0/2283113089 >> 172.21.15.60:0/2283113089 conn(0x562f9304b4b0 msgr2=0x562f9304c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.748+0000 7fc8fac95740 1 -- 172.21.15.60:0/2283113089 shutdown_connections 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.748+0000 7fc8fac95740 1 -- 172.21.15.60:0/2283113089 wait complete. 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.748+0000 7fc8fac95740 1 -- start start 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.748+0000 7fc8fac95740 1 Processor -- start 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304d680 0x562f93056eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93057930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37702/0 (socket says 172.21.15.60:37702) 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/4053295378 learned_addr learned my addr 172.21.15.60:0/4053295378 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562f92e17700 con 0x562f9304dfa0 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f92f565a0 con 0x562f93059f30 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8f259c640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304d680 0x562f93056eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.749+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562f93060d80 con 0x562f9304d680 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8fac95740 10 client.0 _ll_register_callbacks cb 0x562f92f76910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304d680 msgr2=0x562f93056eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304d680 0x562f93056eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/4053295378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 msgr2=0x562f93057930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93057930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f92f49630 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8003710 tx=0x7fc8e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f59ca17425f3593 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e40cf5e0 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.750+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/4053295378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800f910 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.751+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/4053295378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc8e800fab0 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.751+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/4053295378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc8e800fcb0 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.751+0000 7fc8fac95740 10 client.17475 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.751+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=41}) -- 0x562f92d14650 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.751+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/4053295378 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc8e801a070 con 0x562f9304dfa0 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 10 client.17475 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 20 client.17475 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f9306fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 4 client.17475 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8e40cf5e0 con 0x562f9306da70 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8f259c640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f9306fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8f259c640 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f9306fe60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4acd7f5cc7947990 server_cookie=d2542e5ae0115f8a in_seq=0 out_seq=0 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.752+0000 7fc8d9ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/4053295378 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc8e40cf5e0 con 0x562f9306da70 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8d9ffb640 10 client.17475 handle_command_reply: tid=0 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 client.17475 shutdown 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 msgr2=0x562f930573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8003710 tx=0x7fc8e8004730 comp rx=0 tx=0).stop 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 shutdown_connections 2024-07-24T21:56:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f9306fe60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8d9ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.753+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93057930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304dfa0 0x562f930573f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 1 --2- 172.21.15.60:0/4053295378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304d680 0x562f93056eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 >> 172.21.15.60:0/4053295378 conn(0x562f9304b4b0 msgr2=0x562f93051050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 shutdown_connections 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 1 -- 172.21.15.60:0/4053295378 wait complete. 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.754+0000 7fc8fac95740 20 client.17475 trim_cache size 0 max 16384 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 Processor -- start 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 -- start start 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 0x562f93060680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93075c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41ae8f0 con 0x562f9304d680 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41ac130 con 0x562f93059f30 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8fac95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41a9fa0 con 0x562f9304dfa0 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 0x562f93060680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37698/0 (socket says 172.21.15.60:37698) 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/3348715860 learned_addr learned my addr 172.21.15.60:0/3348715860 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 0x562f93060680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37708/0 (socket says 172.21.15.60:37708) 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.781+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93075c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/3348715860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 msgr2=0x562f93075c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93075c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/3348715860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 msgr2=0x562f93060680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 0x562f93060680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 -- 172.21.15.60:0/3348715860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f930701d0 con 0x562f9304dfa0 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8f1d9b640 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800fbd0 tx=0x7fc8e8022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311d902bd352e666 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8dbfff640 1 -- 172.21.15.60:0/3348715860 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e801d7a0 con 0x562f9304dfa0 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8dbfff640 1 -- 172.21.15.60:0/3348715860 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e8033020 con 0x562f9304dfa0 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.782+0000 7fc8dbfff640 1 -- 172.21.15.60:0/3348715860 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8027800 con 0x562f9304dfa0 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 -- 172.21.15.60:0/3348715860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 msgr2=0x562f93075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800fbd0 tx=0x7fc8e8022000 comp rx=0 tx=0).stop 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 -- 172.21.15.60:0/3348715860 shutdown_connections 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f93059f30 0x562f93075c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f93075730 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 --2- 172.21.15.60:0/3348715860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9304d680 0x562f93060680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 -- 172.21.15.60:0/3348715860 >> 172.21.15.60:0/3348715860 conn(0x562f9304b4b0 msgr2=0x562f93058d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 -- 172.21.15.60:0/3348715860 shutdown_connections 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.783+0000 7fc8fac95740 1 -- 172.21.15.60:0/3348715860 wait complete. 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.784+0000 7fc8fac95740 1 -- start start 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.784+0000 7fc8fac95740 1 Processor -- start 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.784+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f93066160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f930666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41aa5f0 con 0x562f93059f30 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41aba60 con 0x562f9304d680 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41aacc0 con 0x562f9304dfa0 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8fac95740 10 client.0 _ll_register_callbacks cb 0x7fc8e80040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37712/0 (socket says 172.21.15.60:37712) 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 -- 172.21.15.60:0/2197432489 learned_addr learned my addr 172.21.15.60:0/2197432489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 -- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 msgr2=0x562f930666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f930666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 -- 172.21.15.60:0/2197432489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 msgr2=0x562f93066160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f93066160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.785+0000 7fc8f2d9d640 1 -- 172.21.15.60:0/2197432489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9305deb0 con 0x562f93059f30 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8f2d9d640 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0042f0 tx=0x7fc8ec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe49c80142e22bc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec011920 con 0x562f93059f30 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7fc8ec011ac0 con 0x562f93059f30 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e8033020 con 0x562f93059f30 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc8ec011cc0 con 0x562f93059f30 2024-07-24T21:56:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8fac95740 10 client.17481 fetch_fsmap learned FSMap version 41 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.786+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562f93063e90 con 0x562f93059f30 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec01b8d0 con 0x562f93059f30 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc8ec0106e0 con 0x562f93059f30 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 10 client.17481 fetch_fsmap finished waiting for FSMap version 41 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 20 client.17481 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f930df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 4 client.17481 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.787+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc8e8033020 con 0x562f9306da70 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.788+0000 7fc8f259c640 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f930df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8d9ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8f259c640 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f930df4f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1199f02cdc66c7b7 server_cookie=8c549722d9ccf49b in_seq=0 out_seq=0 2024-07-24T21:56:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8d9ffb640 1 -- 172.21.15.60:0/2197432489 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc8e8033020 con 0x562f9306da70 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8d9ffb640 10 client.17481 handle_command_reply: tid=0 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 client.17481 shutdown 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 msgr2=0x562f9305d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0042f0 tx=0x7fc8ec003f90 comp rx=0 tx=0).stop 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 shutdown_connections 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:56:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x562f9306da70 0x562f930df4f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8d9ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f93059f30 0x562f9305d680 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f9304dfa0 0x562f930666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 --2- 172.21.15.60:0/2197432489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f9304d680 0x562f93066160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.789+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 >> 172.21.15.60:0/2197432489 conn(0x562f9304b4b0 msgr2=0x562f93049d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.790+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 shutdown_connections 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.790+0000 7fc8fac95740 1 -- 172.21.15.60:0/2197432489 wait complete. 2024-07-24T21:56:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.790+0000 7fc8fac95740 20 client.17481 trim_cache size 0 max 16384 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 msgr2=0x7fc8dc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 0x7fc8dc0b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4106870 tx=0x7fc8e8005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 msgr2=0x7fc8f4198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005940 tx=0x7fc8e40040c0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 shutdown_connections 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8dc0b3d40 0x7fc8dc0b6200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8f41079e0 0x7fc8f419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f4107060 0x7fc8f4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 --2- 172.21.15.60:0/922126067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f4101650 0x7fc8f4198c30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 >> 172.21.15.60:0/922126067 conn(0x7fc8f40fd0c0 msgr2=0x7fc8f406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 shutdown_connections 2024-07-24T21:56:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:19.791+0000 7fc8f88e4640 1 -- 172.21.15.60:0/922126067 wait complete. 2024-07-24T21:56:20.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:56:20.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:56:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:21.595 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:21 smithi060 ceph-mon[33000]: pgmap v675: 97 pgs: 15 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 51 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-07-24T21:56:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:21 smithi123 ceph-mon[37143]: pgmap v675: 97 pgs: 15 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 51 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-07-24T21:56:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:21 smithi086 ceph-mon[37128]: pgmap v675: 97 pgs: 15 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 51 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-07-24T21:56:23.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:23 smithi060 ceph-mon[33000]: pgmap v676: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-07-24T21:56:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:23 smithi123 ceph-mon[37143]: pgmap v676: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-07-24T21:56:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:23 smithi086 ceph-mon[37128]: pgmap v676: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-07-24T21:56:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.356+0000 7f25d2026640 1 -- 172.21.15.60:0/2335403670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc109b40 msgr2=0x7f25cc109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 --2- 172.21.15.60:0/2335403670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc109b40 0x7f25cc109f20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009900 tx=0x7f25bc02a480 comp rx=0 tx=0).stop 2024-07-24T21:56:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 -- 172.21.15.60:0/2335403670 shutdown_connections 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 --2- 172.21.15.60:0/2335403670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 0x7f25cc1177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 --2- 172.21.15.60:0/2335403670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc10a460 0x7f25cc114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 --2- 172.21.15.60:0/2335403670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc109b40 0x7f25cc109f20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 -- 172.21.15.60:0/2335403670 >> 172.21.15.60:0/2335403670 conn(0x7f25cc06d980 msgr2=0x7f25cc06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 -- 172.21.15.60:0/2335403670 shutdown_connections 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.357+0000 7f25d2026640 1 -- 172.21.15.60:0/2335403670 wait complete. 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 Processor -- start 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 -- start start 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc10a460 0x7f25cc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 0x7f25cc1a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc11af60 con 0x7f25cc10a460 2024-07-24T21:56:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc11ade0 con 0x7f25cc115400 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.358+0000 7f25d2026640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc11b0e0 con 0x7f25cc109b40 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 0x7f25cc1a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37736/0 (socket says 172.21.15.60:37736) 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 -- 172.21.15.60:0/2178842098 learned_addr learned my addr 172.21.15.60:0/2178842098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 -- 172.21.15.60:0/2178842098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 msgr2=0x7f25cc1a6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 0x7f25cc1a6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 -- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc10a460 msgr2=0x7f25cc1a19d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc10a460 0x7f25cc1a19d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 -- 172.21.15.60:0/2178842098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc1a7360 con 0x7f25cc109b40 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.359+0000 7f25cb7fe640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005ab0 tx=0x7f25bc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.360+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc03f070 con 0x7f25cc109b40 2024-07-24T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.360+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25bc031b60 con 0x7f25cc109b40 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.360+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc035030 con 0x7f25cc109b40 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.360+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc1a75f0 con 0x7f25cc109b40 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"btime":"2024-07-24T21:54:56:385204+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:54:55.586937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24610,24605,24598]},"id":1}]} 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.360+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc1a7a80 con 0x7f25cc109b40 2024-07-24T21:56:24.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.362+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25cc074300 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.364+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f25bc042070 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.365+0000 7f25c8ff9640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 0x7f25b40b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.365+0000 7f25caffd640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 0x7f25b40b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.365+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f25bc1046b0 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.365+0000 7f25caffd640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 0x7f25b40b6350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f25cc073e80 tx=0x7f25c0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.369+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f25bc049030 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.719+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25cc1a25c0 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.720+0000 7f25c8ff9640 1 -- 172.21.15.60:0/2178842098 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11920 (secure 0 0 0) 0x7f25cc1a25c0 con 0x7f25cc109b40 2024-07-24T21:56:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 msgr2=0x7f25b40b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 0x7f25b40b6350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f25cc073e80 tx=0x7f25c0005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 msgr2=0x7f25cc1a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005ab0 tx=0x7f25bc003c10 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 shutdown_connections 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f25b40b3e90 0x7f25b40b6350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25cc115400 0x7f25cc1a6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc10a460 0x7f25cc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 --2- 172.21.15.60:0/2178842098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25cc109b40 0x7f25cc1a1490 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 >> 172.21.15.60:0/2178842098 conn(0x7f25cc06d980 msgr2=0x7f25cc0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 shutdown_connections 2024-07-24T21:56:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:23.723+0000 7f25d2026640 1 -- 172.21.15.60:0/2178842098 wait complete. 2024-07-24T21:56:24.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:24 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2178842098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:24.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:24 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2178842098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:24.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:24 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2178842098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:24.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:56:25.169 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:25.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:25 smithi060 ceph-mon[33000]: pgmap v677: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 326 op/s 2024-07-24T21:56:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:25 smithi123 ceph-mon[37143]: pgmap v677: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 326 op/s 2024-07-24T21:56:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:25 smithi086 ceph-mon[37128]: pgmap v677: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 326 op/s 2024-07-24T21:56:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.385+0000 7ff068a26640 1 -- 172.21.15.60:0/3055522232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540318a0 con 0x7ff064107090 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 -- 172.21.15.60:0/3055522232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 msgr2=0x7ff0641074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/3055522232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff0641074d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff054009900 tx=0x7ff05402a410 comp rx=0 tx=0).stop 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 -- 172.21.15.60:0/3055522232 shutdown_connections 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/3055522232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064107a10 0x7ff06410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/3055522232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff0641074d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/3055522232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064101680 0x7ff064101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 -- 172.21.15.60:0/3055522232 >> 172.21.15.60:0/3055522232 conn(0x7ff0640fd110 msgr2=0x7ff0640ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 -- 172.21.15.60:0/3055522232 shutdown_connections 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.386+0000 7ff06bc8c640 1 -- 172.21.15.60:0/3055522232 wait complete. 2024-07-24T21:56:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 Processor -- start 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 -- start start 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064101680 0x7ff06419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff06419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff064111610 con 0x7ff064107090 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff064111490 con 0x7ff064107a10 2024-07-24T21:56:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06bc8c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff064111790 con 0x7ff064101680 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff069227640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff06419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06a229640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06a229640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40562/0 (socket says 172.21.15.60:40562) 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff069227640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff06419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41678/0 (socket says 172.21.15.60:41678) 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.387+0000 7ff06a229640 1 -- 172.21.15.60:0/2781441902 learned_addr learned my addr 172.21.15.60:0/2781441902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff069a28640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064101680 0x7ff06419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 -- 172.21.15.60:0/2781441902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064101680 msgr2=0x7ff06419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064101680 0x7ff06419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 -- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 msgr2=0x7ff06419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff06419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 -- 172.21.15.60:0/2781441902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0641a0f30 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06a229640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00eb10 tx=0x7ff05c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff05affd640 1 -- 172.21.15.60:0/2781441902 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004b20 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff05affd640 1 -- 172.21.15.60:0/2781441902 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c004cc0 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.388+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0641a1220 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.389+0000 7ff05affd640 1 -- 172.21.15.60:0/2781441902 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c010670 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.389+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0641a78e0 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.391+0000 7ff05affd640 1 -- 172.21.15.60:0/2781441902 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff05c005020 con 0x7ff064107a10 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 Processor -- start 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 -- start start 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55efa5c5d9b0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efa5c4ceb0 con 0x55efa5ce66d0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.392+0000 7ff06e03d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55efa5c50ef0 con 0x55efa5cf2f80 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff05affd640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 0x7ff0500b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40576/0 (socket says 172.21.15.60:40576) 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 -- 172.21.15.60:0/538176241 learned_addr learned my addr 172.21.15.60:0/538176241 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff06a229640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 0x7ff0500b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 -- 172.21.15.60:0/538176241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 msgr2=0x55efa5cf5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069a28640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 -- 172.21.15.60:0/538176241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 msgr2=0x55efa5cf2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 -- 172.21.15.60:0/538176241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa5bd6320 con 0x55efa5ce66d0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069a28640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff05affd640 1 -- 172.21.15.60:0/2781441902 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7ff05c0e4530 con 0x7ff064107a10 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff069227640 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff05402a930 tx=0x7ff054005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83176998dff9eff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.393+0000 7ff04ffff640 1 -- 172.21.15.60:0/538176241 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054035030 con 0x55efa5ce66d0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.394+0000 7ff04ffff640 1 -- 172.21.15.60:0/538176241 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff054004030 con 0x55efa5ce66d0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.394+0000 7ff04ffff640 1 -- 172.21.15.60:0/538176241 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540039b0 con 0x55efa5ce66d0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 -- 172.21.15.60:0/538176241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 msgr2=0x55efa5ce6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff05402a930 tx=0x7ff054005b50 comp rx=0 tx=0).stop 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06a229640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 0x7ff0500b6350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0e1cd0 tx=0x7ff05c0aaeb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 -- 172.21.15.60:0/538176241 shutdown_connections 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 --2- 172.21.15.60:0/538176241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5ce6ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 -- 172.21.15.60:0/538176241 >> 172.21.15.60:0/538176241 conn(0x55efa5ce4500 msgr2=0x55efa5ce56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 -- 172.21.15.60:0/538176241 shutdown_connections 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.395+0000 7ff06e03d740 1 -- 172.21.15.60:0/538176241 wait complete. 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 -- start start 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 Processor -- start 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5cefee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5cf2f80 0x55efa5cf0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55efa5ab0700 con 0x55efa5ce66d0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efa5bef5f0 con 0x55efa5cf2f80 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.396+0000 7ff06e03d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55efa5cf5ba0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38276/0 (socket says 172.21.15.60:38276) 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 -- 172.21.15.60:0/2529883811 learned_addr learned my addr 172.21.15.60:0/2529883811 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069227640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5cefee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff06e03d740 10 client.0 _ll_register_callbacks cb 0x55efa5c0f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069227640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5cf2f80 0x55efa5cf0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 -- 172.21.15.60:0/2529883811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5cf2f80 msgr2=0x55efa5cf0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5cf2f80 0x55efa5cf0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 -- 172.21.15.60:0/2529883811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 msgr2=0x55efa5cefee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5cefee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 -- 172.21.15.60:0/2529883811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa5be2680 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff069a28640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff060013710 tx=0x7ff060013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40900eef6149e6e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff04d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060004460 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7ff0600119c0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.397+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff054004030 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff060011bc0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff06e03d740 10 client.26923 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55efa59ad650 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff054004030 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff06001a070 con 0x55efa5ce6ff0 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff06e03d740 10 client.26923 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.398+0000 7ff06e03d740 10 client.26923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:56:27.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff06e03d740 20 client.26923 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d02a50 0x55efa5d04ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff06e03d740 4 client.26923 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efa5c5f900 con 0x55efa5d02a50 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff069227640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d02a50 0x55efa5d04ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff069227640 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d02a50 0x55efa5d04ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d130247a1a986054 server_cookie=5cd4e17544e9b249 in_seq=0 out_seq=0 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.399+0000 7ff04d7fa640 10 client.26923 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.400+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2529883811 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efa5c5f900 con 0x55efa5d02a50 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.400+0000 7ff04d7fa640 10 client.26923 handle_command_reply: tid=0 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.400+0000 7ff06e03d740 1 client.26923 shutdown 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.400+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 msgr2=0x55efa5cf0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.400+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff060013710 tx=0x7ff060013bd0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 shutdown_connections 2024-07-24T21:56:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d02a50 0x55efa5d04ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5cf2f80 0x55efa5cf0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf0420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff04d7fa640 0 client.26923 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2529883811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5cefee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 >> 172.21.15.60:0/2529883811 conn(0x55efa5ce4500 msgr2=0x55efa5cea0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 shutdown_connections 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 1 -- 172.21.15.60:0/2529883811 wait complete. 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.401+0000 7ff06e03d740 20 client.26923 trim_cache size 0 max 16384 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.424+0000 7ff06e03d740 1 Processor -- start 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.424+0000 7ff06e03d740 1 -- start start 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 0x55efa5ce9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5d0a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641b2dc0 con 0x55efa5ce66d0 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641b05d0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff06e03d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641ae440 con 0x55efa5cf2f80 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41710/0 (socket says 172.21.15.60:41710) 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 -- 172.21.15.60:0/1846487143 learned_addr learned my addr 172.21.15.60:0/1846487143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069a28640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 0x55efa5ce9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5d0a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 -- 172.21.15.60:0/1846487143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 msgr2=0x55efa5d0a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5d0a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 -- 172.21.15.60:0/1846487143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 msgr2=0x55efa5ce9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 0x55efa5ce9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.425+0000 7ff069227640 1 -- 172.21.15.60:0/1846487143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa5cf5ae0 con 0x55efa5ce66d0 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.426+0000 7ff069a28640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 0x55efa5ce9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.426+0000 7ff069227640 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff054033000 tx=0x7ff0540035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19786501f87d945b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.426+0000 7ff04f7fe640 1 -- 172.21.15.60:0/1846487143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054051020 con 0x55efa5ce66d0 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.426+0000 7ff04f7fe640 1 -- 172.21.15.60:0/1846487143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff054004750 con 0x55efa5ce66d0 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.426+0000 7ff04f7fe640 1 -- 172.21.15.60:0/1846487143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054035040 con 0x55efa5ce66d0 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 -- 172.21.15.60:0/1846487143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 msgr2=0x55efa5ce5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff054033000 tx=0x7ff0540035f0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 -- 172.21.15.60:0/1846487143 shutdown_connections 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5d0a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce6ff0 0x55efa5ce9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 --2- 172.21.15.60:0/1846487143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce66d0 0x55efa5ce5a00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.427+0000 7ff06e03d740 1 -- 172.21.15.60:0/1846487143 >> 172.21.15.60:0/1846487143 conn(0x55efa5ce4500 msgr2=0x55efa5ce9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.428+0000 7ff06e03d740 1 -- 172.21.15.60:0/1846487143 shutdown_connections 2024-07-24T21:56:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.428+0000 7ff06e03d740 1 -- 172.21.15.60:0/1846487143 wait complete. 2024-07-24T21:56:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.428+0000 7ff06e03d740 1 -- start start 2024-07-24T21:56:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.428+0000 7ff06e03d740 1 Processor -- start 2024-07-24T21:56:27.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5cfb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641aea90 con 0x55efa5ce6ff0 2024-07-24T21:56:27.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641aff00 con 0x55efa5ce66d0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.429+0000 7ff06e03d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641af160 con 0x55efa5cf2f80 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff06e03d740 10 client.0 _ll_register_callbacks cb 0x7ff0600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41714/0 (socket says 172.21.15.60:41714) 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 -- 172.21.15.60:0/2371446333 learned_addr learned my addr 172.21.15.60:0/2371446333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 -- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 msgr2=0x55efa5cf6d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf6d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 -- 172.21.15.60:0/2371446333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 msgr2=0x55efa5cfb360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5cfb360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 -- 172.21.15.60:0/2371446333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa5cf7410 con 0x55efa5ce6ff0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff069a28640 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e70 tx=0x7ff060011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9109b6ade57688a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.430+0000 7ff04d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060031060 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7ff0600035f0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff054004750 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff0600037f0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff06e03d740 10 client.17499 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55efa5d6d6c0 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff060003e40 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.431+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff06001a040 con 0x55efa5ce6ff0 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 10 client.17499 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 10 client.17499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 20 client.17499 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d6de80 0x55efa5d702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 4 client.17499 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff054004750 con 0x55efa5d6de80 2024-07-24T21:56:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff069227640 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d6de80 0x55efa5d702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff069227640 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d6de80 0x55efa5d702a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2f3a5606810f425 server_cookie=ba80c715cf1c9adb in_seq=0 out_seq=0 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff04d7fa640 10 client.17499 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.432+0000 7ff04d7fa640 1 -- 172.21.15.60:0/2371446333 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff054004750 con 0x55efa5d6de80 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff04d7fa640 10 client.17499 handle_command_reply: tid=0 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 client.17499 shutdown 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 msgr2=0x55efa5cf6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e70 tx=0x7ff060011f90 comp rx=0 tx=0).stop 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 shutdown_connections 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55efa5d6de80 0x55efa5d702a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efa5cf2f80 0x55efa5cf6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff04d7fa640 0 client.17499 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efa5ce6ff0 0x55efa5cf6830 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 --2- 172.21.15.60:0/2371446333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efa5ce66d0 0x55efa5cfb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 >> 172.21.15.60:0/2371446333 conn(0x55efa5ce4500 msgr2=0x55efa5cf1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 shutdown_connections 2024-07-24T21:56:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 1 -- 172.21.15.60:0/2371446333 wait complete. 2024-07-24T21:56:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.433+0000 7ff06e03d740 20 client.17499 trim_cache size 0 max 16384 2024-07-24T21:56:27.490 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 msgr2=0x7ff0500b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 0x7ff0500b6350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0e1cd0 tx=0x7ff05c0aaeb0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 msgr2=0x7ff0641a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00eb10 tx=0x7ff05c00c2d0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 shutdown_connections 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0500b3e90 0x7ff0500b6350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff064107a10 0x7ff0641a0760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff064107090 0x7ff06419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 --2- 172.21.15.60:0/2781441902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff064101680 0x7ff06419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.434+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 >> 172.21.15.60:0/2781441902 conn(0x7ff0640fd110 msgr2=0x7ff06406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.435+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 shutdown_connections 2024-07-24T21:56:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:27.435+0000 7ff06bc8c640 1 -- 172.21.15.60:0/2781441902 wait complete. 2024-07-24T21:56:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:27 smithi123 ceph-mon[37143]: pgmap v678: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-07-24T21:56:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:27 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:27 smithi086 ceph-mon[37128]: pgmap v678: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-07-24T21:56:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:27 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:27 smithi060 ceph-mon[33000]: pgmap v678: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 416 op/s 2024-07-24T21:56:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:27 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:28.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:56:29.329 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:29.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:29 smithi060 ceph-mon[33000]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 415 op/s 2024-07-24T21:56:29.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:29 smithi123 ceph-mon[37143]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 415 op/s 2024-07-24T21:56:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:29 smithi086 ceph-mon[37128]: pgmap v679: 97 pgs: 97 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 415 op/s 2024-07-24T21:56:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1315318273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 msgr2=0x7f68dc10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1315318273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc10ef10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009ca0 tx=0x7f68d402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1315318273 shutdown_connections 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1315318273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc10ef10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1315318273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 0x7f68dc10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1315318273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 0x7f68dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1315318273 >> 172.21.15.60:0/1315318273 conn(0x7f68dc0fd0c0 msgr2=0x7f68dc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1315318273 shutdown_connections 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.030+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1315318273 wait complete. 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 Processor -- start 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 -- start start 2024-07-24T21:56:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 0x7f68dc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 0x7f68dc1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc112710 con 0x7f68dc10caa0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc112590 con 0x7f68dc1034f0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68e0ef3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc112890 con 0x7f68dc103ef0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.031+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68da59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 0x7f68dc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41734/0 (socket says 172.21.15.60:41734) 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 -- 172.21.15.60:0/1414615726 learned_addr learned my addr 172.21.15.60:0/1414615726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 0x7f68dc1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 -- 172.21.15.60:0/1414615726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 msgr2=0x7f68dc1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 0x7f68dc1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 -- 172.21.15.60:0/1414615726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 msgr2=0x7f68dc198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 0x7f68dc198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 -- 172.21.15.60:0/1414615726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc19ec50 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68dad9d640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098d0 tx=0x7f68d40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68cb7fe640 1 -- 172.21.15.60:0/1414615726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4003b70 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68cb7fe640 1 -- 172.21.15.60:0/1414615726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d4003d10 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68cb7fe640 1 -- 172.21.15.60:0/1414615726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d402fa20 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc19eee0 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.032+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1a5600 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.035+0000 7f68e32a4740 1 Processor -- start 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.035+0000 7f68cb7fe640 1 -- 172.21.15.60:0/1414615726 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f68d402fc40 con 0x7f68dc10caa0 2024-07-24T21:56:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.035+0000 7f68e32a4740 1 -- start start 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e59780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563646dbd9b0 con 0x563646e466d0 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563646daceb0 con 0x563646e57390 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68e32a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563646db0ef0 con 0x563646e46ff0 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38304/0 (socket says 172.21.15.60:38304) 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e46ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41742/0 (socket says 172.21.15.60:41742) 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 -- 172.21.15.60:0/1770216327 learned_addr learned my addr 172.21.15.60:0/1770216327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e59780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 -- 172.21.15.60:0/1770216327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 msgr2=0x563646e59780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e59780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 -- 172.21.15.60:0/1770216327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 msgr2=0x563646e46ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e46ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 -- 172.21.15.60:0/1770216327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563646d36320 con 0x563646e46ff0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68cb7fe640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 0x7f68c40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68cb7fe640 1 -- 172.21.15.60:0/1414615726 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f68d41053d0 con 0x7f68dc10caa0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 0x7f68c40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.036+0000 7f68da59c640 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd80 tx=0x7f68cc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26f9ab9907259ee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68c8ff9640 1 -- 172.21.15.60:0/1770216327 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc005020 con 0x563646e46ff0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68c8ff9640 1 -- 172.21.15.60:0/1770216327 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0047a0 con 0x563646e46ff0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68c8ff9640 1 -- 172.21.15.60:0/1770216327 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc011680 con 0x563646e46ff0 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68e32a4740 1 -- 172.21.15.60:0/1770216327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 msgr2=0x563646e56e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd80 tx=0x7f68cc0107a0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.037+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 0x7f68c40b6280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0ff8d0 tx=0x7f68d00095d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- 172.21.15.60:0/1770216327 shutdown_connections 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e59780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e56e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1770216327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- 172.21.15.60:0/1770216327 >> 172.21.15.60:0/1770216327 conn(0x563646e44500 msgr2=0x563646e456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- 172.21.15.60:0/1770216327 shutdown_connections 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- 172.21.15.60:0/1770216327 wait complete. 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- start start 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 Processor -- start 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 0x563646e54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563646c10700 con 0x563646e57390 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563646d4f5f0 con 0x563646e466d0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563646e5a040 con 0x563646e46ff0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68e32a4740 10 client.0 _ll_register_callbacks cb 0x563646d6f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68da59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 0x563646e54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41756/0 (socket says 172.21.15.60:41756) 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.038+0000 7f68dad9d640 1 -- 172.21.15.60:0/1254369617 learned_addr learned my addr 172.21.15.60:0/1254369617 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 -- 172.21.15.60:0/1254369617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 msgr2=0x563646e548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 -- 172.21.15.60:0/1254369617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 msgr2=0x563646e54df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 -- 172.21.15.60:0/1254369617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563646d42680 con 0x563646e466d0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68dad9d640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68da59c640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 0x563646e54370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd50 tx=0x7f68cc00b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7bff625b87f943 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc004be0 con 0x563646e466d0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68cc0047a0 con 0x563646e466d0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f68cc004020 con 0x563646e466d0 2024-07-24T21:56:31.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68cc004220 con 0x563646e466d0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68e32a4740 10 client.26930 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.039+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563646b0d650 con 0x563646e466d0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0047a0 con 0x563646e466d0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f68cc0038f0 con 0x563646e466d0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 10 client.26930 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 10 client.26930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 20 client.26930 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646e66ee0 0x563646e692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 4 client.26930 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563646dbf900 con 0x563646e66ee0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.040+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646e66ee0 0x563646e692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.041+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646e66ee0 0x563646e692d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b380466887216b3 server_cookie=83e666fd074f8ef9 in_seq=0 out_seq=0 2024-07-24T21:56:31.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.041+0000 7f68c1ffb640 10 client.26930 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:56:31.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68c1ffb640 1 -- 172.21.15.60:0/1254369617 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563646dbf900 con 0x563646e66ee0 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68c1ffb640 10 client.26930 handle_command_reply: tid=0 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 client.26930 shutdown 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 msgr2=0x563646e54370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 0x563646e54370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd50 tx=0x7f68cc00b980 comp rx=0 tx=0).stop 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 shutdown_connections 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646e66ee0 0x563646e692d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e57390 0x563646e54df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e46ff0 0x563646e548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 --2- 172.21.15.60:0/1254369617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e466d0 0x563646e54370 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68c1ffb640 0 client.26930 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 >> 172.21.15.60:0/1254369617 conn(0x563646e44500 msgr2=0x563646e4a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 shutdown_connections 2024-07-24T21:56:31.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 1 -- 172.21.15.60:0/1254369617 wait complete. 2024-07-24T21:56:31.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.071+0000 7f68e32a4740 20 client.26930 trim_cache size 0 max 16384 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 Processor -- start 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 -- start start 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 0x563646e51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 0x563646e42d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1ae910 con 0x563646e466d0 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1ac120 con 0x563646e46ff0 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.085+0000 7f68e32a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1a9f90 con 0x563646e57390 2024-07-24T21:56:31.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41772/0 (socket says 172.21.15.60:41772) 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 -- 172.21.15.60:0/581679438 learned_addr learned my addr 172.21.15.60:0/581679438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 0x563646e51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68dad9d640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 0x563646e42d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 -- 172.21.15.60:0/581679438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 msgr2=0x563646e42d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 0x563646e42d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 -- 172.21.15.60:0/581679438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 msgr2=0x563646e51e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 0x563646e51e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 -- 172.21.15.60:0/581679438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563646e59ab0 con 0x563646e466d0 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68dad9d640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 0x563646e42d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 0x563646e51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:31.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68da59c640 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd80 tx=0x7f68cc010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337d825f1bff0493 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68c3fff640 1 -- 172.21.15.60:0/581679438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc004820 con 0x563646e466d0 2024-07-24T21:56:31.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68c3fff640 1 -- 172.21.15.60:0/581679438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0043c0 con 0x563646e466d0 2024-07-24T21:56:31.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.086+0000 7f68c3fff640 1 -- 172.21.15.60:0/581679438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc02cc40 con 0x563646e466d0 2024-07-24T21:56:31.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 -- 172.21.15.60:0/581679438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 msgr2=0x563646e45a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00bd80 tx=0x7f68cc010ce0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 -- 172.21.15.60:0/581679438 shutdown_connections 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e57390 0x563646e42d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e46ff0 0x563646e51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 --2- 172.21.15.60:0/581679438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e466d0 0x563646e45a90 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 -- 172.21.15.60:0/581679438 >> 172.21.15.60:0/581679438 conn(0x563646e44500 msgr2=0x563646e45380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 -- 172.21.15.60:0/581679438 shutdown_connections 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.087+0000 7f68e32a4740 1 -- 172.21.15.60:0/581679438 wait complete. 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 -- start start 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 Processor -- start 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e466d0 0x563646e6e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e5f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aa5e0 con 0x563646e46ff0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aba50 con 0x563646e57390 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aacb0 con 0x563646e466d0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68e32a4740 10 client.0 _ll_register_callbacks cb 0x7f68d0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41784/0 (socket says 172.21.15.60:41784) 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 -- 172.21.15.60:0/3672102406 learned_addr learned my addr 172.21.15.60:0/3672102406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68dad9d640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e5f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68da59c640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e466d0 0x563646e6e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 -- 172.21.15.60:0/3672102406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e466d0 msgr2=0x563646e6e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e466d0 0x563646e6e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 -- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 msgr2=0x563646e5f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:31.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e5f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.088+0000 7f68d9d9b640 1 -- 172.21.15.60:0/3672102406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563646e6f5b0 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68d9d9b640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f68d00098d0 tx=0x7f68d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba694d2dc6224d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0021620 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68cc0043c0 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f68d001c070 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68d00217c0 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 10 client.17517 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563646e47f10 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0043c0 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f68d002ba60 con 0x563646e46ff0 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 10 client.17517 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 10 client.17517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 20 client.17517 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646ed1900 0x563646ed3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 4 client.17517 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.089+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563646dbb3a0 con 0x563646ed1900 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.090+0000 7f68da59c640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646ed1900 0x563646ed3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.090+0000 7f68c1ffb640 10 client.17517 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:56:31.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.090+0000 7f68da59c640 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646ed1900 0x563646ed3d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f499d161ea9f795d server_cookie=5087382f78d7e56e in_seq=0 out_seq=0 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.113+0000 7f68c1ffb640 1 -- 172.21.15.60:0/3672102406 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563646dbb3a0 con 0x563646ed1900 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68c1ffb640 10 client.17517 handle_command_reply: tid=0 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 client.17517 shutdown 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 msgr2=0x563646e6ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f68d00098d0 tx=0x7f68d0005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 shutdown_connections 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x563646ed1900 0x563646ed3d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563646e57390 0x563646e5f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563646e46ff0 0x563646e6ee70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68c1ffb640 0 client.17517 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 --2- 172.21.15.60:0/3672102406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563646e466d0 0x563646e6e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 >> 172.21.15.60:0/3672102406 conn(0x563646e44500 msgr2=0x563646e45ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 shutdown_connections 2024-07-24T21:56:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 1 -- 172.21.15.60:0/3672102406 wait complete. 2024-07-24T21:56:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e32a4740 20 client.17517 trim_cache size 0 max 16384 2024-07-24T21:56:31.117 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 msgr2=0x7f68c40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 0x7f68c40b6280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0ff8d0 tx=0x7f68d00095d0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 msgr2=0x7f68dc19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.114+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098d0 tx=0x7f68d40317b0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 shutdown_connections 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68c40b3dc0 0x7f68c40b6280 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68dc10caa0 0x7f68dc19e480 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68dc103ef0 0x7f68dc1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 --2- 172.21.15.60:0/1414615726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68dc1034f0 0x7f68dc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 >> 172.21.15.60:0/1414615726 conn(0x7f68dc0fd0c0 msgr2=0x7f68dc10ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 shutdown_connections 2024-07-24T21:56:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:31.115+0000 7f68e0ef3640 1 -- 172.21.15.60:0/1414615726 wait complete. 2024-07-24T21:56:31.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:31 smithi060 ceph-mon[33000]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-07-24T21:56:31.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:31 smithi123 ceph-mon[37143]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-07-24T21:56:31.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:56:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:31 smithi086 ceph-mon[37128]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-07-24T21:56:32.043 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 -- 172.21.15.60:0/1317761285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20107910 msgr2=0x7f6e2010e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1317761285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20107910 0x7f6e2010e110 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009ca0 tx=0x7f6e1802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 -- 172.21.15.60:0/1317761285 shutdown_connections 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1317761285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20107910 0x7f6e2010e110 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1317761285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e201073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1317761285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20101590 0x7f6e20101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 -- 172.21.15.60:0/1317761285 >> 172.21.15.60:0/1317761285 conn(0x7f6e200fd010 msgr2=0x7f6e200ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 -- 172.21.15.60:0/1317761285 shutdown_connections 2024-07-24T21:56:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 -- 172.21.15.60:0/1317761285 wait complete. 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.320+0000 7f6e257a6640 1 Processor -- start 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 -- start start 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e2019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20107910 0x7f6e201a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20111540 con 0x7f6e20101590 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201113c0 con 0x7f6e20107910 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e257a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201116c0 con 0x7f6e20106f90 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e2019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41816/0 (socket says 172.21.15.60:41816) 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1effd640 1 -- 172.21.15.60:0/1658958586 learned_addr learned my addr 172.21.15.60:0/1658958586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20107910 0x7f6e201a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/1658958586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20107910 msgr2=0x7f6e201a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20107910 0x7f6e201a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 msgr2=0x7f6e2019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/1658958586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e201a0e60 con 0x7f6e20106f90 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.321+0000 7f6e1effd640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e2019b300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400e9f0 tx=0x7f6e1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e07fff640 1 -- 172.21.15.60:0/1658958586 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400ccc0 con 0x7f6e20106f90 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e07fff640 1 -- 172.21.15.60:0/1658958586 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e140108e0 con 0x7f6e20106f90 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a1150 con 0x7f6e20106f90 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e07fff640 1 -- 172.21.15.60:0/1658958586 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14018880 con 0x7f6e20106f90 2024-07-24T21:56:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.322+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a7810 con 0x7f6e20106f90 2024-07-24T21:56:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.324+0000 7f6e285cf740 1 Processor -- start 2024-07-24T21:56:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.324+0000 7f6e07fff640 1 -- 172.21.15.60:0/1658958586 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6e14010430 con 0x7f6e20106f90 2024-07-24T21:56:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.324+0000 7f6e285cf740 1 -- start start 2024-07-24T21:56:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf9a540 0x56340bf9c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56340bf03280 con 0x56340bf8e5b0 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56340bef2780 con 0x56340bf8dc90 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e285cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56340bef67c0 con 0x56340bf9a540 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf9a540 0x56340bf9c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40686/0 (socket says 172.21.15.60:40686) 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41828/0 (socket says 172.21.15.60:41828) 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/4154750638 learned_addr learned my addr 172.21.15.60:0/4154750638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/4154750638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf9a540 msgr2=0x56340bf9c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf9a540 0x56340bf9c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/4154750638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 msgr2=0x56340bf9a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/4154750638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340bc56790 con 0x56340bf8dc90 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1effd640 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e07fff640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 0x7f6e080b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1effd640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 0x7f6e080b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009410 tx=0x7f6e18005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92419665d74580 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.325+0000 7f6e07fff640 1 -- 172.21.15.60:0/1658958586 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f6e140e43e0 con 0x7f6e20106f90 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e057fa640 1 -- 172.21.15.60:0/4154750638 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031cf0 con 0x56340bf8dc90 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e057fa640 1 -- 172.21.15.60:0/4154750638 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e18036030 con 0x56340bf8dc90 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e057fa640 1 -- 172.21.15.60:0/4154750638 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18004240 con 0x56340bf8dc90 2024-07-24T21:56:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 -- 172.21.15.60:0/4154750638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 msgr2=0x56340bf8e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e1effd640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 0x7f6e080b6200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009620 tx=0x7f6e10005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009410 tx=0x7f6e18005ef0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 -- 172.21.15.60:0/4154750638 shutdown_connections 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf9a540 0x56340bf9c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf9a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 --2- 172.21.15.60:0/4154750638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8dc90 0x56340bf8e070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 -- 172.21.15.60:0/4154750638 >> 172.21.15.60:0/4154750638 conn(0x56340bf8bac0 msgr2=0x56340bf8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 -- 172.21.15.60:0/4154750638 shutdown_connections 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.326+0000 7f6e285cf740 1 -- 172.21.15.60:0/4154750638 wait complete. 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 -- start start 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 Processor -- start 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56340be2cab0 con 0x56340bf8e5b0 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56340be7a9e0 con 0x56340bf9a540 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56340bf9d040 con 0x56340bf8dc90 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e285cf740 10 client.0 _ll_register_callbacks cb 0x56340bc02a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41834/0 (socket says 172.21.15.60:41834) 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/808151159 learned_addr learned my addr 172.21.15.60:0/808151159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.327+0000 7f6e1effd640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 msgr2=0x56340bf99c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf99c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/808151159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 msgr2=0x56340bf97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf97680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/808151159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340bccf620 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6e140a82b0 tx=0x7f6e1400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8899f7ee4662cc75 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140aa7b0 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f6e140a72d0 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56340bea1ce0 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6e14010720 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 10 client.17529 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56340bfa8850 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56340bea1ce0 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f6e140af020 con 0x56340bf8e5b0 2024-07-24T21:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 10 client.17529 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 20 client.17529 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340bfa9d10 0x56340bfac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 4 client.17529 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.328+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56340bf051d0 con 0x56340bfa9d10 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.329+0000 7f6e1effd640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340bfa9d10 0x56340bfac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.329+0000 7f6e02ff5640 10 client.17529 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.329+0000 7f6e1effd640 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340bfa9d10 0x56340bfac100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9a3035a13f58a4 server_cookie=afd04d6a69d6a1d1 in_seq=0 out_seq=0 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e02ff5640 1 -- 172.21.15.60:0/808151159 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56340bf051d0 con 0x56340bfa9d10 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e02ff5640 10 client.17529 handle_command_reply: tid=0 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 client.17529 shutdown 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 msgr2=0x56340bf97140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6e140a82b0 tx=0x7f6e1400ce60 comp rx=0 tx=0).stop 2024-07-24T21:56:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 shutdown_connections 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340bfa9d10 0x56340bfac100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf97140 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 --2- 172.21.15.60:0/808151159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 >> 172.21.15.60:0/808151159 conn(0x56340bf8bac0 msgr2=0x56340bf91660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e02ff5640 0 client.17529 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 shutdown_connections 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 1 -- 172.21.15.60:0/808151159 wait complete. 2024-07-24T21:56:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.330+0000 7f6e285cf740 20 client.17529 trim_cache size 0 max 16384 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 Processor -- start 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 -- start start 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bfa6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201b2c60 con 0x56340bf8e5b0 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201b0470 con 0x56340bf9a540 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e285cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201ae2e0 con 0x56340bf8dc90 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41850/0 (socket says 172.21.15.60:41850) 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/3412882529 learned_addr learned my addr 172.21.15.60:0/3412882529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1effd640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bfa6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/3412882529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 msgr2=0x56340bfa6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bfa6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/3412882529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 msgr2=0x56340bf8d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf8d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 -- 172.21.15.60:0/3412882529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340bf8e3e0 con 0x56340bf8e5b0 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1effd640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bfa6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.346+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000c00 tx=0x7f6e14010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cff5a992d2e3346 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e04ff9640 1 -- 172.21.15.60:0/3412882529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140c9070 con 0x56340bf8e5b0 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e04ff9640 1 -- 172.21.15.60:0/3412882529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e140aab50 con 0x56340bf8e5b0 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e04ff9640 1 -- 172.21.15.60:0/3412882529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140c5ad0 con 0x56340bf8e5b0 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 -- 172.21.15.60:0/3412882529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 msgr2=0x56340bf8cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000c00 tx=0x7f6e14010ca0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 -- 172.21.15.60:0/3412882529 shutdown_connections 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf9a540 0x56340bf8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf8e5b0 0x56340bf8cb10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3412882529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bfa6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 -- 172.21.15.60:0/3412882529 >> 172.21.15.60:0/3412882529 conn(0x56340bf8bac0 msgr2=0x56340bf97110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 -- 172.21.15.60:0/3412882529 shutdown_connections 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.347+0000 7f6e285cf740 1 -- 172.21.15.60:0/3412882529 wait complete. 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 -- start start 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 Processor -- start 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf9f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8e5b0 0x56340bf9f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201ae930 con 0x56340bf9a540 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201afda0 con 0x56340bf8e5b0 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201af000 con 0x56340bf8dc90 2024-07-24T21:56:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e285cf740 10 client.0 _ll_register_callbacks cb 0x7f6e140e7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41866/0 (socket says 172.21.15.60:41866) 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/3590654659 learned_addr learned my addr 172.21.15.60:0/3590654659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1e7fc640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8e5b0 0x56340bf9f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1effd640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf9f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 msgr2=0x56340bf9f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf9f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/3590654659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8e5b0 msgr2=0x56340bf9f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8e5b0 0x56340bf9f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.348+0000 7f6e1f7fe640 1 -- 172.21.15.60:0/3590654659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340c012f20 con 0x56340bf9a540 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e1f7fe640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6e18005cc0 tx=0x7f6e18003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d53d2365ee41b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18052020 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e140aab50 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8609+0+0 (secure 0 0 0) 0x7f6e18031450 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6e1805d050 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 10 client.17535 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56340c014080 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e140aab50 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f6e18043070 con 0x56340bf9a540 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 10 client.17535 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 20 client.17535 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340c0145c0 0x56340c0169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 4 client.17535 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:56:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.349+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56340bf00c70 con 0x56340c0145c0 2024-07-24T21:56:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.350+0000 7f6e1effd640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340c0145c0 0x56340c0169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.350+0000 7f6e02ff5640 10 client.17535 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:32.350+0000 7f6e1effd640 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340c0145c0 0x56340c0169e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80211871f9157447 server_cookie=ca6de3f0bf6b5526 in_seq=0 out_seq=0 2024-07-24T21:56:33.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:33 smithi060 ceph-mon[33000]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.4 MiB/s wr, 481 op/s 2024-07-24T21:56:33.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:33 smithi060 ceph-mon[33000]: osdmap e96: 12 total, 12 up, 12 in 2024-07-24T21:56:33.618 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.3 2024-07-24T21:56:33.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:56:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:33 smithi123 ceph-mon[37143]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.4 MiB/s wr, 481 op/s 2024-07-24T21:56:33.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:33 smithi123 ceph-mon[37143]: osdmap e96: 12 total, 12 up, 12 in 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.760+0000 7fc2474f5640 1 -- 172.21.15.60:0/1027232737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc240109f20 msgr2=0x7fc24010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.760+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1027232737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc240109f20 0x7fc24010a360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc238009ca0 tx=0x7fc23802a980 comp rx=0 tx=0).stop 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 -- 172.21.15.60:0/1027232737 shutdown_connections 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1027232737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc24011ba90 0x7fc24011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1027232737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc240109f20 0x7fc24010a360 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1027232737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109b40 0x7fc240110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 -- 172.21.15.60:0/1027232737 >> 172.21.15.60:0/1027232737 conn(0x7fc24006d980 msgr2=0x7fc24006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 -- 172.21.15.60:0/1027232737 shutdown_connections 2024-07-24T21:56:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.761+0000 7fc2474f5640 1 -- 172.21.15.60:0/1027232737 wait complete. 2024-07-24T21:56:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 Processor -- start 2024-07-24T21:56:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 -- start start 2024-07-24T21:56:33.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240109b40 0x7fc240118ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc24011ba90 0x7fc240112350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc240121670 con 0x7fc240109f20 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401214f0 con 0x7fc240109b40 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.762+0000 7fc2474f5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401217f0 con 0x7fc24011ba90 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.766+0000 7fc244a90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.766+0000 7fc244a90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41882/0 (socket says 172.21.15.60:41882) 2024-07-24T21:56:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.766+0000 7fc244a90640 1 -- 172.21.15.60:0/1977689794 learned_addr learned my addr 172.21.15.60:0/1977689794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.769+0000 7fc245291640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240109b40 0x7fc240118ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.769+0000 7fc245a92640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc24011ba90 0x7fc240112350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.771+0000 7fc244a90640 1 -- 172.21.15.60:0/1977689794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc24011ba90 msgr2=0x7fc240112350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.771+0000 7fc244a90640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc24011ba90 0x7fc240112350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.771+0000 7fc244a90640 1 -- 172.21.15.60:0/1977689794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240109b40 msgr2=0x7fc240118ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.771+0000 7fc244a90640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240109b40 0x7fc240118ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.771+0000 7fc244a90640 1 -- 172.21.15.60:0/1977689794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc240112c10 con 0x7fc240109f20 2024-07-24T21:56:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.774+0000 7fc244a90640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc238000f80 tx=0x7fc23802f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.776+0000 7fc2367fc640 1 -- 172.21.15.60:0/1977689794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238034070 con 0x7fc240109f20 2024-07-24T21:56:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.776+0000 7fc2367fc640 1 -- 172.21.15.60:0/1977689794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc238004040 con 0x7fc240109f20 2024-07-24T21:56:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.776+0000 7fc2367fc640 1 -- 172.21.15.60:0/1977689794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23802e7f0 con 0x7fc240109f20 2024-07-24T21:56:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.776+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc240112ea0 con 0x7fc240109f20 2024-07-24T21:56:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.776+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc240117d90 con 0x7fc240109f20 2024-07-24T21:56:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.780+0000 7fc2498a6740 1 Processor -- start 2024-07-24T21:56:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.782+0000 7fc2367fc640 1 -- 172.21.15.60:0/1977689794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc238037020 con 0x7fc240109f20 2024-07-24T21:56:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.783+0000 7fc2367fc640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 0x7fc22c0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.784+0000 7fc245a92640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 0x7fc22c0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 -- start start 2024-07-24T21:56:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 0x556f3abb8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 0x556f3abc4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abc7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556f3ab2f9b0 con 0x556f3abb8ff0 2024-07-24T21:56:33.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f3ab1eeb0 con 0x556f3abb86d0 2024-07-24T21:56:33.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.786+0000 7fc2498a6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f3ab22ef0 con 0x556f3abc4f80 2024-07-24T21:56:33.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.788+0000 7fc245291640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 0x556f3abb8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.789+0000 7fc245291640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 0x556f3abb8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40730/0 (socket says 172.21.15.60:40730) 2024-07-24T21:56:33.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.789+0000 7fc245291640 1 -- 172.21.15.60:0/655522263 learned_addr learned my addr 172.21.15.60:0/655522263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc2367fc640 1 -- 172.21.15.60:0/1977689794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fc238104e70 con 0x7fc240109f20 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 0x556f3abc4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc245a92640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 0x7fc22c0b6400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00a750 tx=0x7fc23c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc245a92640 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abc7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 -- 172.21.15.60:0/655522263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 msgr2=0x556f3abc7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abc7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 -- 172.21.15.60:0/655522263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 msgr2=0x556f3abb8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 0x556f3abb8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 -- 172.21.15.60:0/655522263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3aaa8320 con 0x556f3abb8ff0 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.790+0000 7fc244a90640 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 0x556f3abc4a40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc238102530 tx=0x7fc23802e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ddec0279ee43dc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.791+0000 7fc22b7fe640 1 -- 172.21.15.60:0/655522263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380d43f0 con 0x556f3abb8ff0 2024-07-24T21:56:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.791+0000 7fc22b7fe640 1 -- 172.21.15.60:0/655522263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2380e6320 con 0x556f3abb8ff0 2024-07-24T21:56:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:33 smithi086 ceph-mon[37128]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.4 MiB/s wr, 481 op/s 2024-07-24T21:56:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:33 smithi086 ceph-mon[37128]: osdmap e96: 12 total, 12 up, 12 in 2024-07-24T21:56:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.792+0000 7fc2498a6740 1 -- 172.21.15.60:0/655522263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 msgr2=0x556f3abc4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.792+0000 7fc2498a6740 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 0x556f3abc4a40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc238102530 tx=0x7fc23802e330 comp rx=0 tx=0).stop 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.794+0000 7fc2498a6740 1 -- 172.21.15.60:0/655522263 shutdown_connections 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.794+0000 7fc2498a6740 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abc7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.794+0000 7fc2498a6740 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb8ff0 0x556f3abc4a40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.794+0000 7fc2498a6740 1 --2- 172.21.15.60:0/655522263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb86d0 0x556f3abb8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.794+0000 7fc2498a6740 1 -- 172.21.15.60:0/655522263 >> 172.21.15.60:0/655522263 conn(0x556f3abb6500 msgr2=0x556f3abb76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.795+0000 7fc2498a6740 1 -- 172.21.15.60:0/655522263 shutdown_connections 2024-07-24T21:56:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.795+0000 7fc2498a6740 1 -- 172.21.15.60:0/655522263 wait complete. 2024-07-24T21:56:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.795+0000 7fc2498a6740 1 -- start start 2024-07-24T21:56:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 Processor -- start 2024-07-24T21:56:33.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 0x556f3abc1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 0x556f3abc29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556f3a982700 con 0x556f3abc4f80 2024-07-24T21:56:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f3aac15f0 con 0x556f3abb8ff0 2024-07-24T21:56:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f3ab2f9b0 con 0x556f3abb86d0 2024-07-24T21:56:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.796+0000 7fc2498a6740 10 client.0 _ll_register_callbacks cb 0x556f3aae1960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.797+0000 7fc244a90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.800+0000 7fc244a90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40732/0 (socket says 172.21.15.60:40732) 2024-07-24T21:56:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.800+0000 7fc244a90640 1 -- 172.21.15.60:0/491605537 learned_addr learned my addr 172.21.15.60:0/491605537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.800+0000 7fc245291640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 0x556f3abc1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.800+0000 7fc245a92640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 0x556f3abc29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.803+0000 7fc245a92640 1 -- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 msgr2=0x556f3abc1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.803+0000 7fc245a92640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 0x556f3abc1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.803+0000 7fc245a92640 1 -- 172.21.15.60:0/491605537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 msgr2=0x556f3abc24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.803+0000 7fc245a92640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.803+0000 7fc245a92640 1 -- 172.21.15.60:0/491605537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3aab4680 con 0x556f3abc4f80 2024-07-24T21:56:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.804+0000 7fc244a90640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.804+0000 7fc245291640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 0x556f3abc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.805+0000 7fc245a92640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 0x556f3abc29f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006fd0 tx=0x7fc23c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23ada1d02a39d93 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.806+0000 7fc228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.806+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c023020 con 0x556f3abc4f80 2024-07-24T21:56:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.806+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f3ab31900 con 0x556f3abc4f80 2024-07-24T21:56:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.806+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fc23c01f470 con 0x556f3abc4f80 2024-07-24T21:56:33.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.807+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc23c02e050 con 0x556f3abc4f80 2024-07-24T21:56:33.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.807+0000 7fc2498a6740 10 client.17544 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:33.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.807+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556f3a87f650 con 0x556f3abc4f80 2024-07-24T21:56:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556f3ab31900 con 0x556f3abc4f80 2024-07-24T21:56:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc23c01e6c0 con 0x556f3abc4f80 2024-07-24T21:56:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 10 client.17544 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 20 client.17544 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:33.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3abd4ca0 0x556f3abd70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 4 client.17544 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:33.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.810+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3ab2d3a0 con 0x556f3abd4ca0 2024-07-24T21:56:33.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.814+0000 7fc245291640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3abd4ca0 0x556f3abd70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.831+0000 7fc245291640 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3abd4ca0 0x556f3abd70c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbf4fcd3f250322 server_cookie=4ee089a33357899f in_seq=0 out_seq=0 2024-07-24T21:56:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.831+0000 7fc228ff9640 10 client.17544 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:33.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.956+0000 7fc228ff9640 1 -- 172.21.15.60:0/491605537 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f3ab2d3a0 con 0x556f3abd4ca0 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc228ff9640 10 client.17544 handle_command_reply: tid=0 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 client.17544 shutdown 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 msgr2=0x556f3abc29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 0x556f3abc29f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006fd0 tx=0x7fc23c004770 comp rx=0 tx=0).stop 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 shutdown_connections 2024-07-24T21:56:33.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3abd4ca0 0x556f3abd70c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abc4f80 0x556f3abc29f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abc24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 --2- 172.21.15.60:0/491605537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb86d0 0x556f3abc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 >> 172.21.15.60:0/491605537 conn(0x556f3abb6500 msgr2=0x556f3abbc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc228ff9640 0 client.17544 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 shutdown_connections 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.957+0000 7fc2498a6740 1 -- 172.21.15.60:0/491605537 wait complete. 2024-07-24T21:56:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.958+0000 7fc2498a6740 20 client.17544 trim_cache size 0 max 16384 2024-07-24T21:56:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.991+0000 7fc2498a6740 1 Processor -- start 2024-07-24T21:56:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.991+0000 7fc2498a6740 1 -- start start 2024-07-24T21:56:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.992+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abdc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.992+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb8ff0 0x556f3abdcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.992+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.992+0000 7fc2498a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c82d0 con 0x556f3abb86d0 2024-07-24T21:56:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.992+0000 7fc2498a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c5ae0 con 0x556f3abc4f80 2024-07-24T21:56:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.993+0000 7fc2498a6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c3950 con 0x556f3abb8ff0 2024-07-24T21:56:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.993+0000 7fc244a90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb8ff0 0x556f3abdcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.993+0000 7fc245a92640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.993+0000 7fc245a92640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:40742/0 (socket says 172.21.15.60:40742) 2024-07-24T21:56:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.993+0000 7fc245a92640 1 -- 172.21.15.60:0/4055544514 learned_addr learned my addr 172.21.15.60:0/4055544514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.996+0000 7fc245291640 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abdc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:34.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.999+0000 7fc245a92640 1 -- 172.21.15.60:0/4055544514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb8ff0 msgr2=0x556f3abdcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:34.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.999+0000 7fc245a92640 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb8ff0 0x556f3abdcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.999+0000 7fc245a92640 1 -- 172.21.15.60:0/4055544514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 msgr2=0x556f3abdc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:34.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.999+0000 7fc245a92640 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abdc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:33.999+0000 7fc245a92640 1 -- 172.21.15.60:0/4055544514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3abb0b00 con 0x556f3abc4f80 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.006+0000 7fc245a92640 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01eb10 tx=0x7fc23c00ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7013c91885a165ca server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.006+0000 7fc22affd640 1 -- 172.21.15.60:0/4055544514 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c037070 con 0x556f3abc4f80 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.006+0000 7fc22affd640 1 -- 172.21.15.60:0/4055544514 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c01e890 con 0x556f3abc4f80 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.006+0000 7fc22affd640 1 -- 172.21.15.60:0/4055544514 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c023040 con 0x556f3abc4f80 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 -- 172.21.15.60:0/4055544514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 msgr2=0x556f3abc1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01eb10 tx=0x7fc23c00ca40 comp rx=0 tx=0).stop 2024-07-24T21:56:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 -- 172.21.15.60:0/4055544514 shutdown_connections 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abc4f80 0x556f3abc1ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abb8ff0 0x556f3abdcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 --2- 172.21.15.60:0/4055544514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abdc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.007+0000 7fc2498a6740 1 -- 172.21.15.60:0/4055544514 >> 172.21.15.60:0/4055544514 conn(0x556f3abb6500 msgr2=0x556f3abc07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.008+0000 7fc2498a6740 1 -- 172.21.15.60:0/4055544514 shutdown_connections 2024-07-24T21:56:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.008+0000 7fc2498a6740 1 -- 172.21.15.60:0/4055544514 wait complete. 2024-07-24T21:56:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.009+0000 7fc2498a6740 1 -- start start 2024-07-24T21:56:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.009+0000 7fc2498a6740 1 Processor -- start 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abcca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abbe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c3fa0 con 0x556f3abb86d0 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c5410 con 0x556f3abb8ff0 2024-07-24T21:56:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.010+0000 7fc2498a6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401c4670 con 0x556f3abc4f80 2024-07-24T21:56:34.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.011+0000 7fc2498a6740 10 client.0 _ll_register_callbacks cb 0x7fc23c00c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.012+0000 7fc245291640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abcca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.012+0000 7fc245291640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abcca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41926/0 (socket says 172.21.15.60:41926) 2024-07-24T21:56:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.012+0000 7fc245291640 1 -- 172.21.15.60:0/1810735900 learned_addr learned my addr 172.21.15.60:0/1810735900 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.012+0000 7fc244a90640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.012+0000 7fc245a92640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abbe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:34.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.016+0000 7fc244a90640 1 -- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 msgr2=0x556f3abbe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:34.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.016+0000 7fc244a90640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abbe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.016+0000 7fc244a90640 1 -- 172.21.15.60:0/1810735900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 msgr2=0x556f3abcca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:34.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.016+0000 7fc244a90640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abcca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.016+0000 7fc244a90640 1 -- 172.21.15.60:0/1810735900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3abbeae0 con 0x556f3abb8ff0 2024-07-24T21:56:34.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.021+0000 7fc244a90640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abccfd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc23802af70 tx=0x7fc2380cb430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee89f4a80c167761 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:34.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.021+0000 7fc228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:34.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.028+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2380c7e20 con 0x556f3abb8ff0 2024-07-24T21:56:34.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.029+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c01e890 con 0x556f3abb8ff0 2024-07-24T21:56:34.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.029+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fc2380d4820 con 0x556f3abb8ff0 2024-07-24T21:56:34.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.030+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2380e6320 con 0x556f3abb8ff0 2024-07-24T21:56:34.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.030+0000 7fc2498a6740 10 client.26969 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:34.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.030+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556f3ac3f100 con 0x556f3abb8ff0 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c01e890 con 0x556f3abb8ff0 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc23804b030 con 0x556f3abb8ff0 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 10 client.26969 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 10 client.26969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 20 client.26969 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3ac3f6b0 0x556f3ac41ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 4 client.26969 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:56:34.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.037+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f3ab2aae0 con 0x556f3ac3f6b0 2024-07-24T21:56:34.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.045+0000 7fc245291640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3ac3f6b0 0x556f3ac41ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:34.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.046+0000 7fc245291640 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3ac3f6b0 0x556f3ac41ad0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409484adaad46b7e server_cookie=395c1f9fb8d0c263 in_seq=0 out_seq=0 2024-07-24T21:56:34.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.046+0000 7fc228ff9640 10 client.26969 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc228ff9640 1 -- 172.21.15.60:0/1810735900 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f3ab2aae0 con 0x556f3ac3f6b0 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc228ff9640 10 client.26969 handle_command_reply: tid=0 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 client.26969 shutdown 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 msgr2=0x556f3abccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abccfd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc23802af70 tx=0x7fc2380cb430 comp rx=0 tx=0).stop 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 shutdown_connections 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x556f3ac3f6b0 0x556f3ac41ad0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f3abc4f80 0x556f3abbe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f3abb8ff0 0x556f3abccfd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 --2- 172.21.15.60:0/1810735900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556f3abb86d0 0x556f3abcca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 >> 172.21.15.60:0/1810735900 conn(0x556f3abb6500 msgr2=0x556f3abc3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:34.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.057+0000 7fc228ff9640 0 client.26969 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:34.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.058+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 shutdown_connections 2024-07-24T21:56:34.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.058+0000 7fc2498a6740 1 -- 172.21.15.60:0/1810735900 wait complete. 2024-07-24T21:56:34.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.058+0000 7fc2498a6740 20 client.26969 trim_cache size 0 max 16384 2024-07-24T21:56:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:56:34.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.060+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 msgr2=0x7fc22c0b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:34.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.060+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 0x7fc22c0b6400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00a750 tx=0x7fc23c008040 comp rx=0 tx=0).stop 2024-07-24T21:56:34.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.060+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 msgr2=0x7fc240119430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:34.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.060+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc238000f80 tx=0x7fc23802f9f0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 shutdown_connections 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc22c0b3f40 0x7fc22c0b6400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc24011ba90 0x7fc240112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc240109f20 0x7fc240119430 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 --2- 172.21.15.60:0/1977689794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc240109b40 0x7fc240118ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.061+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 >> 172.21.15.60:0/1977689794 conn(0x7fc24006d980 msgr2=0x7fc24011c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.062+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 shutdown_connections 2024-07-24T21:56:34.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:34.062+0000 7fc2474f5640 1 -- 172.21.15.60:0/1977689794 wait complete. 2024-07-24T21:56:35.045 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:35.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:35 smithi060 ceph-mon[33000]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.7 MiB/s wr, 509 op/s 2024-07-24T21:56:35.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:35 smithi123 ceph-mon[37143]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.7 MiB/s wr, 509 op/s 2024-07-24T21:56:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:35 smithi086 ceph-mon[37128]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.7 MiB/s wr, 509 op/s 2024-07-24T21:56:36.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:56:36.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:56:36.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/113097280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f811ba90 msgr2=0x7fd7f811de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/113097280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f811ba90 0x7fd7f811de80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009ca0 tx=0x7fd7f002a980 comp rx=0 tx=0).stop 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/113097280 shutdown_connections 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/113097280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f811ba90 0x7fd7f811de80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/113097280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/113097280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f8109b40 0x7fd7f8110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.546+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/113097280 >> 172.21.15.60:0/113097280 conn(0x7fd7f806d980 msgr2=0x7fd7f806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:37.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.547+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/113097280 shutdown_connections 2024-07-24T21:56:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.550+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/113097280 wait complete. 2024-07-24T21:56:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.550+0000 7fd7fdfd3640 1 Processor -- start 2024-07-24T21:56:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.552+0000 7fd7fdfd3640 1 -- start start 2024-07-24T21:56:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f8109b40 0x7fd7f8118ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 0x7fd7f8112280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8121560 con 0x7fd7f8109f20 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f81213e0 con 0x7fd7f811ba90 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7fdfd3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f81216e0 con 0x7fd7f8109b40 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37296/0 (socket says 172.21.15.60:37296) 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3651214644 learned_addr learned my addr 172.21.15.60:0/3651214644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7f77fe640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f8109b40 0x7fd7f8118ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.554+0000 7fd7f7fff640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 0x7fd7f8112280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.557+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3651214644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f8109b40 msgr2=0x7fd7f8118ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.557+0000 7fd7f6ffd640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f8109b40 0x7fd7f8118ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.557+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3651214644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 msgr2=0x7fd7f8112280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.557+0000 7fd7f6ffd640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 0x7fd7f8112280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.557+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3651214644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8112b40 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.558+0000 7fd7f6ffd640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b4a0 tx=0x7fd7e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.559+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8014070 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.559+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f8112e30 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.560+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f8117e40 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.560+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e8010ac0 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.560+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8017c00 con 0x7fd7f8109f20 2024-07-24T21:56:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.561+0000 7fd7f7fff640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 0x7fd7f8112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.570+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd7e8017070 con 0x7fd7f8109f20 2024-07-24T21:56:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.571+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e4005290 con 0x7fd7f8109f20 2024-07-24T21:56:37.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.575+0000 7fd7f4ff9640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 0x7fd7e00b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.576+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fd7e80e3570 con 0x7fd7f8109f20 2024-07-24T21:56:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.577+0000 7fd7f77fe640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 0x7fd7e00b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:37.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.584+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fd7e80aa900 con 0x7fd7f8109f20 2024-07-24T21:56:37.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:37.586+0000 7fd7f77fe640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 0x7fd7e00b63d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec005fd0 tx=0x7fd7ec005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:37.722 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:37 smithi060 ceph-mon[33000]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 946 op/s 2024-07-24T21:56:37.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:37 smithi123 ceph-mon[37143]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 946 op/s 2024-07-24T21:56:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:37 smithi086 ceph-mon[37128]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 946 op/s 2024-07-24T21:56:38.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.051+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7e4005850 con 0x7fd7f8109f20 2024-07-24T21:56:38.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.058+0000 7fd7f4ff9640 1 -- 172.21.15.60:0/3651214644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11922 (secure 0 0 0) 0x7fd7e80aa200 con 0x7fd7f8109f20 2024-07-24T21:56:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:56:38.061 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"btime":"2024-07-24T21:56:26:404711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26284,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:25.641909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26290":{"gid":26290,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2868776697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2868776697},{"type":"v1","addr":"172.21.15.123:6835","nonce":2868776697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:56:38.061 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-07-24T21:56:38.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.066+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 msgr2=0x7fd7e00b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.066+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 0x7fd7e00b63d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec005fd0 tx=0x7fd7ec005850 comp rx=0 tx=0).stop 2024-07-24T21:56:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.066+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 msgr2=0x7fd7f8119410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.066+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b4a0 tx=0x7fd7e800b960 comp rx=0 tx=0).stop 2024-07-24T21:56:38.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 shutdown_connections 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd7e00b3f10 0x7fd7e00b63d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f811ba90 0x7fd7f8112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f8109f20 0x7fd7f8119410 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 --2- 172.21.15.60:0/3651214644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7f8109b40 0x7fd7f8118ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 >> 172.21.15.60:0/3651214644 conn(0x7fd7f806d980 msgr2=0x7fd7f811c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.069+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 shutdown_connections 2024-07-24T21:56:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:38.070+0000 7fd7fdfd3640 1 -- 172.21.15.60:0/3651214644 wait complete. 2024-07-24T21:56:38.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:38 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3651214644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:38.696 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:38 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3651214644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:38 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3651214644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:39.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0' 2024-07-24T21:56:39.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0 2024-07-24T21:56:39.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:39 smithi060 ceph-mon[33000]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 1.49k op/s 2024-07-24T21:56:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-mon[37143]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 1.49k op/s 2024-07-24T21:56:39.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:39 smithi086 ceph-mon[37128]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 1.49k op/s 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e02ff5640 1 -- 172.21.15.60:0/3590654659 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56340bf00c70 con 0x56340c0145c0 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e02ff5640 10 client.17535 handle_command_reply: tid=0 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 client.17535 shutdown 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 msgr2=0x56340c0129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6e18005cc0 tx=0x7f6e18003bf0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 shutdown_connections 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56340c0145c0 0x56340c0169e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56340bf9a540 0x56340c0129e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56340bf8e5b0 0x56340bf9f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 --2- 172.21.15.60:0/3590654659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56340bf8dc90 0x56340bf9f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e02ff5640 0 client.17535 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 >> 172.21.15.60:0/3590654659 conn(0x56340bf8bac0 msgr2=0x56340bf8cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 shutdown_connections 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.864+0000 7f6e285cf740 1 -- 172.21.15.60:0/3590654659 wait complete. 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e285cf740 20 client.17535 trim_cache size 0 max 16384 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:56:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:56:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 msgr2=0x7f6e080b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 0x7f6e080b6200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009620 tx=0x7f6e10005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 msgr2=0x7f6e2019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e2019b300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400e9f0 tx=0x7f6e1400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 shutdown_connections 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e080b3d40 0x7f6e080b6200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e20107910 0x7f6e201a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20106f90 0x7f6e2019b300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 --2- 172.21.15.60:0/1658958586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20101590 0x7f6e2019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 >> 172.21.15.60:0/1658958586 conn(0x7f6e200fd010 msgr2=0x7f6e2006b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 shutdown_connections 2024-07-24T21:56:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:39.865+0000 7f6e257a6640 1 -- 172.21.15.60:0/1658958586 wait complete. 2024-07-24T21:56:40.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 -- 172.21.15.60:0/1214603553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc10cae0 msgr2=0x7f93dc10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 --2- 172.21.15.60:0/1214603553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc10cae0 0x7f93dc10ef50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009ca0 tx=0x7f93d402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 -- 172.21.15.60:0/1214603553 shutdown_connections 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 --2- 172.21.15.60:0/1214603553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc10cae0 0x7f93dc10ef50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 --2- 172.21.15.60:0/1214603553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc103f30 0x7f93dc10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 --2- 172.21.15.60:0/1214603553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103530 0x7f93dc103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 -- 172.21.15.60:0/1214603553 >> 172.21.15.60:0/1214603553 conn(0x7f93dc0fd100 msgr2=0x7f93dc0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 -- 172.21.15.60:0/1214603553 shutdown_connections 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.003+0000 7f93e3594640 1 -- 172.21.15.60:0/1214603553 wait complete. 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 Processor -- start 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 -- start start 2024-07-24T21:56:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103f30 0x7f93dc1018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10cae0 0x7f93dc101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc112730 con 0x7f93dc103530 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1125b0 con 0x7f93dc10cae0 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e3594640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1128b0 con 0x7f93dc103f30 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e1330640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e1330640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37316/0 (socket says 172.21.15.60:37316) 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e1330640 1 -- 172.21.15.60:0/2584172768 learned_addr learned my addr 172.21.15.60:0/2584172768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e1b31640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10cae0 0x7f93dc101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.004+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103f30 0x7f93dc1018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 -- 172.21.15.60:0/2584172768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103f30 msgr2=0x7f93dc1018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103f30 0x7f93dc1018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 -- 172.21.15.60:0/2584172768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10cae0 msgr2=0x7f93dc101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10cae0 0x7f93dc101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 -- 172.21.15.60:0/2584172768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc1a6fc0 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e1330640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93d27fc640 1 -- 172.21.15.60:0/2584172768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017070 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc1a72b0 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93d27fc640 1 -- 172.21.15.60:0/2584172768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d8009c30 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93d27fc640 1 -- 172.21.15.60:0/2584172768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800f620 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.005+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc1a77f0 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 Processor -- start 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 -- start start 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a3c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93d27fc640 1 -- 172.21.15.60:0/2584172768 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f93d8010460 con 0x7f93dc103530 2024-07-24T21:56:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ebf09b39f0 con 0x55ebf0a3c840 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebf09a2ef0 con 0x55ebf0a487d0 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e5945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ebf09a6f30 con 0x55ebf0a3bf20 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37328/0 (socket says 172.21.15.60:37328) 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 -- 172.21.15.60:0/2848172355 learned_addr learned my addr 172.21.15.60:0/2848172355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1b31640 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a3c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 -- 172.21.15.60:0/2848172355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 msgr2=0x55ebf0a3c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a3c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 -- 172.21.15.60:0/2848172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 msgr2=0x55ebf0a4abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.008+0000 7f93e1330640 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93e1330640 1 -- 172.21.15.60:0/2848172355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf0703930 con 0x55ebf0a3c840 2024-07-24T21:56:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93e1330640 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f93d80e1250 tx=0x7f93d80ec700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13bd9ce9bc5084d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93c77fe640 1 -- 172.21.15.60:0/2848172355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80ee610 con 0x55ebf0a3c840 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93c77fe640 1 -- 172.21.15.60:0/2848172355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d80eec60 con 0x55ebf0a3c840 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93d27fc640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 0x7f93c80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93c77fe640 1 -- 172.21.15.60:0/2848172355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80f6d90 con 0x55ebf0a3c840 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93e1b31640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 0x7f93c80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.009+0000 7f93d27fc640 1 -- 172.21.15.60:0/2584172768 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f93d80e3bc0 con 0x7f93dc103530 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- 172.21.15.60:0/2848172355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 msgr2=0x55ebf0a48290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e1b31640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 0x7f93c80b6200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d40099a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f93d80e1250 tx=0x7f93d80ec700 comp rx=0 tx=0).stop 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- 172.21.15.60:0/2848172355 shutdown_connections 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a48290 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 --2- 172.21.15.60:0/2848172355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a3c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- 172.21.15.60:0/2848172355 >> 172.21.15.60:0/2848172355 conn(0x55ebf0a39d50 msgr2=0x55ebf0a3aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- 172.21.15.60:0/2848172355 shutdown_connections 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- 172.21.15.60:0/2848172355 wait complete. 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 -- start start 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.010+0000 7f93e5945740 1 Processor -- start 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a45790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ebf0806890 con 0x55ebf0a487d0 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ebf0945390 con 0x55ebf0a3c840 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ebf0a4b450 con 0x55ebf0a3bf20 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e5945740 10 client.0 _ll_register_callbacks cb 0x55ebf094f500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e1b31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34988/0 (socket says 172.21.15.60:34988) 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 -- 172.21.15.60:0/3602456007 learned_addr learned my addr 172.21.15.60:0/3602456007 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e1b31640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37334/0 (socket says 172.21.15.60:37334) 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e1330640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a45790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 -- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 msgr2=0x55ebf0a45790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a45790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 -- 172.21.15.60:0/3602456007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 msgr2=0x55ebf0a46210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 -- 172.21.15.60:0/3602456007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf093fd60 con 0x55ebf0a3c840 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e1b31640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.011+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012740 tx=0x7f93cc012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273086d3a905df22 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc004030 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d80eec60 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f93cc0041d0 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f93cc0043d0 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 10 client.26987 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ebf0a57420 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d80eec60 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f93cc023070 con 0x55ebf0a3c840 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 10 client.26987 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 10 client.26987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 20 client.26987 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0a58940 0x55ebf0a5ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 4 client.26987 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.012+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf09b5940 con 0x55ebf0a58940 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.013+0000 7f93e1330640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0a58940 0x55ebf0a5ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.013+0000 7f93e1330640 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0a58940 0x55ebf0a5ad60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5905944f1682adbf server_cookie=1d98c28d39b7056a in_seq=0 out_seq=0 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.013+0000 7f93c4ff9640 10 client.26987 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93c4ff9640 1 -- 172.21.15.60:0/3602456007 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebf09b5940 con 0x55ebf0a58940 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93c4ff9640 10 client.26987 handle_command_reply: tid=0 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 client.26987 shutdown 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 msgr2=0x55ebf0a45cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012740 tx=0x7f93cc012c00 comp rx=0 tx=0).stop 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 shutdown_connections 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0a58940 0x55ebf0a5ad60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a487d0 0x55ebf0a46210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93c4ff9640 0 client.26987 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3c840 0x55ebf0a45cd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 --2- 172.21.15.60:0/3602456007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a45790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 >> 172.21.15.60:0/3602456007 conn(0x55ebf0a39d50 msgr2=0x55ebf0a3f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 shutdown_connections 2024-07-24T21:56:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 1 -- 172.21.15.60:0/3602456007 wait complete. 2024-07-24T21:56:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.014+0000 7f93e5945740 20 client.26987 trim_cache size 0 max 16384 2024-07-24T21:56:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 Processor -- start 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 -- start start 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a60630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b2ce0 con 0x55ebf0a3c840 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b04f0 con 0x55ebf0a487d0 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e5945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ae360 con 0x55ebf0a3bf20 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37340/0 (socket says 172.21.15.60:37340) 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 -- 172.21.15.60:0/1804415813 learned_addr learned my addr 172.21.15.60:0/1804415813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e1b31640 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e1330640 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a60630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 -- 172.21.15.60:0/1804415813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 msgr2=0x55ebf0a60630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a60630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 -- 172.21.15.60:0/1804415813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 msgr2=0x55ebf0a429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.033+0000 7f93e0b2f640 1 -- 172.21.15.60:0/1804415813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf0a39b80 con 0x55ebf0a3c840 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00a010 tx=0x7f93cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04a01bdb44c317a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93c6ffd640 1 -- 172.21.15.60:0/1804415813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc029020 con 0x55ebf0a3c840 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93c6ffd640 1 -- 172.21.15.60:0/1804415813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc012e50 con 0x55ebf0a3c840 2024-07-24T21:56:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93c6ffd640 1 -- 172.21.15.60:0/1804415813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00fd40 con 0x55ebf0a3c840 2024-07-24T21:56:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93e5945740 1 -- 172.21.15.60:0/1804415813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 msgr2=0x55ebf0a60b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.034+0000 7f93e5945740 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00a010 tx=0x7f93cc00f790 comp rx=0 tx=0).stop 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- 172.21.15.60:0/1804415813 shutdown_connections 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a487d0 0x55ebf0a429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a60b70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- 172.21.15.60:0/1804415813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a60630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- 172.21.15.60:0/1804415813 >> 172.21.15.60:0/1804415813 conn(0x55ebf0a39d50 msgr2=0x55ebf0a3f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- 172.21.15.60:0/1804415813 shutdown_connections 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- 172.21.15.60:0/1804415813 wait complete. 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- start start 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 Processor -- start 2024-07-24T21:56:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a50710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ae9b0 con 0x55ebf0a3c840 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1afe20 con 0x55ebf0a3bf20 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1af080 con 0x55ebf0a487d0 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.035+0000 7f93e5945740 10 client.0 _ll_register_callbacks cb 0x7f93cc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e1330640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a50710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37356/0 (socket says 172.21.15.60:37356) 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 -- 172.21.15.60:0/2576174222 learned_addr learned my addr 172.21.15.60:0/2576174222 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e1b31640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 -- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a487d0 msgr2=0x55ebf0a4d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 -- 172.21.15.60:0/2576174222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3bf20 msgr2=0x55ebf0a50710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a50710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 -- 172.21.15.60:0/2576174222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf0a4dc20 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e0b2f640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012d60 tx=0x7f93cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceffa1892591316b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc033060 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f93cc00fc90 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93cc012e50 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.036+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f93cc029030 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 10 client.17571 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ebf0ac3db0 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc012e50 con 0x55ebf0a3c840 2024-07-24T21:56:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f93cc023030 con 0x55ebf0a3c840 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 10 client.17571 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 20 client.17571 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0ac4420 0x55ebf0ac6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 4 client.17571 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ebf09b13e0 con 0x55ebf0ac4420 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.037+0000 7f93e1330640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0ac4420 0x55ebf0ac6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e1330640 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0ac4420 0x55ebf0ac6840 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360cd623b14b0d26 server_cookie=3f98f45ea74404ff in_seq=0 out_seq=0 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93c4ff9640 10 client.17571 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93c4ff9640 1 -- 172.21.15.60:0/2576174222 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ebf09b13e0 con 0x55ebf0ac4420 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93c4ff9640 10 client.17571 handle_command_reply: tid=0 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 client.17571 shutdown 2024-07-24T21:56:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 msgr2=0x55ebf0a50c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012d60 tx=0x7f93cc0057b0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 shutdown_connections 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ebf0ac4420 0x55ebf0ac6840 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ebf0a487d0 0x55ebf0a4d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ebf0a3c840 0x55ebf0a50c50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 --2- 172.21.15.60:0/2576174222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ebf0a3bf20 0x55ebf0a50710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93c4ff9640 0 client.17571 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.038+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 >> 172.21.15.60:0/2576174222 conn(0x55ebf0a39d50 msgr2=0x55ebf0a53590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 shutdown_connections 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e5945740 1 -- 172.21.15.60:0/2576174222 wait complete. 2024-07-24T21:56:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e5945740 20 client.17571 trim_cache size 0 max 16384 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "bf91fbd5-0ad1-4978-92a9-268bd452d98f", 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:56:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 msgr2=0x7f93c80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 0x7f93c80b6200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d40099a0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 msgr2=0x7f93dc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.039+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d950 tx=0x7f93d800de10 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 shutdown_connections 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f93c80b3d40 0x7f93c80b6200 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10cae0 0x7f93dc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93dc103f30 0x7f93dc1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 --2- 172.21.15.60:0/2584172768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93dc103530 0x7f93dc1013b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 >> 172.21.15.60:0/2584172768 conn(0x7f93dc0fd100 msgr2=0x7f93dc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 shutdown_connections 2024-07-24T21:56:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:40.040+0000 7f93e3594640 1 -- 172.21.15.60:0/2584172768 wait complete. 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f3d224b9640 time 2024-07-24T21:56:39.846731+0000 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3d2ff48698] 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.244 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: *** Caught signal (Aborted) ** 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: in thread 7f3d224b9640 thread_name:md_log_replay 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:39.846+0000 7f3d224b9640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f3d224b9640 time 2024-07-24T21:56:39.846731+0000 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3d2ff48698] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.245 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3d2f8926f0] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3d2f8df94c] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: raise() 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: abort() 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3d2ff486f2] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 7: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 8: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 9: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:39.847+0000 7f3d224b9640 -1 *** Caught signal (Aborted) ** 2024-07-24T21:56:40.246 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: in thread 7f3d224b9640 thread_name:md_log_replay 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3d2f8926f0] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3d2f8df94c] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: raise() 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: abort() 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3d2ff486f2] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 7: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 8: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 9: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-24T21:56:40.247 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: -1> 2024-07-24T21:56:39.846+0000 7f3d224b9640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f3d224b9640 time 2024-07-24T21:56:39.846731+0000 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3d2ff48698] 2024-07-24T21:56:40.248 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 0> 2024-07-24T21:56:39.847+0000 7f3d224b9640 -1 *** Caught signal (Aborted) ** 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: in thread 7f3d224b9640 thread_name:md_log_replay 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3d2f8926f0] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3d2f8df94c] 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: raise() 2024-07-24T21:56:40.249 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: abort() 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3d2ff486f2] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 7: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 8: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 9: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: -1> 2024-07-24T21:56:39.846+0000 7f3d224b9640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f3d224b9640 time 2024-07-24T21:56:39.846731+0000 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5197-g86385b1d/rpm/el9/BUILD/ceph-19.0.0-5197-g86385b1d/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.250 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3d2ff48698] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 0> 2024-07-24T21:56:39.847+0000 7f3d224b9640 -1 *** Caught signal (Aborted) ** 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: in thread 7f3d224b9640 thread_name:md_log_replay 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: ceph version 19.0.0-5197-g86385b1d (86385b1daf5235400bbcd34a02515d168bde88e2) squid (dev) 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3d2f8926f0] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3d2f8df94c] 2024-07-24T21:56:40.251 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 3: raise() 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 4: abort() 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3d2ff486f2] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x184856) [0x7f3d2ff48856] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 7: (MDLog::_replay_thread()+0xcc) [0x563ba0ecce8c] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 8: /usr/bin/ceph-mds(+0x43e401) [0x563ba0eba401] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 9: /lib64/libc.so.6(+0x89c02) [0x7f3d2f8ddc02] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3d2f962c40] 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[72466]: 2024-07-24T21:56:40.252 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:39 smithi123 podman[73950]: 2024-07-24 21:56:39.919406057 +0000 UTC m=+0.019539791 container died c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_CLEAN=True) 2024-07-24T21:56:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:41.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:40 smithi123 podman[73950]: 2024-07-24 21:56:40.778848141 +0000 UTC m=+0.878981907 container remove c923562590d3d870d7c19b7cfc8052fd70d2d860be0e2afd3146281c6b00c444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1) 2024-07-24T21:56:41.079 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:40 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-07-24T21:56:41.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:56:41.442 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:41 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-07-24T21:56:41.442 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:41 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Consumed 21.904s CPU time. 2024-07-24T21:56:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:41 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:56:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:41 smithi123 ceph-mon[37143]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 2.04k op/s 2024-07-24T21:56:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:41 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:56:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:41 smithi086 ceph-mon[37128]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 2.04k op/s 2024-07-24T21:56:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:41 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:56:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:41 smithi060 ceph-mon[33000]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 2.04k op/s 2024-07-24T21:56:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:42.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:42.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:42 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:42.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:42 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:42 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:42.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:42 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:42 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:42 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 -- 172.21.15.60:0/3053901528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4107150 msgr2=0x7f34b4107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 --2- 172.21.15.60:0/3053901528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4107150 0x7f34b4107590 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009930 tx=0x7f34a802a420 comp rx=0 tx=0).stop 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 -- 172.21.15.60:0/3053901528 shutdown_connections 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 --2- 172.21.15.60:0/3053901528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107ad0 0x7f34b410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 --2- 172.21.15.60:0/3053901528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4107150 0x7f34b4107590 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 --2- 172.21.15.60:0/3053901528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4102360 0x7f34b4102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 -- 172.21.15.60:0/3053901528 >> 172.21.15.60:0/3053901528 conn(0x7f34b4076890 msgr2=0x7f34b4076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 -- 172.21.15.60:0/3053901528 shutdown_connections 2024-07-24T21:56:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.634+0000 7f34b89d6640 1 -- 172.21.15.60:0/3053901528 wait complete. 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.635+0000 7f34b89d6640 1 Processor -- start 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.635+0000 7f34b89d6640 1 -- start start 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.635+0000 7f34b89d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 0x7f34b4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.635+0000 7f34b89d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 0x7f34b4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.635+0000 7f34b89d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4107ad0 0x7f34b419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b89d6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41115e0 con 0x7f34b4102360 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b89d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111460 con 0x7f34b4107150 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b89d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111760 con 0x7f34b4107ad0 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 0x7f34b4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 0x7f34b4197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37378/0 (socket says 172.21.15.60:37378) 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b259c640 1 -- 172.21.15.60:0/2134433390 learned_addr learned my addr 172.21.15.60:0/2134433390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 0x7f34b4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4107ad0 0x7f34b419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 -- 172.21.15.60:0/2134433390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4107ad0 msgr2=0x7f34b419d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4107ad0 0x7f34b419d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 -- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 msgr2=0x7f34b4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 0x7f34b4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 -- 172.21.15.60:0/2134433390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b419dce0 con 0x7f34b4107150 2024-07-24T21:56:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.636+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 0x7f34b4198170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34a8031440 tx=0x7f34a80316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.637+0000 7f34a37fe640 1 -- 172.21.15.60:0/2134433390 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a804c070 con 0x7f34b4107150 2024-07-24T21:56:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.637+0000 7f34a37fe640 1 -- 172.21.15.60:0/2134433390 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a80035f0 con 0x7f34b4107150 2024-07-24T21:56:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.637+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b419df70 con 0x7f34b4107150 2024-07-24T21:56:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.637+0000 7f34a37fe640 1 -- 172.21.15.60:0/2134433390 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8035050 con 0x7f34b4107150 2024-07-24T21:56:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.637+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b4100ff0 con 0x7f34b4107150 2024-07-24T21:56:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34a37fe640 1 -- 172.21.15.60:0/2134433390 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f34a804f070 con 0x7f34b4107150 2024-07-24T21:56:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 Processor -- start 2024-07-24T21:56:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 -- start start 2024-07-24T21:56:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 0x559114231230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 0x55911423f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5591141a7670 con 0x559114231770 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.640+0000 7f34bad87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559114196b70 con 0x559114232cd0 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34bad87740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55911419abb0 con 0x55911423d5a0 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 0x559114231230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37386/0 (socket says 172.21.15.60:37386) 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b259c640 1 -- 172.21.15.60:0/2405695850 learned_addr learned my addr 172.21.15.60:0/2405695850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 0x55911423f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.641+0000 7f34b259c640 1 -- 172.21.15.60:0/2405695850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 msgr2=0x55911423f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b259c640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 0x55911423f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b259c640 1 -- 172.21.15.60:0/2405695850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 msgr2=0x559114231230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b259c640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 0x559114231230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b259c640 1 -- 172.21.15.60:0/2405695850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55911412ed30 con 0x559114231770 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 0x559114231230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 0x55911423f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34a37fe640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 0x7f349c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 0x7f349c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34b259c640 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f34a4010b30 tx=0x7f34a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f990e84f4c9df7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.642+0000 7f34a0ff9640 1 -- 172.21.15.60:0/2405695850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a401d070 con 0x559114231770 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34a0ff9640 1 -- 172.21.15.60:0/2405695850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a40188b0 con 0x559114231770 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34a37fe640 1 -- 172.21.15.60:0/2134433390 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f34a810c080 con 0x7f34b4107150 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34a0ff9640 1 -- 172.21.15.60:0/2405695850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a400e9b0 con 0x559114231770 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 -- 172.21.15.60:0/2405695850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 msgr2=0x559114231b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 0x7f349c0b6280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009fd0 tx=0x7f34ac009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f34a4010b30 tx=0x7f34a4005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 -- 172.21.15.60:0/2405695850 shutdown_connections 2024-07-24T21:56:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55911423d5a0 0x55911423f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114231b90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.643+0000 7f34bad87740 1 --2- 172.21.15.60:0/2405695850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114232cd0 0x559114231230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.644+0000 7f34bad87740 1 -- 172.21.15.60:0/2405695850 >> 172.21.15.60:0/2405695850 conn(0x5591141fb720 msgr2=0x55911422f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.644+0000 7f34bad87740 1 -- 172.21.15.60:0/2405695850 shutdown_connections 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.644+0000 7f34bad87740 1 -- 172.21.15.60:0/2405695850 wait complete. 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.644+0000 7f34bad87740 1 -- start start 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.644+0000 7f34bad87740 1 Processor -- start 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x55911423a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x55911423ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559113ffa7a0 con 0x55911423d5a0 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559114138ff0 con 0x559114231770 2024-07-24T21:56:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5591142404b0 con 0x559114232cd0 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34bad87740 10 client.0 _ll_register_callbacks cb 0x559113ea4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35026/0 (socket says 172.21.15.60:35026) 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 -- 172.21.15.60:0/615149070 learned_addr learned my addr 172.21.15.60:0/615149070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x55911423ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x55911423a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 -- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 msgr2=0x55911423a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x55911423a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 -- 172.21.15.60:0/615149070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 msgr2=0x55911423ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x55911423ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.645+0000 7f34b259c640 1 -- 172.21.15.60:0/615149070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55911413d0a0 con 0x559114231770 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x55911423a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f34b259c640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e4b0 tx=0x7f34a40186b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242573219cbe148f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f3499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a401d030 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f34a4005250 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a40188b0 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.646+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f34a4032050 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 10 client.27005 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559113f06b90 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a40188b0 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f34a4023070 con 0x559114231770 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 10 client.27005 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 10 client.27005 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:56:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 20 client.27005 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55911424d4f0 0x55911424f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 4 client.27005 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591141a95c0 con 0x55911424d4f0 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.647+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55911424d4f0 0x55911424f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.648+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55911424d4f0 0x55911424f8e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389032022346798b server_cookie=4d859f34c91938ca in_seq=0 out_seq=0 2024-07-24T21:56:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.648+0000 7f3499ffb640 10 client.27005 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f3499ffb640 1 -- 172.21.15.60:0/615149070 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591141a95c0 con 0x55911424d4f0 2024-07-24T21:56:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f3499ffb640 10 client.27005 handle_command_reply: tid=0 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 client.27005 shutdown 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 msgr2=0x55911423cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e4b0 tx=0x7f34a40186b0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 shutdown_connections 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55911424d4f0 0x55911424f8e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x55911423ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f3499ffb640 0 client.27005 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x55911423a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 --2- 172.21.15.60:0/615149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x55911423cf90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.649+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 >> 172.21.15.60:0/615149070 conn(0x5591141fb720 msgr2=0x559114234c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.650+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 shutdown_connections 2024-07-24T21:56:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.650+0000 7f34bad87740 1 -- 172.21.15.60:0/615149070 wait complete. 2024-07-24T21:56:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.650+0000 7f34bad87740 20 client.27005 trim_cache size 0 max 16384 2024-07-24T21:56:43.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 Processor -- start 2024-07-24T21:56:43.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 -- start start 2024-07-24T21:56:43.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114229fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55911423d5a0 0x55911423c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41ae970 con 0x559114231770 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41ac180 con 0x55911423d5a0 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34bad87740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41a9ff0 con 0x559114232cd0 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34b1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114229fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37400/0 (socket says 172.21.15.60:37400) 2024-07-24T21:56:43.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34b259c640 1 -- 172.21.15.60:0/1458725144 learned_addr learned my addr 172.21.15.60:0/1458725144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.674+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55911423d5a0 0x55911423c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 -- 172.21.15.60:0/1458725144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 msgr2=0x559114229fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114229fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 -- 172.21.15.60:0/1458725144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55911423d5a0 msgr2=0x55911423c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55911423d5a0 0x55911423c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 -- 172.21.15.60:0/1458725144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55911422f110 con 0x559114231770 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114229fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f34b259c640 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f34a400f010 tx=0x7f34a4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20674a6239263f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f349bfff640 1 -- 172.21.15.60:0/1458725144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a401d040 con 0x559114231770 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f349bfff640 1 -- 172.21.15.60:0/1458725144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a4029070 con 0x559114231770 2024-07-24T21:56:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.675+0000 7f349bfff640 1 -- 172.21.15.60:0/1458725144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4003730 con 0x559114231770 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 -- 172.21.15.60:0/1458725144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 msgr2=0x559114230e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f34a400f010 tx=0x7f34a4005f10 comp rx=0 tx=0).stop 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 -- 172.21.15.60:0/1458725144 shutdown_connections 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55911423d5a0 0x55911423c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114229fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 --2- 172.21.15.60:0/1458725144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559114231770 0x559114230e70 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 -- 172.21.15.60:0/1458725144 >> 172.21.15.60:0/1458725144 conn(0x5591141fb720 msgr2=0x55911423ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 -- 172.21.15.60:0/1458725144 shutdown_connections 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.676+0000 7f34bad87740 1 -- 172.21.15.60:0/1458725144 wait complete. 2024-07-24T21:56:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 -- start start 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 Processor -- start 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114241f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41aa640 con 0x55911423d5a0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41abab0 con 0x559114231770 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41aad10 con 0x559114232cd0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34bad87740 10 client.0 _ll_register_callbacks cb 0x7f34a401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114241f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35044/0 (socket says 172.21.15.60:35044) 2024-07-24T21:56:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37404/0 (socket says 172.21.15.60:37404) 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.677+0000 7f34b259c640 1 -- 172.21.15.60:0/3539245538 learned_addr learned my addr 172.21.15.60:0/3539245538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 -- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 msgr2=0x559114241f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114241f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 -- 172.21.15.60:0/3539245538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 msgr2=0x559114241a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 -- 172.21.15.60:0/3539245538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55911422a770 con 0x55911423d5a0 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b259c640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b1d9b640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114241f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34b2d9d640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00f820 tx=0x7f34ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e61c3054f109e18 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f3499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac02e060 con 0x55911423d5a0 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a4029070 con 0x55911423d5a0 2024-07-24T21:56:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.678+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f34ac003b80 con 0x55911423d5a0 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f34ac003d80 con 0x55911423d5a0 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 10 client.17589 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55911423aee0 con 0x55911423d5a0 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a4029070 con 0x55911423d5a0 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f34ac01e650 con 0x55911423d5a0 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 10 client.17589 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 10 client.17589 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 20 client.17589 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591142b8c70 0x5591142bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 4 client.17589 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}] 2024-07-24T21:56:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.679+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}) -- 0x5591141a5060 con 0x5591142b8c70 2024-07-24T21:56:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.680+0000 7f34b259c640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591142b8c70 0x5591142bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.680+0000 7f3499ffb640 10 client.17589 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:43.680+0000 7f34b259c640 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591142b8c70 0x5591142bb090 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905c113a57704a08 server_cookie=37c58852b3dc9d80 in_seq=0 out_seq=0 2024-07-24T21:56:43.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:43 smithi060 ceph-mon[33000]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.5 MiB/s wr, 2.54k op/s 2024-07-24T21:56:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:43 smithi123 ceph-mon[37143]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.5 MiB/s wr, 2.54k op/s 2024-07-24T21:56:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:43 smithi086 ceph-mon[37128]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.5 MiB/s wr, 2.54k op/s 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.619+0000 7f4d59b72640 1 -- 172.21.15.60:0/4108338393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d54101650 msgr2=0x7f4d54101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.619+0000 7f4d59b72640 1 --2- 172.21.15.60:0/4108338393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d54101650 0x7f4d54101a30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009900 tx=0x7f4d4402a410 comp rx=0 tx=0).stop 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 -- 172.21.15.60:0/4108338393 shutdown_connections 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 --2- 172.21.15.60:0/4108338393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d541079e0 0x7f4d5410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 --2- 172.21.15.60:0/4108338393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d541074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 --2- 172.21.15.60:0/4108338393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d54101650 0x7f4d54101a30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 -- 172.21.15.60:0/4108338393 >> 172.21.15.60:0/4108338393 conn(0x7f4d540fd0c0 msgr2=0x7f4d540ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 -- 172.21.15.60:0/4108338393 shutdown_connections 2024-07-24T21:56:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.620+0000 7f4d59b72640 1 -- 172.21.15.60:0/4108338393 wait complete. 2024-07-24T21:56:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 Processor -- start 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 -- start start 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d54101650 0x7f4d54198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d541079e0 0x7f4d5419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54111630 con 0x7f4d541079e0 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541114b0 con 0x7f4d54107060 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.621+0000 7f4d59b72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541117b0 con 0x7f4d54101650 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.622+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.622+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35056/0 (socket says 172.21.15.60:35056) 2024-07-24T21:56:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.622+0000 7f4d52ffd640 1 -- 172.21.15.60:0/1287961762 learned_addr learned my addr 172.21.15.60:0/1287961762 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 -- 172.21.15.60:0/1287961762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d54101650 msgr2=0x7f4d54198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d54101650 0x7f4d54198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 -- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d541079e0 msgr2=0x7f4d5419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d541079e0 0x7f4d5419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 -- 172.21.15.60:0/1287961762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419edb0 con 0x7f4d54107060 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9f0 tx=0x7f4d4800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d50ff9640 1 -- 172.21.15.60:0/1287961762 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800cd80 con 0x7f4d54107060 2024-07-24T21:56:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d50ff9640 1 -- 172.21.15.60:0/1287961762 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d480054d0 con 0x7f4d54107060 2024-07-24T21:56:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d50ff9640 1 -- 172.21.15.60:0/1287961762 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48010430 con 0x7f4d54107060 2024-07-24T21:56:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419f0a0 con 0x7f4d54107060 2024-07-24T21:56:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.623+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541081b0 con 0x7f4d54107060 2024-07-24T21:56:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.626+0000 7f4d50ff9640 1 -- 172.21.15.60:0/1287961762 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4d480046c0 con 0x7f4d54107060 2024-07-24T21:56:44.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:44 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.627+0000 7f4d50ff9640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 0x7f4d3c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.628+0000 7f4d50ff9640 1 -- 172.21.15.60:0/1287961762 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f4d480e4570 con 0x7f4d54107060 2024-07-24T21:56:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.628+0000 7f4d5bf23740 1 Processor -- start 2024-07-24T21:56:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.628+0000 7f4d537fe640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 0x7f4d3c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.628+0000 7f4d5bf23740 1 -- start start 2024-07-24T21:56:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.628+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5609c2bc6990 con 0x5609c2c4ffd0 2024-07-24T21:56:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609c2bb5e90 con 0x5609c2c4f6b0 2024-07-24T21:56:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5609c2bb9ed0 con 0x5609c2c5bf60 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35060/0 (socket says 172.21.15.60:35060) 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 -- 172.21.15.60:0/2692713444 learned_addr learned my addr 172.21.15.60:0/2692713444 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 0x7f4d3c0b6300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4d440096c0 tx=0x7f4d44031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d53fff640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 -- 172.21.15.60:0/2692713444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 msgr2=0x5609c2c5e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c5e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.629+0000 7f4d537fe640 1 -- 172.21.15.60:0/2692713444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 msgr2=0x5609c2c5ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.630+0000 7f4d537fe640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c5ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.630+0000 7f4d537fe640 1 -- 172.21.15.60:0/2692713444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c2b3f300 con 0x5609c2c4f6b0 2024-07-24T21:56:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.630+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c5ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:56:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.631+0000 7f4d537fe640 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d44033000 tx=0x7f4d4403a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbef78ce7e20b14 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.632+0000 7f4d39ffb640 1 -- 172.21.15.60:0/2692713444 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44035040 con 0x5609c2c4f6b0 2024-07-24T21:56:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.632+0000 7f4d39ffb640 1 -- 172.21.15.60:0/2692713444 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d44041070 con 0x5609c2c4f6b0 2024-07-24T21:56:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.633+0000 7f4d39ffb640 1 -- 172.21.15.60:0/2692713444 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440038c0 con 0x5609c2c4f6b0 2024-07-24T21:56:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- 172.21.15.60:0/2692713444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 msgr2=0x5609c2c4fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d44033000 tx=0x7f4d4403a000 comp rx=0 tx=0).stop 2024-07-24T21:56:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- 172.21.15.60:0/2692713444 shutdown_connections 2024-07-24T21:56:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c5ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/2692713444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4fa90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- 172.21.15.60:0/2692713444 >> 172.21.15.60:0/2692713444 conn(0x5609c2c4d4e0 msgr2=0x5609c2c4e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- 172.21.15.60:0/2692713444 shutdown_connections 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- 172.21.15.60:0/2692713444 wait complete. 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 -- start start 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.634+0000 7f4d5bf23740 1 Processor -- start 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c58eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c59640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5609c2a19700 con 0x5609c2c4ffd0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609c2b585d0 con 0x5609c2c4f6b0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5609c2c6fbe0 con 0x5609c2c5bf60 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d5bf23740 10 client.0 _ll_register_callbacks cb 0x5609c2b78940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35062/0 (socket says 172.21.15.60:35062) 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 -- 172.21.15.60:0/3515330566 learned_addr learned my addr 172.21.15.60:0/3515330566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c58eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d53fff640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 -- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 msgr2=0x5609c2c59640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c59640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 -- 172.21.15.60:0/3515330566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 msgr2=0x5609c2c58eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c58eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 -- 172.21.15.60:0/3515330566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c2b4b660 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d53fff640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c59640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d537fe640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d44004400 tx=0x7f4d44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16290fff6b0eb6bd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.635+0000 7f4d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44035040 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d44041070 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f4d4404ca90 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4d440318e0 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 10 client.27023 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5609c2916650 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d44041070 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4d44055740 con 0x5609c2c4f6b0 2024-07-24T21:56:44.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 10 client.27023 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 10 client.27023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 20 client.27023 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.636+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2c7c860 0x5609c2c7ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.637+0000 7f4d5bf23740 4 client.27023 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.637+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c2bc88e0 con 0x5609c2c7c860 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.637+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2c7c860 0x5609c2c7ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.637+0000 7f4d32ffd640 10 client.27023 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:44.637+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2c7c860 0x5609c2c7ec50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f36607459367f7 server_cookie=ebc562552863af8c in_seq=0 out_seq=0 2024-07-24T21:56:44.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:44 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:44 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:45 smithi123 ceph-mon[37143]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 2.43k op/s 2024-07-24T21:56:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:45 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:45 smithi086 ceph-mon[37128]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 2.43k op/s 2024-07-24T21:56:45.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:45 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:45 smithi060 ceph-mon[33000]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 2.43k op/s 2024-07-24T21:56:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:45 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:46 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:46.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:46 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:46 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:56:46.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:46 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:46 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:46.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:46 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:46 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:56:46.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:46 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:46 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:46 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:46 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:56:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:46 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:47 smithi123 ceph-mon[37143]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 3.50k op/s 2024-07-24T21:56:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:47 smithi086 ceph-mon[37128]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 3.50k op/s 2024-07-24T21:56:47.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:47 smithi060 ceph-mon[33000]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 3.50k op/s 2024-07-24T21:56:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d32ffd640 1 -- 172.21.15.60:0/3515330566 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609c2bc88e0 con 0x5609c2c7c860 2024-07-24T21:56:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d32ffd640 10 client.27023 handle_command_reply: tid=0 2024-07-24T21:56:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 client.27023 shutdown 2024-07-24T21:56:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 msgr2=0x5609c2c5ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d44004400 tx=0x7f4d44000bc0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 shutdown_connections 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2c7c860 0x5609c2c7ec50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c59640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4ffd0 0x5609c2c58eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/3515330566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c5ba20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d32ffd640 0 client.27023 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 >> 172.21.15.60:0/3515330566 conn(0x5609c2c4d4e0 msgr2=0x5609c2c53080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 shutdown_connections 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 1 -- 172.21.15.60:0/3515330566 wait complete. 2024-07-24T21:56:48.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.101+0000 7f4d5bf23740 20 client.27023 trim_cache size 0 max 16384 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 Processor -- start 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 -- start start 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 0x5609c2c4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.122+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541a6270 con 0x5609c2c4f6b0 2024-07-24T21:56:48.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54073090 con 0x5609c2c4ffd0 2024-07-24T21:56:48.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54070f00 con 0x5609c2c5bf60 2024-07-24T21:56:48.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 0x5609c2c4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34700/0 (socket says 172.21.15.60:34700) 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 0x5609c2c4e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36944/0 (socket says 172.21.15.60:36944) 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d537fe640 1 -- 172.21.15.60:0/89708517 learned_addr learned my addr 172.21.15.60:0/89708517 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47938/0 (socket says 172.21.15.60:47938) 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 -- 172.21.15.60:0/89708517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 msgr2=0x5609c2c4e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 0x5609c2c4e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 -- 172.21.15.60:0/89708517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 msgr2=0x5609c2c4e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:48.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d53fff640 1 -- 172.21.15.60:0/89708517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c2c5bd90 con 0x5609c2c5bf60 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.123+0000 7f4d537fe640 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.124+0000 7f4d53fff640 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50064501d7215143 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.124+0000 7f4d397fa640 1 -- 172.21.15.60:0/89708517 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0047a0 con 0x5609c2c5bf60 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.124+0000 7f4d397fa640 1 -- 172.21.15.60:0/89708517 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c016070 con 0x5609c2c5bf60 2024-07-24T21:56:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.124+0000 7f4d397fa640 1 -- 172.21.15.60:0/89708517 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0117d0 con 0x5609c2c5bf60 2024-07-24T21:56:48.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- 172.21.15.60:0/89708517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 msgr2=0x5609c2c81c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c009cb0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- 172.21.15.60:0/89708517 shutdown_connections 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c5bf60 0x5609c2c81c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4ffd0 0x5609c2c4e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/89708517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c4f6b0 0x5609c2c4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- 172.21.15.60:0/89708517 >> 172.21.15.60:0/89708517 conn(0x5609c2c4d4e0 msgr2=0x5609c2c541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- 172.21.15.60:0/89708517 shutdown_connections 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- 172.21.15.60:0/89708517 wait complete. 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.125+0000 7f4d5bf23740 1 -- start start 2024-07-24T21:56:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 Processor -- start 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 0x5609c2c7f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c5bf60 0x5609c2c7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54071550 con 0x5609c2c5bf60 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540729c0 con 0x5609c2c4f6b0 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54071c20 con 0x5609c2c4ffd0 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d5bf23740 10 client.0 _ll_register_callbacks cb 0x7f4d4c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36950/0 (socket says 172.21.15.60:36950) 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 0x5609c2c7f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d537fe640 1 -- 172.21.15.60:0/672053096 learned_addr learned my addr 172.21.15.60:0/672053096 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d53fff640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c5bf60 0x5609c2c7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d52ffd640 1 -- 172.21.15.60:0/672053096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 msgr2=0x5609c2c822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d52ffd640 1 -- 172.21.15.60:0/672053096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c5bf60 msgr2=0x5609c2c7fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.126+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c5bf60 0x5609c2c7fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d52ffd640 1 -- 172.21.15.60:0/672053096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c2c80520 con 0x5609c2c4ffd0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d537fe640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d52ffd640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 0x5609c2c7f7e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e1c30 tx=0x7f4d480a70e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc652852b8e2ef6e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480b41a0 con 0x5609c2c4ffd0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f4d480e1eb0 con 0x5609c2c4ffd0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4c016070 con 0x5609c2c4ffd0 2024-07-24T21:56:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4d480e20b0 con 0x5609c2c4ffd0 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d5bf23740 10 client.27025 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5609c2ce7240 con 0x5609c2c4ffd0 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.127+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c016070 con 0x5609c2c4ffd0 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4d480e4020 con 0x5609c2c4ffd0 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 10 client.27025 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 20 client.27025 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2ce7a70 0x5609c2ce9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 4 client.27025 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609c2bc4380 con 0x5609c2ce7a70 2024-07-24T21:56:48.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d537fe640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2ce7a70 0x5609c2ce9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:48.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d537fe640 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2ce7a70 0x5609c2ce9e90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d591df0f6154a02 server_cookie=e93d0908274775b1 in_seq=0 out_seq=0 2024-07-24T21:56:48.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.128+0000 7f4d32ffd640 10 client.27025 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d32ffd640 1 -- 172.21.15.60:0/672053096 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5609c2bc4380 con 0x5609c2ce7a70 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d32ffd640 10 client.27025 handle_command_reply: tid=0 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 client.27025 shutdown 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 msgr2=0x5609c2c7f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 0x5609c2c7f7e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e1c30 tx=0x7f4d480a70e0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 shutdown_connections 2024-07-24T21:56:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5609c2ce7a70 0x5609c2ce9e90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5609c2c5bf60 0x5609c2c7fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5609c2c4ffd0 0x5609c2c7f7e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d32ffd640 0 client.27025 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 --2- 172.21.15.60:0/672053096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609c2c4f6b0 0x5609c2c822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 >> 172.21.15.60:0/672053096 conn(0x5609c2c4d4e0 msgr2=0x5609c2c572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 shutdown_connections 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 1 -- 172.21.15.60:0/672053096 wait complete. 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.145+0000 7f4d5bf23740 20 client.27025 trim_cache size 0 max 16384 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:56:48.148 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (29597 inodes in the stack)", 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: "bf91fbd5-0ad1-4978-92a9-268bd452d98f": { 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "bf91fbd5-0ad1-4978-92a9-268bd452d98f", 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:48.149 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 msgr2=0x7f4d3c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 0x7f4d3c0b6300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4d440096c0 tx=0x7f4d44031400 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 msgr2=0x7f4d54199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9f0 tx=0x7f4d4800edf0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 shutdown_connections 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4d3c0b3e40 0x7f4d3c0b6300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d541079e0 0x7f4d5419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54107060 0x7f4d54199240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 --2- 172.21.15.60:0/1287961762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d54101650 0x7f4d54198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 >> 172.21.15.60:0/1287961762 conn(0x7f4d540fd0c0 msgr2=0x7f4d54105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 shutdown_connections 2024-07-24T21:56:48.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:48.146+0000 7f4d59b72640 1 -- 172.21.15.60:0/1287961762 wait complete. 2024-07-24T21:56:49.244 INFO:tasks.cephfs.filesystem:scrub status for tag:bf91fbd5-0ad1-4978-92a9-268bd452d98f - {'path': '/', 'tag': 'bf91fbd5-0ad1-4978-92a9-268bd452d98f', 'options': 'recursive,force'} 2024-07-24T21:56:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:49 smithi123 ceph-mon[37143]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 3.67k op/s 2024-07-24T21:56:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:49 smithi086 ceph-mon[37128]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 3.67k op/s 2024-07-24T21:56:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:49 smithi060 ceph-mon[33000]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 3.67k op/s 2024-07-24T21:56:51.581 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:51 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 2. 2024-07-24T21:56:51.581 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:51 smithi123 systemd[1]: Stopped Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:56:51.582 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:51 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service: Consumed 21.904s CPU time. 2024-07-24T21:56:51.582 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:51 smithi123 systemd[1]: Starting Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T21:56:51.582 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:51 smithi123 ceph-mon[37143]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 3.19k op/s 2024-07-24T21:56:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:51 smithi086 ceph-mon[37128]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 3.19k op/s 2024-07-24T21:56:51.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:51 smithi060 ceph-mon[33000]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 3.19k op/s 2024-07-24T21:56:51.993 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:51 smithi123 podman[74042]: 2024-07-24 21:56:51.48301442 +0000 UTC m=+0.023199948 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f3499ffb640 1 -- 172.21.15.60:0/3539245538 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5591141a5060 con 0x5591142b8c70 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f3499ffb640 10 client.17589 handle_command_reply: tid=0 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 client.17589 shutdown 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 msgr2=0x5591142b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00f820 tx=0x7f34ac004a40 comp rx=0 tx=0).stop 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 shutdown_connections 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5591142b8c70 0x5591142bb090 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55911423d5a0 0x5591142b6fa0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f3499ffb640 0 client.17589 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559114232cd0 0x559114241f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 --2- 172.21.15.60:0/3539245538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559114231770 0x559114241a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.363+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 >> 172.21.15.60:0/3539245538 conn(0x5591141fb720 msgr2=0x55911422e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 shutdown_connections 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34bad87740 1 -- 172.21.15.60:0/3539245538 wait complete. 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34bad87740 20 client.17589 trim_cache size 0 max 16384 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 42, 2024-07-24T21:56:52.367 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 26, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "f8725f21": { 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 26, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 141.3, 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:56:52.368 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:56:52.369 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 msgr2=0x7f349c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 0x7f349c0b6280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009fd0 tx=0x7f34ac009500 comp rx=0 tx=0).stop 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 msgr2=0x7f34b4198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.364+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 0x7f34b4198170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34a8031440 tx=0x7f34a80316d0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 shutdown_connections 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f349c0b3dc0 0x7f349c0b6280 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b4107ad0 0x7f34b419d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b4107150 0x7f34b4198170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 --2- 172.21.15.60:0/2134433390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b4102360 0x7f34b4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 >> 172.21.15.60:0/2134433390 conn(0x7f34b4076890 msgr2=0x7f34b40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:52.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 shutdown_connections 2024-07-24T21:56:52.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:52.365+0000 7f34b89d6640 1 -- 172.21.15.60:0/2134433390 wait complete. 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 -- 172.21.15.60:0/2761682081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c810c9f0 msgr2=0x7fb0c810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/2761682081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c810c9f0 0x7fb0c810ee40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ca0 tx=0x7fb0c002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 -- 172.21.15.60:0/2761682081 shutdown_connections 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/2761682081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c810c9f0 0x7fb0c810ee40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/2761682081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c8103e40 0x7fb0c810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/2761682081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103440 0x7fb0c8103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 -- 172.21.15.60:0/2761682081 >> 172.21.15.60:0/2761682081 conn(0x7fb0c80fd010 msgr2=0x7fb0c80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 -- 172.21.15.60:0/2761682081 shutdown_connections 2024-07-24T21:56:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.972+0000 7fb0ce68d640 1 -- 172.21.15.60:0/2761682081 wait complete. 2024-07-24T21:56:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.973+0000 7fb0ce68d640 1 Processor -- start 2024-07-24T21:56:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.973+0000 7fb0ce68d640 1 -- start start 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.973+0000 7fb0ce68d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ce68d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ce68d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810c9f0 0x7fb0c819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ce68d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8112640 con 0x7fb0c8103440 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ce68d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81124c0 con 0x7fb0c8103e40 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ce68d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81127c0 con 0x7fb0c810c9f0 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34730/0 (socket says 172.21.15.60:34730) 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36970/0 (socket says 172.21.15.60:36970) 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0bffff640 1 -- 172.21.15.60:0/697969978 learned_addr learned my addr 172.21.15.60:0/697969978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810c9f0 0x7fb0c819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 -- 172.21.15.60:0/697969978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810c9f0 msgr2=0x7fb0c819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810c9f0 0x7fb0c819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 -- 172.21.15.60:0/697969978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 msgr2=0x7fb0c81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0c7fff640 1 -- 172.21.15.60:0/697969978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c819ec20 con 0x7fb0c8103440 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.974+0000 7fb0bffff640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0c5ffb640 1 -- 172.21.15.60:0/697969978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800cd80 con 0x7fb0c8103440 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0c5ffb640 1 -- 172.21.15.60:0/697969978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b80035f0 con 0x7fb0c8103440 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c819ef10 con 0x7fb0c8103440 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0c5ffb640 1 -- 172.21.15.60:0/697969978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8010670 con 0x7fb0c8103440 2024-07-24T21:56:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.975+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a57b0 con 0x7fb0c8103440 2024-07-24T21:56:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.978+0000 7fb0c5ffb640 1 -- 172.21.15.60:0/697969978 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fb0b80037a0 con 0x7fb0c8103440 2024-07-24T21:56:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.978+0000 7fb0d13ed740 1 Processor -- start 2024-07-24T21:56:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.978+0000 7fb0d13ed740 1 -- start start 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbb9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 0x56009dbbc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56009db23280 con 0x56009dbad8c0 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56009db12780 con 0x56009dbba170 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56009db167c0 con 0x56009dbae1e0 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbb9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 0x56009dbbc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c5ffb640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 0x7fb0b40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34746/0 (socket says 172.21.15.60:34746) 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0ccc2a640 1 -- 172.21.15.60:0/1613584982 learned_addr learned my addr 172.21.15.60:0/1613584982 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 0x7fb0b40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c5ffb640 1 -- 172.21.15.60:0/697969978 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fb0b80e4270 con 0x7fb0c8103440 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 -- 172.21.15.60:0/1613584982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 msgr2=0x56009dbbc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 0x56009dbbc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 -- 172.21.15.60:0/1613584982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 msgr2=0x56009dbadca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0c7fff640 1 -- 172.21.15.60:0/1613584982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009d876790 con 0x56009dbae1e0 2024-07-24T21:56:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0bffff640 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 0x56009dbbc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.979+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.980+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbb9c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80e1930 tx=0x7fb0b80b1be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d1e33633f2035f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.980+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 0x7fb0b40b6280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002f500 tx=0x7fb0c002a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.980+0000 7fb0be7fc640 1 -- 172.21.15.60:0/1613584982 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80aadf0 con 0x56009dbae1e0 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.980+0000 7fb0be7fc640 1 -- 172.21.15.60:0/1613584982 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b80af3f0 con 0x56009dbae1e0 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.980+0000 7fb0be7fc640 1 -- 172.21.15.60:0/1613584982 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80e1db0 con 0x56009dbae1e0 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- 172.21.15.60:0/1613584982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 msgr2=0x56009dbb9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbb9c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80e1930 tx=0x7fb0b80b1be0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- 172.21.15.60:0/1613584982 shutdown_connections 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbba170 0x56009dbbc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbb9c30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/1613584982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbad8c0 0x56009dbadca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- 172.21.15.60:0/1613584982 >> 172.21.15.60:0/1613584982 conn(0x56009dbab6f0 msgr2=0x56009dbac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- 172.21.15.60:0/1613584982 shutdown_connections 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- 172.21.15.60:0/1613584982 wait complete. 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 -- start start 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.981+0000 7fb0d13ed740 1 Processor -- start 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbb6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56009da4cab0 con 0x56009dbba170 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbb6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34754/0 (socket says 172.21.15.60:34754) 2024-07-24T21:56:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0ccc2a640 1 -- 172.21.15.60:0/3558645987 learned_addr learned my addr 172.21.15.60:0/3558645987 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56009da9a9e0 con 0x56009dbae1e0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56009dbbcdf0 con 0x56009dbad8c0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 10 client.0 _ll_register_callbacks cb 0x56009d822a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 -- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 msgr2=0x56009dbb98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbb98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 -- 172.21.15.60:0/3558645987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 msgr2=0x56009dbb7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0bffff640 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009d8ef620 con 0x56009dbae1e0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.982+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56009dac1ce0 con 0x56009dbae1e0 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbb6dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb0c810d9e0 tx=0x7fb0b000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6ecf0589f9b4b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0a77fe640 10 client.27041 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0a77fe640 1 -- 172.21.15.60:0/3558645987 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000fe20 con 0x56009dbae1e0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0a77fe640 1 -- 172.21.15.60:0/3558645987 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fb0b0005020 con 0x56009dbae1e0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0a77fe640 1 -- 172.21.15.60:0/3558645987 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb0b0005220 con 0x56009dbae1e0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 10 client.27041 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=42}) -- 0x56009dbc8590 con 0x56009dbae1e0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0a77fe640 1 -- 172.21.15.60:0/3558645987 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fb0b00053c0 con 0x56009dbae1e0 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 10 client.27041 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 10 client.27041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 20 client.27041 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dbc9a10 0x56009dbcbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 4 client.27041 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.983+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009db251d0 con 0x56009dbc9a10 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.984+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dbc9a10 0x56009dbcbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.984+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dbc9a10 0x56009dbcbe00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ef82e3bd38f6a2 server_cookie=cc8de0a1baf8f57e in_seq=0 out_seq=0 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.984+0000 7fb0a77fe640 10 client.27041 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0a77fe640 1 -- 172.21.15.60:0/3558645987 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56009db251d0 con 0x56009dbc9a10 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0a77fe640 10 client.27041 handle_command_reply: tid=0 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 client.27041 shutdown 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 msgr2=0x56009dbb6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbb6dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb0c810d9e0 tx=0x7fb0b000f790 comp rx=0 tx=0).stop 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 shutdown_connections 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dbc9a10 0x56009dbcbe00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbb7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbb6dc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0a77fe640 0 client.27041 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3558645987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 >> 172.21.15.60:0/3558645987 conn(0x56009dbab6f0 msgr2=0x56009dbb1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 shutdown_connections 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3558645987 wait complete. 2024-07-24T21:56:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:52.985+0000 7fb0d13ed740 20 client.27041 trim_cache size 0 max 16384 2024-07-24T21:56:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.004+0000 7fb0d13ed740 1 Processor -- start 2024-07-24T21:56:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.004+0000 7fb0d13ed740 1 -- start start 2024-07-24T21:56:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbc5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbacc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ae880 con 0x56009dbba170 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ac090 con 0x56009dbae1e0 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81a9f00 con 0x56009dbad8c0 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbc5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbacc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36990/0 (socket says 172.21.15.60:36990) 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbc5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47982/0 (socket says 172.21.15.60:47982) 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0ccc2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbacc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34762/0 (socket says 172.21.15.60:34762) 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0c7fff640 1 -- 172.21.15.60:0/3307885999 learned_addr learned my addr 172.21.15.60:0/3307885999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 -- 172.21.15.60:0/3307885999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 msgr2=0x56009dbc5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbc5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 -- 172.21.15.60:0/3307885999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 msgr2=0x56009dbacc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbacc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.005+0000 7fb0bffff640 1 -- 172.21.15.60:0/3307885999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009dbae010 con 0x56009dbae1e0 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.006+0000 7fb0bffff640 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005820 tx=0x7fb0b0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3955b038ebf2d996 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.006+0000 7fb0bdffb640 1 -- 172.21.15.60:0/3307885999 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00053c0 con 0x56009dbae1e0 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.006+0000 7fb0bdffb640 1 -- 172.21.15.60:0/3307885999 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56009dac1ce0 con 0x56009dbae1e0 2024-07-24T21:56:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.006+0000 7fb0bdffb640 1 -- 172.21.15.60:0/3307885999 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b001c7f0 con 0x56009dbae1e0 2024-07-24T21:56:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3307885999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 msgr2=0x56009dbac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005820 tx=0x7fb0b0014000 comp rx=0 tx=0).stop 2024-07-24T21:56:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3307885999 shutdown_connections 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbacc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbae1e0 0x56009dbac740 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/3307885999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbad8c0 0x56009dbc5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3307885999 >> 172.21.15.60:0/3307885999 conn(0x56009dbab6f0 msgr2=0x56009dbb6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3307885999 shutdown_connections 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- 172.21.15.60:0/3307885999 wait complete. 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 -- start start 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.007+0000 7fb0d13ed740 1 Processor -- start 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbbf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbbf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81aa550 con 0x56009dbba170 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ab9c0 con 0x56009dbad8c0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81aac20 con 0x56009dbae1e0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0d13ed740 10 client.0 _ll_register_callbacks cb 0x7fb0b0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbbf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:37000/0 (socket says 172.21.15.60:37000) 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 -- 172.21.15.60:0/2129381426 learned_addr learned my addr 172.21.15.60:0/2129381426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0ccc2a640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbbf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 -- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 msgr2=0x56009dbbf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbbf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 -- 172.21.15.60:0/2129381426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 msgr2=0x56009dbbf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbbf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0c7fff640 1 -- 172.21.15.60:0/2129381426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009dbbfdf0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.008+0000 7fb0bffff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbbf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0c7fff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80a6e70 tx=0x7fb0b80af2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a808a9ef019d8fc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80aadf0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56009dac1ce0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fb0b80a7080 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb0b80b3ea0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 10 client.27047 fetch_fsmap learned FSMap version 42 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56009dc337d0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56009dac1ce0 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fb0b80cd070 con 0x56009dbad8c0 2024-07-24T21:56:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 10 client.27047 fetch_fsmap finished waiting for FSMap version 42 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 10 client.27047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 20 client.27047 populate_metadata read hostname 'smithi060' 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dc33bd0 0x56009dc35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 4 client.27047 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.009+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56009db20c70 con 0x56009dc33bd0 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.010+0000 7fb0bffff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dc33bd0 0x56009dc35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.010+0000 7fb0a77fe640 10 client.27047 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:53.010+0000 7fb0bffff640 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dc33bd0 0x56009dc35fc0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ece1dc4c0f3596 server_cookie=18c8bab0618ea398 in_seq=0 out_seq=0 2024-07-24T21:56:53.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:52 smithi123 podman[74042]: 2024-07-24 21:56:52.820429673 +0000 UTC m=+1.360615184 container create 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-24T21:56:53.558 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f8725f21', quiesce took 19.9 seconds. Will release after: 1.4000000000000021 2024-07-24T21:56:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:53 smithi123 ceph-mon[37143]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 2.69k op/s 2024-07-24T21:56:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:53 smithi086 ceph-mon[37128]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 2.69k op/s 2024-07-24T21:56:53.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:53 smithi060 ceph-mon[33000]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 2.69k op/s 2024-07-24T21:56:54.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:56:55.379 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.719+0000 7fb0a77fe640 1 -- 172.21.15.60:0/2129381426 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56009db20c70 con 0x56009dc33bd0 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.719+0000 7fb0a77fe640 10 client.27047 handle_command_reply: tid=0 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.719+0000 7fb0d13ed740 1 client.27047 shutdown 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 msgr2=0x56009dbc2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80a6e70 tx=0x7fb0b80af2c0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 shutdown_connections 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56009dc33bd0 0x56009dc35fc0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56009dbba170 0x56009dbbf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56009dbae1e0 0x56009dbbf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 --2- 172.21.15.60:0/2129381426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56009dbad8c0 0x56009dbc2770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 >> 172.21.15.60:0/2129381426 conn(0x56009dbab6f0 msgr2=0x56009dbac740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0a77fe640 0 client.27047 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 shutdown_connections 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 1 -- 172.21.15.60:0/2129381426 wait complete. 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.720+0000 7fb0d13ed740 20 client.27047 trim_cache size 0 max 16384 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:56:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:56:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 msgr2=0x7fb0b40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 0x7fb0b40b6280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002f500 tx=0x7fb0c002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 msgr2=0x7fb0c8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 shutdown_connections 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0b40b3dc0 0x7fb0b40b6280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810c9f0 0x7fb0c819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c8103e40 0x7fb0c81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 --2- 172.21.15.60:0/697969978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c8103440 0x7fb0c8198b80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.721+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 >> 172.21.15.60:0/697969978 conn(0x7fb0c80fd010 msgr2=0x7fb0c80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.722+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 shutdown_connections 2024-07-24T21:56:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:56:55.722+0000 7fb0ce68d640 1 -- 172.21.15.60:0/697969978 wait complete. 2024-07-24T21:56:55.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:55 smithi123 ceph-mon[37143]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 897 KiB/s wr, 2.31k op/s 2024-07-24T21:56:55.747 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:55 smithi123 podman[74042]: 2024-07-24 21:56:55.514072178 +0000 UTC m=+4.054257700 container init 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.build-date=20240716, RELEASE=main-86385b1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T21:56:55.747 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:55 smithi123 podman[74042]: 2024-07-24 21:56:55.520404761 +0000 UTC m=+4.060590278 container start 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-24T21:56:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:55 smithi086 ceph-mon[37128]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 897 KiB/s wr, 2.31k op/s 2024-07-24T21:56:55.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:55 smithi060 ceph-mon[33000]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 897 KiB/s wr, 2.31k op/s 2024-07-24T21:56:56.480 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:56 smithi123 ceph-mds[74060]: -- 172.21.15.123:0/3020897999 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6e5b81860 con 0x55a6e5b5f000 2024-07-24T21:56:56.480 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[74056]: starting mds.f at 2024-07-24T21:56:56.480 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:56 smithi123 bash[74042]: 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 2024-07-24T21:56:56.481 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 21:56:56 smithi123 systemd[1]: Started Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T21:56:56.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:56 smithi123 ceph-mon[37143]: daemon mds.f restarted 2024-07-24T21:56:56.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:56 smithi086 ceph-mon[37128]: daemon mds.f restarted 2024-07-24T21:56:56.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:56 smithi060 ceph-mon[33000]: daemon mds.f restarted 2024-07-24T21:56:57.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 -- 172.21.15.60:0/1068817523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c107060 msgr2=0x7f560c1074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:57.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 --2- 172.21.15.60:0/1068817523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c107060 0x7f560c1074a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009900 tx=0x7f55fc02a410 comp rx=0 tx=0).stop 2024-07-24T21:56:57.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 -- 172.21.15.60:0/1068817523 shutdown_connections 2024-07-24T21:56:57.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 --2- 172.21.15.60:0/1068817523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 0x7f560c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 --2- 172.21.15.60:0/1068817523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c107060 0x7f560c1074a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 --2- 172.21.15.60:0/1068817523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c101650 0x7f560c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.286+0000 7f5613c67640 1 -- 172.21.15.60:0/1068817523 >> 172.21.15.60:0/1068817523 conn(0x7f560c0fd0c0 msgr2=0x7f560c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 -- 172.21.15.60:0/1068817523 shutdown_connections 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 -- 172.21.15.60:0/1068817523 wait complete. 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 Processor -- start 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 -- start start 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c101650 0x7f560c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.287+0000 7f5613c67640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5613c67640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 0x7f560c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5613c67640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f560c111630 con 0x7f560c101650 2024-07-24T21:56:57.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5613c67640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1114b0 con 0x7f560c107060 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5613c67640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1117b0 con 0x7f560c1079e0 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41796/0 (socket says 172.21.15.60:41796) 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 -- 172.21.15.60:0/2792760407 learned_addr learned my addr 172.21.15.60:0/2792760407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5612204640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 0x7f560c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611a03640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c101650 0x7f560c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 -- 172.21.15.60:0/2792760407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 msgr2=0x7f560c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 0x7f560c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 -- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c101650 msgr2=0x7f560c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c101650 0x7f560c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 -- 172.21.15.60:0/2792760407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a4ea0 con 0x7f560c107060 2024-07-24T21:56:57.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.288+0000 7f5611202640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.289+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f070 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.289+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc031b60 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.289+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1978b0 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.289+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc035030 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.289+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c197d40 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.290+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c101a30 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.291+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f55fc042070 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.292+0000 7f5602ffd640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 0x7f55f80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.292+0000 7f5611a03640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 0x7f55f80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.292+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f55fc1043b0 con 0x7f560c107060 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.293+0000 7f5611a03640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 0x7f55f80b6200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f56080095f0 tx=0x7f5608005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:56:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.297+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f55fc0cb070 con 0x7f560c107060 2024-07-24T21:56:57.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.551+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560c02e000 con 0x7f560c107060 2024-07-24T21:56:57.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.552+0000 7f5602ffd640 1 -- 172.21.15.60:0/2792760407 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f55fc0cf5c0 con 0x7f560c107060 2024-07-24T21:56:57.554 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:56:57.554 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:56:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 msgr2=0x7f55f80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 0x7f55f80b6200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f56080095f0 tx=0x7f5608005f90 comp rx=0 tx=0).stop 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 msgr2=0x7f560c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc003c10 comp rx=0 tx=0).stop 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 shutdown_connections 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f55f80b3d40 0x7f55f80b6200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c1079e0 0x7f560c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c107060 0x7f560c19a000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 --2- 172.21.15.60:0/2792760407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f560c101650 0x7f560c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:56:57.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 >> 172.21.15.60:0/2792760407 conn(0x7f560c0fd0c0 msgr2=0x7f560c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:56:57.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 shutdown_connections 2024-07-24T21:56:57.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:56:57.554+0000 7f5613c67640 1 -- 172.21.15.60:0/2792760407 wait complete. 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 3.25k op/s 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: osdmap e97: 12 total, 12 up, 12 in 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6834/2832733292,v1:172.21.15.123:6835/2832733292] up:boot 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:56:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:57 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 3.25k op/s 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: osdmap e97: 12 total, 12 up, 12 in 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6834/2832733292,v1:172.21.15.123:6835/2832733292] up:boot 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:56:57.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:57 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:57.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 3.25k op/s 2024-07-24T21:56:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: osdmap e97: 12 total, 12 up, 12 in 2024-07-24T21:56:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6834/2832733292,v1:172.21.15.123:6835/2832733292] up:boot 2024-07-24T21:56:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T21:56:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-24T21:56:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:57 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T21:56:58.222 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id f8725f21 --release --await' 2024-07-24T21:56:58.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id f8725f21 --release --await 2024-07-24T21:56:58.489 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:58 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:58 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2792760407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:58 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:58 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:58 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2792760407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:58.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:58 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:58.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:58 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:56:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:58 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2792760407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:56:58.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:58 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:59.693 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:59 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:59.693 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:59 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:56:59.693 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:56:59 smithi060 ceph-mon[33000]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 904 KiB/s wr, 2.80k op/s 2024-07-24T21:56:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:59 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:59 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:56:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:56:59 smithi123 ceph-mon[37143]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 904 KiB/s wr, 2.80k op/s 2024-07-24T21:56:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:59 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:59 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:56:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:56:59 smithi086 ceph-mon[37128]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 904 KiB/s wr, 2.80k op/s 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 -- 172.21.15.60:0/987199027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 msgr2=0x7f965c10f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 --2- 172.21.15.60:0/987199027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c10f300 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ca0 tx=0x7f965402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 -- 172.21.15.60:0/987199027 shutdown_connections 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 --2- 172.21.15.60:0/987199027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c10f300 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 --2- 172.21.15.60:0/987199027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c108180 0x7f965c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 --2- 172.21.15.60:0/987199027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c102770 0x7f965c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 -- 172.21.15.60:0/987199027 >> 172.21.15.60:0/987199027 conn(0x7f965c0fe1c0 msgr2=0x7f965c1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 -- 172.21.15.60:0/987199027 shutdown_connections 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.227+0000 7f9661092640 1 -- 172.21.15.60:0/987199027 wait complete. 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 Processor -- start 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 -- start start 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 0x7f965c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c108180 0x7f965c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1127b0 con 0x7f965c108b00 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f965c112630 con 0x7f965c108180 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f9661092640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f965c112930 con 0x7f965c102770 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f965b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f965b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45722/0 (socket says 172.21.15.60:45722) 2024-07-24T21:57:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f965b59e640 1 -- 172.21.15.60:0/2845924741 learned_addr learned my addr 172.21.15.60:0/2845924741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f965a59c640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c108180 0x7f965c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.228+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 0x7f965c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 -- 172.21.15.60:0/2845924741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 msgr2=0x7f965c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 0x7f965c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 -- 172.21.15.60:0/2845924741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c108180 msgr2=0x7f965c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c108180 0x7f965c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 -- 172.21.15.60:0/2845924741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c19ecf0 con 0x7f965c108b00 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 0x7f965c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f965b59e640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9654005cc0 tx=0x7f965402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f9643fff640 1 -- 172.21.15.60:0/2845924741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654003cf0 con 0x7f965c108b00 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c19ef80 con 0x7f965c108b00 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f9643fff640 1 -- 172.21.15.60:0/2845924741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96540044f0 con 0x7f965c108b00 2024-07-24T21:57:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f9643fff640 1 -- 172.21.15.60:0/2845924741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965402fd10 con 0x7f965c108b00 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.229+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c1a5720 con 0x7f965c108b00 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 Processor -- start 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9643fff640 1 -- 172.21.15.60:0/2845924741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9654004040 con 0x7f965c108b00 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 -- start start 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x562922552dd0 con 0x5629225e8780 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629225422d0 con 0x5629225dc7f0 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f9663443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562922546310 con 0x5629225dbed0 2024-07-24T21:57:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45736/0 (socket says 172.21.15.60:45736) 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.232+0000 7f965a59c640 1 -- 172.21.15.60:0/1105962257 learned_addr learned my addr 172.21.15.60:0/1105962257 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965ad9d640 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 -- 172.21.15.60:0/1105962257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 msgr2=0x5629225dc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225dc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 -- 172.21.15.60:0/1105962257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 msgr2=0x5629225e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 -- 172.21.15.60:0/1105962257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629222a2b00 con 0x5629225e8780 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f9643fff640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 0x7f96440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965a59c640 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9650011490 tx=0x7f965000a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74dd211cf12289 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 0x7f96440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f9643fff640 1 -- 172.21.15.60:0/2845924741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9654105770 con 0x7f965c108b00 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f96417fa640 1 -- 172.21.15.60:0/1105962257 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650004220 con 0x5629225e8780 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f96417fa640 1 -- 172.21.15.60:0/1105962257 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96500043c0 con 0x5629225e8780 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.233+0000 7f96417fa640 1 -- 172.21.15.60:0/1105962257 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000f690 con 0x5629225e8780 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- 172.21.15.60:0/1105962257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 msgr2=0x5629225eab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9650011490 tx=0x7f965000a750 comp rx=0 tx=0).stop 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 0x7f96440b6200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5629225dabe0 tx=0x7f964c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- 172.21.15.60:0/1105962257 shutdown_connections 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225eab70 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 --2- 172.21.15.60:0/1105962257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225dc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- 172.21.15.60:0/1105962257 >> 172.21.15.60:0/1105962257 conn(0x5629225d9d00 msgr2=0x5629225daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- 172.21.15.60:0/1105962257 shutdown_connections 2024-07-24T21:57:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- 172.21.15.60:0/1105962257 wait complete. 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.234+0000 7f9663443740 1 -- start start 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 Processor -- start 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5629223a5bb0 con 0x5629225e8780 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629224e4900 con 0x5629225dc7f0 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629225eb400 con 0x5629225dbed0 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f9663443740 10 client.0 _ll_register_callbacks cb 0x562922510390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45742/0 (socket says 172.21.15.60:45742) 2024-07-24T21:57:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 -- 172.21.15.60:0/2904490853 learned_addr learned my addr 172.21.15.60:0/2904490853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 -- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 msgr2=0x5629225e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 -- 172.21.15.60:0/2904490853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 msgr2=0x5629225e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965b59e640 1 -- 172.21.15.60:0/2904490853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922319960 con 0x5629225e8780 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.235+0000 7f965a59c640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f965b59e640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9654103030 tx=0x7f9654004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fb143d0bf27b96 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965402feb0 con 0x5629225e8780 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f96540c8720 con 0x5629225e8780 2024-07-24T21:57:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96500043c0 con 0x5629225e8780 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f965402f660 con 0x5629225e8780 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f9663443740 10 client.17643 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629225f6f60 con 0x5629225e8780 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96500043c0 con 0x5629225e8780 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.236+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f96540d50b0 con 0x5629225e8780 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 10 client.17643 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 10 client.17643 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 20 client.17643 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x5629225fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 4 client.17643 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562922554d20 con 0x5629225f8370 2024-07-24T21:57:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.237+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x5629225fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.238+0000 7f963eff5640 10 client.17643 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.238+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x5629225fa790 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdee4c4357ec6c9e server_cookie=c6d6487a210abdb6 in_seq=0 out_seq=0 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.238+0000 7f963eff5640 1 -- 172.21.15.60:0/2904490853 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562922554d20 con 0x5629225f8370 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.238+0000 7f963eff5640 10 client.17643 handle_command_reply: tid=0 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.238+0000 7f9663443740 1 client.17643 shutdown 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 msgr2=0x5629225e6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9654103030 tx=0x7f9654004330 comp rx=0 tx=0).stop 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 shutdown_connections 2024-07-24T21:57:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x5629225fa790 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f963eff5640 0 client.17643 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225e8780 0x5629225e6160 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dc7f0 0x5629225e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 --2- 172.21.15.60:0/2904490853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225dbed0 0x5629225e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 >> 172.21.15.60:0/2904490853 conn(0x5629225d9d00 msgr2=0x5629225df8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 shutdown_connections 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 1 -- 172.21.15.60:0/2904490853 wait complete. 2024-07-24T21:57:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.239+0000 7f9663443740 20 client.17643 trim_cache size 0 max 16384 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 Processor -- start 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 -- start start 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x562922600060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1b0ad0 con 0x5629225dc7f0 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1ae2e0 con 0x5629225dbed0 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.257+0000 7f9663443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1ac130 con 0x5629225e8780 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45746/0 (socket says 172.21.15.60:45746) 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 -- 172.21.15.60:0/2753892765 learned_addr learned my addr 172.21.15.60:0/2753892765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965ad9d640 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x562922600060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965b59e640 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 -- 172.21.15.60:0/2753892765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 msgr2=0x5629225e28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225e28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 -- 172.21.15.60:0/2753892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 msgr2=0x562922600060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x562922600060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 -- 172.21.15.60:0/2753892765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629225dc620 con 0x5629225dc7f0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f965a59c640 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9650011de0 tx=0x7f9650005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845517e7cc0b865 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f9640ff9640 1 -- 172.21.15.60:0/2753892765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965002b070 con 0x5629225dc7f0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f9640ff9640 1 -- 172.21.15.60:0/2753892765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965000fd40 con 0x5629225dc7f0 2024-07-24T21:57:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.258+0000 7f9640ff9640 1 -- 172.21.15.60:0/2753892765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500055d0 con 0x5629225dc7f0 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- 172.21.15.60:0/2753892765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 msgr2=0x5629226005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9650011de0 tx=0x7f9650005ac0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- 172.21.15.60:0/2753892765 shutdown_connections 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225e28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629226005a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 --2- 172.21.15.60:0/2753892765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x562922600060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- 172.21.15.60:0/2753892765 >> 172.21.15.60:0/2753892765 conn(0x5629225d9d00 msgr2=0x5629225e7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- 172.21.15.60:0/2753892765 shutdown_connections 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- 172.21.15.60:0/2753892765 wait complete. 2024-07-24T21:57:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.259+0000 7f9663443740 1 -- start start 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 Processor -- start 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x5629225ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1ac7a0 con 0x5629225dc7f0 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1adc10 con 0x5629225dbed0 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1ace70 con 0x5629225e8780 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f9663443740 10 client.0 _ll_register_callbacks cb 0x7f9654003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45754/0 (socket says 172.21.15.60:45754) 2024-07-24T21:57:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 -- 172.21.15.60:0/3075932532 learned_addr learned my addr 172.21.15.60:0/3075932532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965b59e640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 -- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 msgr2=0x5629225ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225ed150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 -- 172.21.15.60:0/3075932532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 msgr2=0x5629225ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965ad9d640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x5629225ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x5629225ec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.260+0000 7f965a59c640 1 -- 172.21.15.60:0/3075932532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922666640 con 0x5629225dc7f0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f965a59c640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9650005e70 tx=0x7f9650004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b801687368de15d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965002b040 con 0x5629225dc7f0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f965000f890 con 0x5629225dc7f0 2024-07-24T21:57:00.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f965000fd40 con 0x5629225dc7f0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f965003d050 con 0x5629225dc7f0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f9663443740 10 client.17649 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562922667960 con 0x5629225dc7f0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965000fd40 con 0x5629225dc7f0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.261+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f965002e070 con 0x5629225dc7f0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 10 client.17649 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 10 client.17649 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 20 client.17649 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x562922669f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 4 client.17649 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f8725f21", "release": true, "await": true}] 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f8725f21", "release": true, "await": true}) -- 0x5629225507c0 con 0x5629225f8370 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f965ad9d640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x562922669f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f965ad9d640 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x562922669f40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c717ce82bba2812 server_cookie=8d5fd0df65971baa in_seq=0 out_seq=0 2024-07-24T21:57:00.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:00.262+0000 7f963eff5640 10 client.17649 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f963eff5640 1 -- 172.21.15.60:0/3075932532 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5629225507c0 con 0x5629225f8370 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f963eff5640 10 client.17649 handle_command_reply: tid=0 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 client.17649 shutdown 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 msgr2=0x5629225ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9650005e70 tx=0x7f9650004de0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 shutdown_connections 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5629225f8370 0x562922669f40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629225e8780 0x5629225ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629225dc7f0 0x5629225ecc10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 --2- 172.21.15.60:0/3075932532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629225dbed0 0x5629225ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f963eff5640 0 client.17649 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.643+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 >> 172.21.15.60:0/3075932532 conn(0x5629225d9d00 msgr2=0x5629225e7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.644+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 shutdown_connections 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.644+0000 7f9663443740 1 -- 172.21.15.60:0/3075932532 wait complete. 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.644+0000 7f9663443740 20 client.17649 trim_cache size 0 max 16384 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:57:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 28, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "f8725f21": { 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 28, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 141.3, 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:57:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:01.649 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:57:01.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 msgr2=0x7f96440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 0x7f96440b6200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5629225dabe0 tx=0x7f964c008040 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 msgr2=0x7f965c19e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9654005cc0 tx=0x7f965402f470 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 shutdown_connections 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f96440b3d40 0x7f96440b6200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f965c108b00 0x7f965c19e520 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f965c108180 0x7f965c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 --2- 172.21.15.60:0/2845924741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f965c102770 0x7f965c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 >> 172.21.15.60:0/2845924741 conn(0x7f965c0fe1c0 msgr2=0x7f965c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.645+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 shutdown_connections 2024-07-24T21:57:01.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:01.646+0000 7f9661092640 1 -- 172.21.15.60:0/2845924741 wait complete. 2024-07-24T21:57:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:01 smithi123 ceph-mon[37143]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 902 KiB/s wr, 3.49k op/s 2024-07-24T21:57:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:01 smithi086 ceph-mon[37128]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 902 KiB/s wr, 3.49k op/s 2024-07-24T21:57:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:01 smithi060 ceph-mon[33000]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 902 KiB/s wr, 3.49k op/s 2024-07-24T21:57:02.446 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f8725f21', total seconds elapsed: 28.8 2024-07-24T21:57:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:03 smithi086 ceph-mon[37128]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 901 KiB/s wr, 3.96k op/s 2024-07-24T21:57:03.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:03 smithi060 ceph-mon[33000]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 901 KiB/s wr, 3.96k op/s 2024-07-24T21:57:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:03 smithi123 ceph-mon[37143]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 901 KiB/s wr, 3.96k op/s 2024-07-24T21:57:05.795 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:05 smithi123 ceph-mon[37143]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 KiB/s wr, 3.82k op/s 2024-07-24T21:57:05.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:05 smithi086 ceph-mon[37128]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 KiB/s wr, 3.82k op/s 2024-07-24T21:57:05.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:05 smithi060 ceph-mon[33000]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 KiB/s wr, 3.82k op/s 2024-07-24T21:57:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 224 KiB/s wr, 2.45k op/s 2024-07-24T21:57:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:57:07.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 224 KiB/s wr, 2.45k op/s 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:57:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:57:07.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 224 KiB/s wr, 2.45k op/s 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:57:07.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:57:07.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:57:08.743 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 21:57:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T21:57:08.394+0000 7f827ed99640 -1 mds.0.bal find_exports balancer runs too long 2024-07-24T21:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 -- 172.21.15.60:0/2225276517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008103ed0 msgr2=0x7f300810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2225276517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008103ed0 0x7f300810d8d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3000009ca0 tx=0x7f300002a8e0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 -- 172.21.15.60:0/2225276517 shutdown_connections 2024-07-24T21:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2225276517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008103ed0 0x7f300810d8d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2225276517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2225276517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008101170 0x7f3008101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 -- 172.21.15.60:0/2225276517 >> 172.21.15.60:0/2225276517 conn(0x7f3008076760 msgr2=0x7f3008076b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.866+0000 7f300cb1e640 1 -- 172.21.15.60:0/2225276517 shutdown_connections 2024-07-24T21:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.867+0000 7f300cb1e640 1 -- 172.21.15.60:0/2225276517 wait complete. 2024-07-24T21:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.868+0000 7f300cb1e640 1 Processor -- start 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.870+0000 7f300cb1e640 1 -- start start 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.870+0000 7f300cb1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 0x7f30081967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300cb1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300cb1e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008103ed0 0x7f300819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300cb1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3008110560 con 0x7f3008101170 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300cb1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30081103e0 con 0x7f3008103530 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300cb1e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30081106e0 con 0x7f3008103ed0 2024-07-24T21:57:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 0x7f30081967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 0x7f30081967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39120/0 (socket says 172.21.15.60:39120) 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f300659c640 1 -- 172.21.15.60:0/2339931878 learned_addr learned my addr 172.21.15.60:0/2339931878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.871+0000 7f2ffffff640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f3006d9d640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008103ed0 0x7f300819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f2ffffff640 1 -- 172.21.15.60:0/2339931878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008103ed0 msgr2=0x7f300819c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f2ffffff640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008103ed0 0x7f300819c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f2ffffff640 1 -- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 msgr2=0x7f30081967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f2ffffff640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 0x7f30081967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.872+0000 7f2ffffff640 1 -- 172.21.15.60:0/2339931878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300819c860 con 0x7f3008103530 2024-07-24T21:57:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.873+0000 7f2ffffff640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008196d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000aec0 tx=0x7f2ff000d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.876+0000 7f2fff7fe640 1 -- 172.21.15.60:0/2339931878 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff001a070 con 0x7f3008103530 2024-07-24T21:57:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.876+0000 7f2fff7fe640 1 -- 172.21.15.60:0/2339931878 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff000c3d0 con 0x7f3008103530 2024-07-24T21:57:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.876+0000 7f2fff7fe640 1 -- 172.21.15.60:0/2339931878 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff001d5c0 con 0x7f3008103530 2024-07-24T21:57:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.877+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300819cb50 con 0x7f3008103530 2024-07-24T21:57:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.877+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30081a3210 con 0x7f3008103530 2024-07-24T21:57:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.883+0000 7f2fff7fe640 1 -- 172.21.15.60:0/2339931878 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2ff001d070 con 0x7f3008103530 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.884+0000 7f300f87c740 1 Processor -- start 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.885+0000 7f300f87c740 1 -- start start 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f795a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f795aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f2f78cc280 con 0x55f2f795f9f0 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2f78bb780 con 0x55f2f795f0d0 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f300f87c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2f78bf7c0 con 0x55f2f795a5a0 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f3006d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f3006d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54426/0 (socket says 172.21.15.60:54426) 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.886+0000 7f3006d9d640 1 -- 172.21.15.60:0/320381973 learned_addr learned my addr 172.21.15.60:0/320381973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f2ffffff640 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f795aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 -- 172.21.15.60:0/320381973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 msgr2=0x55f2f795aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f795aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 -- 172.21.15.60:0/320381973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 msgr2=0x55f2f795a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f795a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 -- 172.21.15.60:0/320381973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f761f790 con 0x55f2f795f0d0 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f3006d9d640 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f30000095d0 tx=0x7f3000001140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182651ac16defa24 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.887+0000 7f2fff7fe640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 0x7f2ff40b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.888+0000 7f2fff7fe640 1 -- 172.21.15.60:0/2339931878 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f2ff00e79e0 con 0x7f3008103530 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.888+0000 7f2ffcff9640 1 -- 172.21.15.60:0/320381973 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300003f070 con 0x55f2f795f0d0 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.888+0000 7f2ffcff9640 1 -- 172.21.15.60:0/320381973 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3000036040 con 0x55f2f795f0d0 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.888+0000 7f2ffcff9640 1 -- 172.21.15.60:0/320381973 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30000042a0 con 0x55f2f795f0d0 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.888+0000 7f300659c640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 0x7f2ff40b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.889+0000 7f300f87c740 1 -- 172.21.15.60:0/320381973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 msgr2=0x55f2f795f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.889+0000 7f300659c640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 0x7f2ff40b63d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005e50 tx=0x7f2ff8005f00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f30000095d0 tx=0x7f3000001140 comp rx=0 tx=0).stop 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 -- 172.21.15.60:0/320381973 shutdown_connections 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f795aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f795a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 --2- 172.21.15.60:0/320381973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f0d0 0x55f2f795f4b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.890+0000 7f300f87c740 1 -- 172.21.15.60:0/320381973 >> 172.21.15.60:0/320381973 conn(0x55f2f795cf00 msgr2=0x55f2f795e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.892+0000 7f300f87c740 1 -- 172.21.15.60:0/320381973 shutdown_connections 2024-07-24T21:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.892+0000 7f300f87c740 1 -- 172.21.15.60:0/320381973 wait complete. 2024-07-24T21:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.893+0000 7f300f87c740 1 -- start start 2024-07-24T21:57:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.893+0000 7f300f87c740 1 Processor -- start 2024-07-24T21:57:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795a5a0 0x55f2f7958140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f79586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f2f77f5ab0 con 0x55f2f795f9f0 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2f78439e0 con 0x55f2f795a5a0 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f3006d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f3006d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39136/0 (socket says 172.21.15.60:39136) 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f3006d9d640 1 -- 172.21.15.60:0/2911281146 learned_addr learned my addr 172.21.15.60:0/2911281146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2f795b230 con 0x55f2f795f0d0 2024-07-24T21:57:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.894+0000 7f300f87c740 10 client.0 _ll_register_callbacks cb 0x55f2f75cba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f300659c640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795a5a0 0x55f2f7958140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f3006d9d640 1 -- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 msgr2=0x55f2f79586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f3006d9d640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f79586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f3006d9d640 1 -- 172.21.15.60:0/2911281146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795a5a0 msgr2=0x55f2f7958140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f3006d9d640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795a5a0 0x55f2f7958140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.895+0000 7f3006d9d640 1 -- 172.21.15.60:0/2911281146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f7698620 con 0x55f2f795f9f0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f3006d9d640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3000009bf0 tx=0x7f3000031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21978ef6c3f81c82 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f2fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000051060 con 0x55f2f795f9f0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f3000003c80 con 0x55f2f795f9f0 2024-07-24T21:57:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2f786ace0 con 0x55f2f795f9f0 2024-07-24T21:57:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.896+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f300003f040 con 0x55f2f795f9f0 2024-07-24T21:57:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.897+0000 7f300f87c740 10 client.17658 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.897+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f2f7982640 con 0x55f2f795f9f0 2024-07-24T21:57:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.897+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3000036070 con 0x55f2f795f9f0 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3000030940 con 0x55f2f795f9f0 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 10 client.17658 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 20 client.17658 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7985d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 4 client.17658 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2f786ace0 con 0x55f2f7983990 2024-07-24T21:57:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.898+0000 7f300659c640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7985d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.903+0000 7f300659c640 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7985d80 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194a4044ce5a3e8a server_cookie=d3a84ef44f1159c6 in_seq=0 out_seq=0 2024-07-24T21:57:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.903+0000 7f2fedffb640 10 client.17658 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f2fedffb640 1 -- 172.21.15.60:0/2911281146 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2f786ace0 con 0x55f2f7983990 2024-07-24T21:57:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f2fedffb640 10 client.17658 handle_command_reply: tid=0 2024-07-24T21:57:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 client.17658 shutdown 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 msgr2=0x55f2f79784a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3000009bf0 tx=0x7f3000031c20 comp rx=0 tx=0).stop 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 shutdown_connections 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7985d80 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f9f0 0x55f2f79784a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.905+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f79586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f2fedffb640 0 client.17658 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f300f87c740 1 --2- 172.21.15.60:0/2911281146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795a5a0 0x55f2f7958140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 >> 172.21.15.60:0/2911281146 conn(0x55f2f795cf00 msgr2=0x55f2f7962aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 shutdown_connections 2024-07-24T21:57:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f300f87c740 1 -- 172.21.15.60:0/2911281146 wait complete. 2024-07-24T21:57:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.906+0000 7f300f87c740 20 client.17658 trim_cache size 0 max 16384 2024-07-24T21:57:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300f87c740 1 Processor -- start 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300f87c740 1 -- start start 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300f87c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300f87c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 0x55f2f7952f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300f87c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.931+0000 7f300659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52680/0 (socket says 172.21.15.60:52680) 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f300659c640 1 -- 172.21.15.60:0/122062118 learned_addr learned my addr 172.21.15.60:0/122062118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30081ae850 con 0x55f2f795f0d0 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f2ffffff640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 0x55f2f7952f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30081ac060 con 0x55f2f795f9f0 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30081a9eb0 con 0x55f2f795a5a0 2024-07-24T21:57:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.932+0000 7f3006d9d640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f2ffffff640 1 -- 172.21.15.60:0/122062118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 msgr2=0x55f2f7961130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f2ffffff640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f2ffffff640 1 -- 172.21.15.60:0/122062118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 msgr2=0x55f2f79534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f2ffffff640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f2ffffff640 1 -- 172.21.15.60:0/122062118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f7956430 con 0x55f2f795f0d0 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.933+0000 7f300659c640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:57:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.934+0000 7f2ffffff640 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 0x55f2f7952f90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00e51e0 tx=0x7f2ff00ae490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604bc77d9cfe8dcc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.934+0000 7f2feffff640 1 -- 172.21.15.60:0/122062118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00b7090 con 0x55f2f795f0d0 2024-07-24T21:57:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.934+0000 7f2feffff640 1 -- 172.21.15.60:0/122062118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff00ae1b0 con 0x55f2f795f0d0 2024-07-24T21:57:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f2feffff640 1 -- 172.21.15.60:0/122062118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00d10d0 con 0x55f2f795f0d0 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 msgr2=0x55f2f7952f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 0x55f2f7952f90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00e51e0 tx=0x7f2ff00ae490 comp rx=0 tx=0).stop 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 shutdown_connections 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795f0d0 0x55f2f7952f90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 --2- 172.21.15.60:0/122062118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795a5a0 0x55f2f7961130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 >> 172.21.15.60:0/122062118 conn(0x55f2f795cf00 msgr2=0x55f2f795c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 shutdown_connections 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.935+0000 7f300f87c740 1 -- 172.21.15.60:0/122062118 wait complete. 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 -- start start 2024-07-24T21:57:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 Processor -- start 2024-07-24T21:57:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f7979c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30081aa520 con 0x55f2f795a5a0 2024-07-24T21:57:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30081ab990 con 0x55f2f795f9f0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30081aabf0 con 0x55f2f795f0d0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.936+0000 7f300f87c740 10 client.0 _ll_register_callbacks cb 0x7f3000009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39162/0 (socket says 172.21.15.60:39162) 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 -- 172.21.15.60:0/523111656 learned_addr learned my addr 172.21.15.60:0/523111656 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f2ffffff640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f7979c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f3006d9d640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 -- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 msgr2=0x55f2f7979c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f7979c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 -- 172.21.15.60:0/523111656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 msgr2=0x55f2f79fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.937+0000 7f300659c640 1 -- 172.21.15.60:0/523111656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f79fdaa0 con 0x55f2f795a5a0 2024-07-24T21:57:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.938+0000 7f300659c640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8010660 tx=0x7f2ff8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc44da6d67b09e95 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.938+0000 7f2fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.938+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005520 con 0x55f2f795a5a0 2024-07-24T21:57:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.938+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f2ff80047c0 con 0x55f2f795a5a0 2024-07-24T21:57:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.938+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff00ae1b0 con 0x55f2f795a5a0 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.939+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8033050 con 0x55f2f795a5a0 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.939+0000 7f300f87c740 10 client.17664 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.939+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f2f79fed80 con 0x55f2f795a5a0 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff00ae1b0 con 0x55f2f795a5a0 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f2ff801d910 con 0x55f2f795a5a0 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 10 client.17664 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 10 client.17664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 20 client.17664 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7a015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 4 client.17664 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:57:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.940+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2f78ce1d0 con 0x55f2f7983990 2024-07-24T21:57:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.943+0000 7f2ffffff640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7a015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.946+0000 7f2fedffb640 10 client.17664 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:08.946+0000 7f2ffffff640 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7a015b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec12aa1aca747f72 server_cookie=a3030726ab8b7907 in_seq=0 out_seq=0 2024-07-24T21:57:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:09 smithi086 ceph-mon[37128]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 380 KiB/s wr, 2.08k op/s 2024-07-24T21:57:09.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:09 smithi060 ceph-mon[33000]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 380 KiB/s wr, 2.08k op/s 2024-07-24T21:57:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:09 smithi123 ceph-mon[37143]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 380 KiB/s wr, 2.08k op/s 2024-07-24T21:57:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f2fedffb640 1 -- 172.21.15.60:0/523111656 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2f78ce1d0 con 0x55f2f7983990 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f2fedffb640 10 client.17664 handle_command_reply: tid=0 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f300f87c740 1 client.17664 shutdown 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 msgr2=0x55f2f79796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8010660 tx=0x7f2ff8005de0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 shutdown_connections 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.320+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f2f7983990 0x55f2f7a015b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2f795f9f0 0x55f2f79fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2f795f0d0 0x55f2f7979c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 --2- 172.21.15.60:0/523111656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f795a5a0 0x55f2f79796c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 >> 172.21.15.60:0/523111656 conn(0x55f2f795cf00 msgr2=0x55f2f7952f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f2fedffb640 0 client.17664 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 shutdown_connections 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 1 -- 172.21.15.60:0/523111656 wait complete. 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300f87c740 20 client.17664 trim_cache size 0 max 16384 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:57:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 msgr2=0x7f2ff40b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.321+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 0x7f2ff40b63d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005e50 tx=0x7f2ff8005f00 comp rx=0 tx=0).stop 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 msgr2=0x7f3008196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008196d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000aec0 tx=0x7f2ff000d7b0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 shutdown_connections 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2ff40b3f10 0x7f2ff40b63d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3008103ed0 0x7f300819c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008103530 0x7f3008196d00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 --2- 172.21.15.60:0/2339931878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3008101170 0x7f30081967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 >> 172.21.15.60:0/2339931878 conn(0x7f3008076760 msgr2=0x7f30081055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 shutdown_connections 2024-07-24T21:57:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:11.322+0000 7f300cb1e640 1 -- 172.21.15.60:0/2339931878 wait complete. 2024-07-24T21:57:11.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:11 smithi086 ceph-mon[37128]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-07-24T21:57:11.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:11 smithi060 ceph-mon[33000]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-07-24T21:57:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:11 smithi123 ceph-mon[37143]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 1.19k op/s 2024-07-24T21:57:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:13 smithi086 ceph-mon[37128]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-07-24T21:57:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:13 smithi060 ceph-mon[33000]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-07-24T21:57:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:13 smithi123 ceph-mon[37143]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 659 op/s 2024-07-24T21:57:14.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:14 smithi086 ceph-mon[37128]: osdmap e98: 12 total, 12 up, 12 in 2024-07-24T21:57:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:14 smithi060 ceph-mon[33000]: osdmap e98: 12 total, 12 up, 12 in 2024-07-24T21:57:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:14 smithi123 ceph-mon[37143]: osdmap e98: 12 total, 12 up, 12 in 2024-07-24T21:57:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:15 smithi086 ceph-mon[37128]: pgmap v705: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-07-24T21:57:15.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:15 smithi060 ceph-mon[33000]: pgmap v705: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-07-24T21:57:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:15 smithi123 ceph-mon[37143]: pgmap v705: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 278 op/s 2024-07-24T21:57:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:17 smithi086 ceph-mon[37128]: pgmap v706: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 353 op/s 2024-07-24T21:57:17.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:17 smithi060 ceph-mon[33000]: pgmap v706: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 353 op/s 2024-07-24T21:57:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:17 smithi123 ceph-mon[37143]: pgmap v706: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 353 op/s 2024-07-24T21:57:19.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:57:19.703 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:19 smithi086 ceph-mon[37128]: pgmap v707: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-07-24T21:57:19.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:19 smithi060 ceph-mon[33000]: pgmap v707: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-07-24T21:57:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:19 smithi123 ceph-mon[37143]: pgmap v707: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-07-24T21:57:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:21 smithi086 ceph-mon[37128]: pgmap v708: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 344 op/s 2024-07-24T21:57:21.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:21 smithi060 ceph-mon[33000]: pgmap v708: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 344 op/s 2024-07-24T21:57:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 -- 172.21.15.60:0/734291050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7124103f90 msgr2=0x7f712410c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 --2- 172.21.15.60:0/734291050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7124103f90 0x7f712410c570 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7110009900 tx=0x7f711002a410 comp rx=0 tx=0).stop 2024-07-24T21:57:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 -- 172.21.15.60:0/734291050 shutdown_connections 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 --2- 172.21.15.60:0/734291050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410cb40 0x7f712410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 --2- 172.21.15.60:0/734291050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7124103f90 0x7f712410c570 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 --2- 172.21.15.60:0/734291050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103590 0x7f7124103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 -- 172.21.15.60:0/734291050 >> 172.21.15.60:0/734291050 conn(0x7f71240fd120 msgr2=0x7f71240ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 -- 172.21.15.60:0/734291050 shutdown_connections 2024-07-24T21:57:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.845+0000 7f712abdc640 1 -- 172.21.15.60:0/734291050 wait complete. 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.846+0000 7f712abdc640 1 Processor -- start 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.846+0000 7f712abdc640 1 -- start start 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7124103590 0x7f712410b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 0x7f712410bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71241127c0 con 0x7f712410cb40 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112640 con 0x7f7124103590 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f712abdc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112940 con 0x7f7124103f90 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7128978640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7124103590 0x7f712410b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59494/0 (socket says 172.21.15.60:59494) 2024-07-24T21:57:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 -- 172.21.15.60:0/3313337090 learned_addr learned my addr 172.21.15.60:0/3313337090 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7123fff640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 0x7f712410bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 -- 172.21.15.60:0/3313337090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 msgr2=0x7f712410bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 0x7f712410bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 -- 172.21.15.60:0/3313337090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7124103590 msgr2=0x7f712410b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7124103590 0x7f712410b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 -- 172.21.15.60:0/3313337090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712419fd40 con 0x7f712410cb40 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7123fff640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 0x7f712410bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.847+0000 7f7129179640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b9e0 tx=0x7f711c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.848+0000 7f7121ffb640 1 -- 172.21.15.60:0/3313337090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c019070 con 0x7f712410cb40 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.848+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241a0030 con 0x7f712410cb40 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.848+0000 7f7121ffb640 1 -- 172.21.15.60:0/3313337090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c014440 con 0x7f712410cb40 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.848+0000 7f7121ffb640 1 -- 172.21.15.60:0/3313337090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0125a0 con 0x7f712410cb40 2024-07-24T21:57:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.848+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712419c610 con 0x7f712410cb40 2024-07-24T21:57:21.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f7121ffb640 1 -- 172.21.15.60:0/3313337090 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f711c01c070 con 0x7f712410cb40 2024-07-24T21:57:21.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 Processor -- start 2024-07-24T21:57:21.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 -- start start 2024-07-24T21:57:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c5911989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c591103960 con 0x55c59118c680 2024-07-24T21:57:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5910f2e60 con 0x55c59118cfa0 2024-07-24T21:57:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f712cf8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c5910f6ea0 con 0x55c591198f30 2024-07-24T21:57:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.851+0000 7f7123fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59498/0 (socket says 172.21.15.60:59498) 2024-07-24T21:57:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 -- 172.21.15.60:0/3331670795 learned_addr learned my addr 172.21.15.60:0/3331670795 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 -- 172.21.15.60:0/3331670795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 msgr2=0x55c59119b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 -- 172.21.15.60:0/3331670795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 msgr2=0x55c5911989f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c5911989f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 -- 172.21.15.60:0/3331670795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59107c2d0 con 0x55c59118c680 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.852+0000 7f7123fff640 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f711002a930 tx=0x7f71100058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab254b800e6d0fbe server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.853+0000 7f7121ffb640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 0x7f71140b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.853+0000 7f7121ffb640 1 -- 172.21.15.60:0/3313337090 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f711c0e3450 con 0x7f712410cb40 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.854+0000 7f7128978640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 0x7f71140b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.855+0000 7f7128978640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 0x7f71140b6280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71180095f0 tx=0x7f7118005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.855+0000 7f710effd640 1 -- 172.21.15.60:0/3331670795 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110035030 con 0x55c59118c680 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.855+0000 7f710effd640 1 -- 172.21.15.60:0/3331670795 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110005050 con 0x55c59118c680 2024-07-24T21:57:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.855+0000 7f710effd640 1 -- 172.21.15.60:0/3331670795 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004920 con 0x55c59118c680 2024-07-24T21:57:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 -- 172.21.15.60:0/3331670795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 msgr2=0x55c59118ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f711002a930 tx=0x7f71100058b0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 -- 172.21.15.60:0/3331670795 shutdown_connections 2024-07-24T21:57:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c5911989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3331670795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c59118ca60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.856+0000 7f712cf8d740 1 -- 172.21.15.60:0/3331670795 >> 172.21.15.60:0/3331670795 conn(0x55c59118a4b0 msgr2=0x55c59118b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.857+0000 7f712cf8d740 1 -- 172.21.15.60:0/3331670795 shutdown_connections 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.857+0000 7f712cf8d740 1 -- 172.21.15.60:0/3331670795 wait complete. 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.857+0000 7f712cf8d740 1 -- start start 2024-07-24T21:57:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.857+0000 7f712cf8d740 1 Processor -- start 2024-07-24T21:57:21.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.858+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c591195f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.858+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c591196490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.858+0000 7f7128978640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c591195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.858+0000 7f7128978640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c591195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59500/0 (socket says 172.21.15.60:59500) 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.858+0000 7f7128978640 1 -- 172.21.15.60:0/3065419112 learned_addr learned my addr 172.21.15.60:0/3065419112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c5911969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c590f56700 con 0x55c59118c680 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5910955a0 con 0x55c59118cfa0 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c59119bb50 con 0x55c591198f30 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f712cf8d740 10 client.0 _ll_register_callbacks cb 0x55c5910b5910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c591196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 -- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 msgr2=0x55c5911969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c5911969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 -- 172.21.15.60:0/3065419112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 msgr2=0x55c591195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c591195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.859+0000 7f7123fff640 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c591088630 con 0x55c59118cfa0 2024-07-24T21:57:21.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.861+0000 7f7123fff640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c591196490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7110031440 tx=0x7f71100048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d202b9e519ed531a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.861+0000 7f7103fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:21.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.861+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711004d020 con 0x55c59118cfa0 2024-07-24T21:57:21.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.861+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110005050 con 0x55c59118cfa0 2024-07-24T21:57:21.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.861+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f7110004c70 con 0x55c59118cfa0 2024-07-24T21:57:21.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.862+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7110058050 con 0x55c59118cfa0 2024-07-24T21:57:21.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.862+0000 7f712cf8d740 10 client.27089 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:21.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.862+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c590e53650 con 0x55c59118cfa0 2024-07-24T21:57:21.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.862+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110005050 con 0x55c59118cfa0 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f7110049b60 con 0x55c59118cfa0 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 10 client.27089 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 10 client.27089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 20 client.27089 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5911a8a70 0x55c5911aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 4 client.27089 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:21.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.863+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5911058b0 con 0x55c5911a8a70 2024-07-24T21:57:21.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.866+0000 7f7128978640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5911a8a70 0x55c5911aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.866+0000 7f7103fff640 10 client.27089 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:21.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.866+0000 7f7128978640 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5911a8a70 0x55c5911aaee0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d59d4ffb3f85ad3 server_cookie=80ef50d7bb8df497 in_seq=0 out_seq=0 2024-07-24T21:57:21.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f7103fff640 1 -- 172.21.15.60:0/3065419112 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5911058b0 con 0x55c5911a8a70 2024-07-24T21:57:21.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f7103fff640 10 client.27089 handle_command_reply: tid=0 2024-07-24T21:57:21.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 client.27089 shutdown 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 msgr2=0x55c591196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c591196490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7110031440 tx=0x7f71100048f0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 shutdown_connections 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c5911a8a70 0x55c5911aaee0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c5911969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118cfa0 0x55c591196490 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f7103fff640 0 client.27089 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3065419112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118c680 0x55c591195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 >> 172.21.15.60:0/3065419112 conn(0x55c59118a4b0 msgr2=0x55c591190050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 shutdown_connections 2024-07-24T21:57:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 1 -- 172.21.15.60:0/3065419112 wait complete. 2024-07-24T21:57:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.868+0000 7f712cf8d740 20 client.27089 trim_cache size 0 max 16384 2024-07-24T21:57:21.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.898+0000 7f712cf8d740 1 Processor -- start 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.898+0000 7f712cf8d740 1 -- start start 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.898+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c59118cfa0 0x55c5911b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 0x55c59118f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f7128978640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f7128978640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48586/0 (socket says 172.21.15.60:48586) 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f7128978640 1 -- 172.21.15.60:0/1638855067 learned_addr learned my addr 172.21.15.60:0/1638855067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f712cf8d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bfa70 con 0x55c591198f30 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bd280 con 0x55c59118c680 2024-07-24T21:57:21.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f7123fff640 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c59118cfa0 0x55c5911b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bb0f0 con 0x55c59118cfa0 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.899+0000 7f7129179640 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 0x55c59118f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 -- 172.21.15.60:0/1638855067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c59118cfa0 msgr2=0x55c5911b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c59118cfa0 0x55c5911b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 -- 172.21.15.60:0/1638855067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 msgr2=0x55c5911b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 -- 172.21.15.60:0/1638855067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5911ab560 con 0x55c591198f30 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f7129179640 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 0x55c59118f7c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f711c0e0bf0 tx=0x7f711c0aa160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27db8ef15f9d50d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.900+0000 7f710e7fc640 1 -- 172.21.15.60:0/1638855067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0e0d60 con 0x55c591198f30 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.901+0000 7f710e7fc640 1 -- 172.21.15.60:0/1638855067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c0a63f0 con 0x55c591198f30 2024-07-24T21:57:21.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.901+0000 7f710e7fc640 1 -- 172.21.15.60:0/1638855067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0b2f30 con 0x55c591198f30 2024-07-24T21:57:21.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 msgr2=0x55c59118f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 0x55c59118f7c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f711c0e0bf0 tx=0x7f711c0aa160 comp rx=0 tx=0).stop 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 shutdown_connections 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c591198f30 0x55c59118f7c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c59118cfa0 0x55c5911b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 --2- 172.21.15.60:0/1638855067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 >> 172.21.15.60:0/1638855067 conn(0x55c59118a4b0 msgr2=0x55c591197da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 shutdown_connections 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.902+0000 7f712cf8d740 1 -- 172.21.15.60:0/1638855067 wait complete. 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.903+0000 7f712cf8d740 1 -- start start 2024-07-24T21:57:21.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.903+0000 7f712cf8d740 1 Processor -- start 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 0x55c5911a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bb740 con 0x55c59118cfa0 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bcbb0 con 0x55c59118c680 2024-07-24T21:57:21.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71241bbe10 con 0x55c591198f30 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f712cf8d740 10 client.0 _ll_register_callbacks cb 0x7f7110009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f7128978640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f7128978640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48598/0 (socket says 172.21.15.60:48598) 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.904+0000 7f7128978640 1 -- 172.21.15.60:0/3526727604 learned_addr learned my addr 172.21.15.60:0/3526727604 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 0x55c5911a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7129179640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 -- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 msgr2=0x55c59119c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 -- 172.21.15.60:0/3526727604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 msgr2=0x55c5911a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.905+0000 7f7123fff640 1 -- 172.21.15.60:0/3526727604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59118a2e0 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.906+0000 7f7123fff640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 0x55c5911a16b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f711003d4d0 tx=0x7f71100057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0214e5674b9aa5c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.906+0000 7f7103fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.906+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110061060 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.906+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711c0a63f0 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.907+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f7110004c70 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.907+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f711004d040 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.907+0000 7f712cf8d740 10 client.17682 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.907+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c591224320 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110032d90 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f7110031a20 con 0x55c59118cfa0 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 10 client.17682 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 20 client.17682 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c591224a10 0x55c591226e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 4 client.17682 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:57:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f711c0a63f0 con 0x55c591224a10 2024-07-24T21:57:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.908+0000 7f7128978640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c591224a10 0x55c591226e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f7103fff640 10 client.17682 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f7128978640 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c591224a10 0x55c591226e30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d13e800f77ae8f3 server_cookie=e3282e82b96df45d in_seq=0 out_seq=0 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f7103fff640 1 -- 172.21.15.60:0/3526727604 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f711c0a63f0 con 0x55c591224a10 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f7103fff640 10 client.17682 handle_command_reply: tid=0 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 client.17682 shutdown 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 msgr2=0x55c5911a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 0x55c5911a16b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f711003d4d0 tx=0x7f71100057c0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 shutdown_connections 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c591224a10 0x55c591226e30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c591198f30 0x55c59119c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c59118cfa0 0x55c5911a16b0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.909+0000 7f712cf8d740 1 --2- 172.21.15.60:0/3526727604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c59118c680 0x55c5911a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.910+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 >> 172.21.15.60:0/3526727604 conn(0x55c59118a4b0 msgr2=0x55c591197580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.910+0000 7f7103fff640 0 client.17682 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.910+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 shutdown_connections 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.910+0000 7f712cf8d740 1 -- 172.21.15.60:0/3526727604 wait complete. 2024-07-24T21:57:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.910+0000 7f712cf8d740 20 client.17682 trim_cache size 0 max 16384 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 msgr2=0x7f71140b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 0x7f71140b6280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71180095f0 tx=0x7f7118005f90 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 msgr2=0x7f712419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b9e0 tx=0x7f711c00bea0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 shutdown_connections 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f71140b3dc0 0x7f71140b6280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712410cb40 0x7f712419f5c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7124103f90 0x7f712410bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 --2- 172.21.15.60:0/3313337090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7124103590 0x7f712410b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 >> 172.21.15.60:0/3313337090 conn(0x7f71240fd120 msgr2=0x7f712410ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 shutdown_connections 2024-07-24T21:57:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:21.911+0000 7f712abdc640 1 -- 172.21.15.60:0/3313337090 wait complete. 2024-07-24T21:57:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:21 smithi123 ceph-mon[37143]: pgmap v708: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 344 op/s 2024-07-24T21:57:22.805 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:57:22.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:57:23.313 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:23.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:23 smithi060 ceph-mon[33000]: pgmap v709: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-07-24T21:57:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:23 smithi123 ceph-mon[37143]: pgmap v709: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-07-24T21:57:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:23 smithi086 ceph-mon[37128]: pgmap v709: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-07-24T21:57:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.471+0000 7f74b2ffd640 1 -- 172.21.15.60:0/1714230250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0307c0 con 0x7f74b410c9f0 2024-07-24T21:57:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.472+0000 7f74ba5e8640 1 -- 172.21.15.60:0/1714230250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b410c9f0 msgr2=0x7f74b410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.472+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/1714230250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b410c9f0 0x7f74b410ee70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009ca0 tx=0x7f74ac02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.478+0000 7f74ba5e8640 1 -- 172.21.15.60:0/1714230250 shutdown_connections 2024-07-24T21:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.478+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/1714230250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b410c9f0 0x7f74b410ee70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.478+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/1714230250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b4103ec0 0x7f74b410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.478+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/1714230250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b41035a0 0x7f74b4103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.478+0000 7f74ba5e8640 1 -- 172.21.15.60:0/1714230250 >> 172.21.15.60:0/1714230250 conn(0x7f74b40fd010 msgr2=0x7f74b40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.483+0000 7f74ba5e8640 1 -- 172.21.15.60:0/1714230250 shutdown_connections 2024-07-24T21:57:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.483+0000 7f74ba5e8640 1 -- 172.21.15.60:0/1714230250 wait complete. 2024-07-24T21:57:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.484+0000 7f74ba5e8640 1 Processor -- start 2024-07-24T21:57:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.484+0000 7f74ba5e8640 1 -- start start 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 0x7f74b410b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b4103ec0 0x7f74b410bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 0x7f74b419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4112730 con 0x7f74b41035a0 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41125b0 con 0x7f74b410c9f0 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.485+0000 7f74ba5e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41128b0 con 0x7f74b4103ec0 2024-07-24T21:57:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.486+0000 7f74b8b85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 0x7f74b419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b8b85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 0x7f74b419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48622/0 (socket says 172.21.15.60:48622) 2024-07-24T21:57:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b8b85640 1 -- 172.21.15.60:0/3938679198 learned_addr learned my addr 172.21.15.60:0/3938679198 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 0x7f74b410b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 -- 172.21.15.60:0/3938679198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b4103ec0 msgr2=0x7f74b410bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b4103ec0 0x7f74b410bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 -- 172.21.15.60:0/3938679198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 msgr2=0x7f74b419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 0x7f74b419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.487+0000 7f74b3fff640 1 -- 172.21.15.60:0/3938679198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b419ba50 con 0x7f74b41035a0 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.488+0000 7f74b3fff640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 0x7f74b410b6a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.489+0000 7f74b17fa640 1 -- 172.21.15.60:0/3938679198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a400ccc0 con 0x7f74b41035a0 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.489+0000 7f74b17fa640 1 -- 172.21.15.60:0/3938679198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a40108e0 con 0x7f74b41035a0 2024-07-24T21:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.489+0000 7f74b17fa640 1 -- 172.21.15.60:0/3938679198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4018810 con 0x7f74b41035a0 2024-07-24T21:57:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.490+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b419bce0 con 0x7f74b41035a0 2024-07-24T21:57:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.490+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b419bfd0 con 0x7f74b41035a0 2024-07-24T21:57:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.494+0000 7f74bd419740 1 Processor -- start 2024-07-24T21:57:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.494+0000 7f74bd419740 1 -- start start 2024-07-24T21:57:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561f72738280 con 0x561f727cf170 2024-07-24T21:57:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f72727780 con 0x561f727c28c0 2024-07-24T21:57:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74bd419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f7272b7c0 con 0x561f727c31e0 2024-07-24T21:57:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74b8b85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74b8b85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48624/0 (socket says 172.21.15.60:48624) 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.495+0000 7f74b8b85640 1 -- 172.21.15.60:0/3688111301 learned_addr learned my addr 172.21.15.60:0/3688111301 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.496+0000 7f74b8b85640 1 -- 172.21.15.60:0/3688111301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 msgr2=0x561f727cec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.496+0000 7f74b8b85640 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.496+0000 7f74b8b85640 1 -- 172.21.15.60:0/3688111301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 msgr2=0x561f727d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.496+0000 7f74b8b85640 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.496+0000 7f74b8b85640 1 -- 172.21.15.60:0/3688111301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7248b790 con 0x561f727c28c0 2024-07-24T21:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.497+0000 7f74b8b85640 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74ac031c30 tx=0x7f74ac009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0490b17b859a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.497+0000 7f74aa7fc640 1 -- 172.21.15.60:0/3688111301 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0317d0 con 0x561f727c28c0 2024-07-24T21:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.498+0000 7f74aa7fc640 1 -- 172.21.15.60:0/3688111301 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac036040 con 0x561f727c28c0 2024-07-24T21:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.498+0000 7f74aa7fc640 1 -- 172.21.15.60:0/3688111301 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0305c0 con 0x561f727c28c0 2024-07-24T21:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.498+0000 7f74b17fa640 1 -- 172.21.15.60:0/3938679198 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f74a4010430 con 0x7f74b41035a0 2024-07-24T21:57:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.499+0000 7f74bd419740 1 -- 172.21.15.60:0/3688111301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 msgr2=0x561f727c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.499+0000 7f74bd419740 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74ac031c30 tx=0x7f74ac009a20 comp rx=0 tx=0).stop 2024-07-24T21:57:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 -- 172.21.15.60:0/3688111301 shutdown_connections 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 --2- 172.21.15.60:0/3688111301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727c2ca0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 -- 172.21.15.60:0/3688111301 >> 172.21.15.60:0/3688111301 conn(0x561f727c06f0 msgr2=0x561f727c1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 -- 172.21.15.60:0/3688111301 shutdown_connections 2024-07-24T21:57:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.500+0000 7f74bd419740 1 -- 172.21.15.60:0/3688111301 wait complete. 2024-07-24T21:57:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.501+0000 7f74b17fa640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 0x7f74980b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.501+0000 7f74b17fa640 1 -- 172.21.15.60:0/3938679198 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f74a40e4710 con 0x7f74b41035a0 2024-07-24T21:57:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.503+0000 7f74bd419740 1 -- start start 2024-07-24T21:57:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.503+0000 7f74bd419740 1 Processor -- start 2024-07-24T21:57:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74b8b85640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 0x7f74980b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561f72661ab0 con 0x561f727cf170 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f726af9e0 con 0x561f727c28c0 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f727d1df0 con 0x561f727c31e0 2024-07-24T21:57:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.505+0000 7f74bd419740 10 client.0 _ll_register_callbacks cb 0x561f72437a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.507+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.507+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44350/0 (socket says 172.21.15.60:44350) 2024-07-24T21:57:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.507+0000 7f74b37fe640 1 -- 172.21.15.60:0/750795161 learned_addr learned my addr 172.21.15.60:0/750795161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.507+0000 7f74b3fff640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.509+0000 7f74b3fff640 1 -- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 msgr2=0x561f727cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.509+0000 7f74b3fff640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.509+0000 7f74b3fff640 1 -- 172.21.15.60:0/750795161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 msgr2=0x561f727cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.509+0000 7f74b3fff640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.509+0000 7f74b3fff640 1 -- 172.21.15.60:0/750795161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f72504620 con 0x561f727c28c0 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.510+0000 7f74b8b85640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 0x7f74980b6210 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0045f0 tx=0x7f74ac02bfd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.510+0000 7f74b3fff640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727ce8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74a40e1da0 tx=0x7f74a40aa9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb55dbd73653f235 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.510+0000 7f74977fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.513+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40ab960 con 0x561f727c28c0 2024-07-24T21:57:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.513+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f74a40ab6d0 con 0x561f727c28c0 2024-07-24T21:57:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.513+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f726d6ce0 con 0x561f727c28c0 2024-07-24T21:57:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.514+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f74a40ab260 con 0x561f727c28c0 2024-07-24T21:57:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.517+0000 7f74bd419740 10 client.27104 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.517+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561f727dd650 con 0x561f727c28c0 2024-07-24T21:57:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.517+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f726d6ce0 con 0x561f727c28c0 2024-07-24T21:57:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f74a40a6c60 con 0x561f727c28c0 2024-07-24T21:57:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 10 client.27104 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 10 client.27104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 20 client.27104 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f727def80 0x561f727e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 4 client.27104 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.518+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7273a1d0 con 0x561f727def80 2024-07-24T21:57:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.519+0000 7f74b37fe640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f727def80 0x561f727e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.521+0000 7f74b37fe640 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f727def80 0x561f727e13a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6269ed9ca76f595 server_cookie=14b86cd45d0089d5 in_seq=0 out_seq=0 2024-07-24T21:57:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.521+0000 7f74977fe640 10 client.27104 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.524+0000 7f74977fe640 1 -- 172.21.15.60:0/750795161 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f7273a1d0 con 0x561f727def80 2024-07-24T21:57:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.524+0000 7f74977fe640 10 client.27104 handle_command_reply: tid=0 2024-07-24T21:57:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.524+0000 7f74bd419740 1 client.27104 shutdown 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 msgr2=0x561f727ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727ce8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74a40e1da0 tx=0x7f74a40aa9d0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 shutdown_connections 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f727def80 0x561f727e13a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f727cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f727cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 --2- 172.21.15.60:0/750795161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727ce8f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 >> 172.21.15.60:0/750795161 conn(0x561f727c06f0 msgr2=0x561f727c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74977fe640 0 client.27104 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 shutdown_connections 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 1 -- 172.21.15.60:0/750795161 wait complete. 2024-07-24T21:57:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.526+0000 7f74bd419740 20 client.27104 trim_cache size 0 max 16384 2024-07-24T21:57:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.558+0000 7f74bd419740 1 Processor -- start 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.558+0000 7f74bd419740 1 -- start start 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727cf170 0x561f727e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bfbc0 con 0x561f727c31e0 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bd3d0 con 0x561f727c28c0 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74bd419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bb240 con 0x561f727cf170 2024-07-24T21:57:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59556/0 (socket says 172.21.15.60:59556) 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 -- 172.21.15.60:0/2664253233 learned_addr learned my addr 172.21.15.60:0/2664253233 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 -- 172.21.15.60:0/2664253233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727cf170 msgr2=0x561f727e6c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727cf170 0x561f727e6c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 -- 172.21.15.60:0/2664253233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 msgr2=0x561f727daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.559+0000 7f74b37fe640 1 -- 172.21.15.60:0/2664253233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f727ca2d0 con 0x561f727c31e0 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.560+0000 7f74b37fe640 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f74a00094f0 tx=0x7f74a0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8114ca25f375ef49 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.560+0000 7f74a9ffb640 1 -- 172.21.15.60:0/2664253233 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0022070 con 0x561f727c31e0 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.560+0000 7f74a9ffb640 1 -- 172.21.15.60:0/2664253233 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a0014dc0 con 0x561f727c31e0 2024-07-24T21:57:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.560+0000 7f74a9ffb640 1 -- 172.21.15.60:0/2664253233 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a001dc00 con 0x561f727c31e0 2024-07-24T21:57:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.561+0000 7f74bd419740 1 -- 172.21.15.60:0/2664253233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 msgr2=0x561f727cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.561+0000 7f74bd419740 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f74a00094f0 tx=0x7f74a0009d50 comp rx=0 tx=0).stop 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 -- 172.21.15.60:0/2664253233 shutdown_connections 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727cf170 0x561f727e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727c31e0 0x561f727cc2d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 --2- 172.21.15.60:0/2664253233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f727daf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 -- 172.21.15.60:0/2664253233 >> 172.21.15.60:0/2664253233 conn(0x561f727c06f0 msgr2=0x561f727cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 -- 172.21.15.60:0/2664253233 shutdown_connections 2024-07-24T21:57:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.563+0000 7f74bd419740 1 -- 172.21.15.60:0/2664253233 wait complete. 2024-07-24T21:57:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 -- start start 2024-07-24T21:57:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 Processor -- start 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f72841c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f72842140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f72842680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bb890 con 0x561f727cf170 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bcd00 con 0x561f727c28c0 2024-07-24T21:57:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41bbf60 con 0x561f727c31e0 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.564+0000 7f74bd419740 10 client.0 _ll_register_callbacks cb 0x7f74a40056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f72842140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f72842140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44364/0 (socket says 172.21.15.60:44364) 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b37fe640 1 -- 172.21.15.60:0/2188099625 learned_addr learned my addr 172.21.15.60:0/2188099625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f72841c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b8b85640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f72842680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 -- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 msgr2=0x561f72842140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f72842140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 -- 172.21.15.60:0/2188099625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 msgr2=0x561f72842680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f72842680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.565+0000 7f74b3fff640 1 -- 172.21.15.60:0/2188099625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f727d60d0 con 0x561f727c28c0 2024-07-24T21:57:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.566+0000 7f74b3fff640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f72841c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a72e0 tx=0x7f74a4010870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfac787ea99aa42 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.566+0000 7f74977fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.566+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40ab960 con 0x561f727c28c0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.566+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a0014dc0 con 0x561f727c28c0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.566+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f74a40cc3a0 con 0x561f727c28c0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.568+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f74a408c050 con 0x561f727c28c0 2024-07-24T21:57:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.569+0000 7f74bd419740 10 client.27110 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.569+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561f727d73e0 con 0x561f727c28c0 2024-07-24T21:57:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.570+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a0014dc0 con 0x561f727c28c0 2024-07-24T21:57:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f74a40b3f50 con 0x561f727c28c0 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 10 client.27110 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 10 client.27110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 20 client.27110 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f7285ac50 0x561f7285d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 4 client.27110 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:57:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.571+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f72735c70 con 0x561f7285ac50 2024-07-24T21:57:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.573+0000 7f74b37fe640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f7285ac50 0x561f7285d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.575+0000 7f74b37fe640 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f7285ac50 0x561f7285d040 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219e4fdabaa08a71 server_cookie=755e69cfbd71c6c3 in_seq=0 out_seq=0 2024-07-24T21:57:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:24.575+0000 7f74977fe640 10 client.27110 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74977fe640 1 -- 172.21.15.60:0/2188099625 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f72735c70 con 0x561f7285ac50 2024-07-24T21:57:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74977fe640 10 client.27110 handle_command_reply: tid=0 2024-07-24T21:57:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 client.27110 shutdown 2024-07-24T21:57:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 msgr2=0x561f72841c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f72841c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a72e0 tx=0x7f74a4010870 comp rx=0 tx=0).stop 2024-07-24T21:57:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 shutdown_connections 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561f7285ac50 0x561f7285d040 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f727cf170 0x561f72842680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f727c31e0 0x561f72842140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 --2- 172.21.15.60:0/2188099625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f727c28c0 0x561f72841c00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.296+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 >> 172.21.15.60:0/2188099625 conn(0x561f727c06f0 msgr2=0x561f727cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.297+0000 7f74977fe640 0 client.27110 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.297+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 shutdown_connections 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.297+0000 7f74bd419740 1 -- 172.21.15.60:0/2188099625 wait complete. 2024-07-24T21:57:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.297+0000 7f74bd419740 20 client.27110 trim_cache size 0 max 16384 2024-07-24T21:57:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:57:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:57:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:57:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 msgr2=0x7f74980b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 0x7f74980b6210 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0045f0 tx=0x7f74ac02bfd0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 msgr2=0x7f74b410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 0x7f74b410b6a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 shutdown_connections 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3d50 0x7f74980b6210 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b410c9f0 0x7f74b419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b4103ec0 0x7f74b410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 --2- 172.21.15.60:0/3938679198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74b41035a0 0x7f74b410b6a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 >> 172.21.15.60:0/3938679198 conn(0x7f74b40fd010 msgr2=0x7f74b410eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.298+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 shutdown_connections 2024-07-24T21:57:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:25.299+0000 7f74ba5e8640 1 -- 172.21.15.60:0/3938679198 wait complete. 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.346+0000 7fa3d1af0640 1 -- 172.21.15.60:0/155540253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10ad00 msgr2=0x7fa3cc111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.346+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/155540253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc111500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402a410 comp rx=0 tx=0).stop 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 -- 172.21.15.60:0/155540253 shutdown_connections 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/155540253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc111500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/155540253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc10a380 0x7fa3cc10a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/155540253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc104970 0x7fa3cc104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 -- 172.21.15.60:0/155540253 >> 172.21.15.60:0/155540253 conn(0x7fa3cc1003e0 msgr2=0x7fa3cc102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 -- 172.21.15.60:0/155540253 shutdown_connections 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.347+0000 7fa3d1af0640 1 -- 172.21.15.60:0/155540253 wait complete. 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.348+0000 7fa3d1af0640 1 Processor -- start 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.348+0000 7fa3d1af0640 1 -- start start 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.348+0000 7fa3d1af0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 0x7fa3cc10fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3d1af0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 0x7fa3cc0745c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:25.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3d1af0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc110430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3d1af0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc078120 con 0x7fa3cc104970 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3d1af0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc077fa0 con 0x7fa3cc10ad00 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3d1af0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0782a0 con 0x7fa3cc10a380 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3cb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 0x7fa3cc10fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3cb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 0x7fa3cc10fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59584/0 (socket says 172.21.15.60:59584) 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3cb7fe640 1 -- 172.21.15.60:0/3111124095 learned_addr learned my addr 172.21.15.60:0/3111124095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:25.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3cbfff640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc110430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.349+0000 7fa3caffd640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 0x7fa3cc0745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.351+0000 7fa3caffd640 1 -- 172.21.15.60:0/3111124095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc10ad00 msgr2=0x7fa3cc110430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.351+0000 7fa3caffd640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc110430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.351+0000 7fa3caffd640 1 -- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 msgr2=0x7fa3cc10fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.351+0000 7fa3caffd640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 0x7fa3cc10fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.351+0000 7fa3caffd640 1 -- 172.21.15.60:0/3111124095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc113870 con 0x7fa3cc10a380 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.353+0000 7fa3caffd640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 0x7fa3cc0745c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400b420 tx=0x7fa3c400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.354+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4011830 con 0x7fa3cc10a380 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.354+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c4016070 con 0x7fa3cc10a380 2024-07-24T21:57:25.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.354+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4010dd0 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.356+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc113b60 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.356+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1140a0 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.358+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc105c20 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.362+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa3c401a410 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.363+0000 7fa3c8ff9640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 0x7fa3b80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.363+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7fa3c40e3570 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.364+0000 7fa3cb7fe640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 0x7fa3b80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.367+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa3c401f060 con 0x7fa3cc10a380 2024-07-24T21:57:25.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.367+0000 7fa3cb7fe640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 0x7fa3b80b6300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009fd0 tx=0x7fa3bc009720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:25.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:25 smithi060 ceph-mon[33000]: pgmap v710: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-07-24T21:57:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.829+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3cc104d50 con 0x7fa3cc10a380 2024-07-24T21:57:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.830+0000 7fa3c8ff9640 1 -- 172.21.15.60:0/3111124095 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fa3c40aa960 con 0x7fa3cc10a380 2024-07-24T21:57:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:57:25.833 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:57:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:57:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.836+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 msgr2=0x7fa3b80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.836+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 0x7fa3b80b6300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009fd0 tx=0x7fa3bc009720 comp rx=0 tx=0).stop 2024-07-24T21:57:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.836+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 msgr2=0x7fa3cc0745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.836+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 0x7fa3cc0745c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400b420 tx=0x7fa3c400b8e0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 shutdown_connections 2024-07-24T21:57:25.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3b80b3e40 0x7fa3b80b6300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3cc10ad00 0x7fa3cc110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3cc10a380 0x7fa3cc0745c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 --2- 172.21.15.60:0/3111124095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104970 0x7fa3cc10fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:25.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.838+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 >> 172.21.15.60:0/3111124095 conn(0x7fa3cc1003e0 msgr2=0x7fa3cc1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:25.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.842+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 shutdown_connections 2024-07-24T21:57:25.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:25.846+0000 7fa3d1af0640 1 -- 172.21.15.60:0/3111124095 wait complete. 2024-07-24T21:57:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:25 smithi123 ceph-mon[37143]: pgmap v710: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-07-24T21:57:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:25 smithi086 ceph-mon[37128]: pgmap v710: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-07-24T21:57:26.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:57:26.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:26 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3111124095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:26.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:26 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3111124095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:27.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:26 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3111124095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:27 smithi060 ceph-mon[33000]: pgmap v711: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 298 op/s 2024-07-24T21:57:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:27 smithi123 ceph-mon[37143]: pgmap v711: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 298 op/s 2024-07-24T21:57:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:27 smithi086 ceph-mon[37128]: pgmap v711: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 298 op/s 2024-07-24T21:57:29.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.089+0000 7f6d6e181640 1 -- 172.21.15.60:0/405858380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68074f30 msgr2=0x7f6d68075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.089+0000 7f6d6e181640 1 --2- 172.21.15.60:0/405858380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68074f30 0x7f6d68075310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009860 tx=0x7f6d6402a340 comp rx=0 tx=0).stop 2024-07-24T21:57:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.090+0000 7f6d6e181640 1 -- 172.21.15.60:0/405858380 shutdown_connections 2024-07-24T21:57:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.090+0000 7f6d6e181640 1 --2- 172.21.15.60:0/405858380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d68113330 0x7f6d68115720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.090+0000 7f6d6e181640 1 --2- 172.21.15.60:0/405858380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68075850 0x7f6d68112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.090+0000 7f6d6e181640 1 --2- 172.21.15.60:0/405858380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68074f30 0x7f6d68075310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.090+0000 7f6d6e181640 1 -- 172.21.15.60:0/405858380 >> 172.21.15.60:0/405858380 conn(0x7f6d680fe060 msgr2=0x7f6d68100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.093+0000 7f6d6e181640 1 -- 172.21.15.60:0/405858380 shutdown_connections 2024-07-24T21:57:29.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.093+0000 7f6d6e181640 1 -- 172.21.15.60:0/405858380 wait complete. 2024-07-24T21:57:29.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.095+0000 7f6d6e181640 1 Processor -- start 2024-07-24T21:57:29.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.095+0000 7f6d6e181640 1 -- start start 2024-07-24T21:57:29.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.095+0000 7f6d6e181640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.096+0000 7f6d6e181640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68075850 0x7f6d6819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.096+0000 7f6d6d17f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.096+0000 7f6d6d17f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60886/0 (socket says 172.21.15.60:60886) 2024-07-24T21:57:29.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.096+0000 7f6d6d17f640 1 -- 172.21.15.60:0/493596785 learned_addr learned my addr 172.21.15.60:0/493596785 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.096+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d68113330 0x7f6d681a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68075850 0x7f6d6819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68118bb0 con 0x7f6d68113330 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68118a30 con 0x7f6d68074f30 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68118d30 con 0x7f6d68075850 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6d17f640 1 -- 172.21.15.60:0/493596785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68075850 msgr2=0x7f6d6819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68075850 0x7f6d6819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6d17f640 1 -- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d68113330 msgr2=0x7f6d681a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d68113330 0x7f6d681a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.097+0000 7f6d6d17f640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d681a5090 con 0x7f6d68074f30 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.098+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6d640306b0 tx=0x7f6d64033e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.098+0000 7f6d5e7fc640 1 -- 172.21.15.60:0/493596785 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64041020 con 0x7f6d68074f30 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.098+0000 7f6d5e7fc640 1 -- 172.21.15.60:0/493596785 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d64004390 con 0x7f6d68074f30 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.098+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d681a5320 con 0x7f6d68074f30 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.099+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d681aba40 con 0x7f6d68074f30 2024-07-24T21:57:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.099+0000 7f6d5e7fc640 1 -- 172.21.15.60:0/493596785 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037070 con 0x7f6d68074f30 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 Processor -- start 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 -- start start 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b0dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560a18a789b0 con 0x560a18b01850 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a18a67eb0 con 0x560a18b02170 2024-07-24T21:57:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.103+0000 7f6d71794740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a18a6bef0 con 0x560a18b0e100 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d5e7fc640 1 -- 172.21.15.60:0/493596785 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6d64044020 con 0x7f6d68074f30 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33792/0 (socket says 172.21.15.60:33792) 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 -- 172.21.15.60:0/4227427971 learned_addr learned my addr 172.21.15.60:0/4227427971 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b0dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 -- 172.21.15.60:0/4227427971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 msgr2=0x560a18b104f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:57:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b104f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 -- 172.21.15.60:0/4227427971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 msgr2=0x560a18b0dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b0dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.104+0000 7f6d6d980640 1 -- 172.21.15.60:0/4227427971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a189f1320 con 0x560a18b01850 2024-07-24T21:57:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b0dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:57:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d6d980640 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009d50 tx=0x7f6d6000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31044848dc3af5ae server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d537fe640 1 -- 172.21.15.60:0/4227427971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60016070 con 0x560a18b01850 2024-07-24T21:57:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d537fe640 1 -- 172.21.15.60:0/4227427971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x560a18b01850 2024-07-24T21:57:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d537fe640 1 -- 172.21.15.60:0/4227427971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600116b0 con 0x560a18b01850 2024-07-24T21:57:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.105+0000 7f6d5e7fc640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 0x7f6d540b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d5e7fc640 1 -- 172.21.15.60:0/493596785 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f6d641044c0 con 0x7f6d68074f30 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 -- 172.21.15.60:0/4227427971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 msgr2=0x560a18b01c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009d50 tx=0x7f6d6000e6c0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 -- 172.21.15.60:0/4227427971 shutdown_connections 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b0dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.106+0000 7f6d71794740 1 --2- 172.21.15.60:0/4227427971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b01c30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.107+0000 7f6d71794740 1 -- 172.21.15.60:0/4227427971 >> 172.21.15.60:0/4227427971 conn(0x560a18aff680 msgr2=0x560a18b00820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.107+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 0x7f6d540b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.107+0000 7f6d6d17f640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 0x7f6d540b6320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d64101d70 tx=0x7f6d64005c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.109+0000 7f6d71794740 1 -- 172.21.15.60:0/4227427971 shutdown_connections 2024-07-24T21:57:29.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.110+0000 7f6d71794740 1 -- 172.21.15.60:0/4227427971 wait complete. 2024-07-24T21:57:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.110+0000 7f6d71794740 1 -- start start 2024-07-24T21:57:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.111+0000 7f6d71794740 1 Processor -- start 2024-07-24T21:57:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.111+0000 7f6d71794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 0x560a18b0b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.111+0000 7f6d71794740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.111+0000 7f6d71794740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d71794740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560a188cb700 con 0x560a18b02170 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d71794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a18a0a5f0 con 0x560a18b01850 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d71794740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a18b10d80 con 0x560a18b0e100 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d71794740 10 client.0 _ll_register_callbacks cb 0x560a18a2a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33796/0 (socket says 172.21.15.60:33796) 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d6c97e640 1 -- 172.21.15.60:0/35404961 learned_addr learned my addr 172.21.15.60:0/35404961 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d6d980640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 0x560a18b0b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.112+0000 7f6d6d980640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6c97e640 1 -- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 msgr2=0x560a18b0bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b0bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6c97e640 1 -- 172.21.15.60:0/35404961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 msgr2=0x560a18b0b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 0x560a18b0b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6c97e640 1 -- 172.21.15.60:0/35404961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a189fd680 con 0x560a18b02170 2024-07-24T21:57:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.113+0000 7f6d6d980640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 0x560a18b0b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.114+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b910 tx=0x7f6d5800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ff1a9a265c3fcb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.114+0000 7f6d50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.115+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5801a470 con 0x560a18b02170 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.115+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f6d5801a610 con 0x560a18b02170 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.115+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d60005020 con 0x560a18b02170 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.116+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6d5801a810 con 0x560a18b02170 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.116+0000 7f6d71794740 10 client.17706 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.116+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560a187c8650 con 0x560a18b02170 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.116+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x560a18b02170 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.116+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6d58011b30 con 0x560a18b02170 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 10 client.17706 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 10 client.17706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 20 client.17706 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b20450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 4 client.17706 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a18a7a900 con 0x560a18b1e010 2024-07-24T21:57:29.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.117+0000 7f6d6d980640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b20450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.122+0000 7f6d6d980640 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b20450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55f0d9b317086f7 server_cookie=fd70724ccd02475c in_seq=0 out_seq=0 2024-07-24T21:57:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.123+0000 7f6d50ff9640 10 client.17706 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:57:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.129+0000 7f6d50ff9640 1 -- 172.21.15.60:0/35404961 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a18a7a900 con 0x560a18b1e010 2024-07-24T21:57:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.129+0000 7f6d50ff9640 10 client.17706 handle_command_reply: tid=0 2024-07-24T21:57:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.129+0000 7f6d71794740 1 client.17706 shutdown 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 msgr2=0x560a18b0b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b910 tx=0x7f6d5800bdd0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 shutdown_connections 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b20450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b0bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d50ff9640 0 client.17706 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b02170 0x560a18b0b570 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 --2- 172.21.15.60:0/35404961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b01850 0x560a18b0b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.131+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 >> 172.21.15.60:0/35404961 conn(0x560a18aff680 msgr2=0x560a18b05220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.132+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 shutdown_connections 2024-07-24T21:57:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.132+0000 7f6d71794740 1 -- 172.21.15.60:0/35404961 wait complete. 2024-07-24T21:57:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.132+0000 7f6d71794740 20 client.17706 trim_cache size 0 max 16384 2024-07-24T21:57:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.160+0000 7f6d71794740 1 Processor -- start 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d71794740 1 -- start start 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d71794740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b09f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d71794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d71794740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b25d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d71794740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b70a0 con 0x560a18b01850 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.161+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.162+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60908/0 (socket says 172.21.15.60:60908) 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.162+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1004486457 learned_addr learned my addr 172.21.15.60:0/1004486457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.162+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b48b0 con 0x560a18b02170 2024-07-24T21:57:29.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.162+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b2720 con 0x560a18b0e100 2024-07-24T21:57:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6d980640 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b25d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6d980640 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b09f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1004486457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 msgr2=0x560a18b25d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b25d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1004486457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 msgr2=0x560a18b09f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b09f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.165+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1004486457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a18b01fa0 con 0x560a18b02170 2024-07-24T21:57:29.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.167+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6d58001eb0 tx=0x7f6d580041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c24f142ca23c75c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.168+0000 7f6d52ffd640 1 -- 172.21.15.60:0/1004486457 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802f070 con 0x560a18b02170 2024-07-24T21:57:29.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.168+0000 7f6d52ffd640 1 -- 172.21.15.60:0/1004486457 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58015070 con 0x560a18b02170 2024-07-24T21:57:29.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.168+0000 7f6d52ffd640 1 -- 172.21.15.60:0/1004486457 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005540 con 0x560a18b02170 2024-07-24T21:57:29.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.169+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 msgr2=0x560a18b050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.169+0000 7f6d71794740 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6d58001eb0 tx=0x7f6d580041b0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.170+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 shutdown_connections 2024-07-24T21:57:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.170+0000 7f6d71794740 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b0e100 0x560a18b25d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.170+0000 7f6d71794740 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b050a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.170+0000 7f6d71794740 1 --2- 172.21.15.60:0/1004486457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b01850 0x560a18b09f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.170+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 >> 172.21.15.60:0/1004486457 conn(0x560a18aff680 msgr2=0x560a18b04990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.171+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 shutdown_connections 2024-07-24T21:57:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.171+0000 7f6d71794740 1 -- 172.21.15.60:0/1004486457 wait complete. 2024-07-24T21:57:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.172+0000 7f6d71794740 1 -- start start 2024-07-24T21:57:29.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.172+0000 7f6d71794740 1 Processor -- start 2024-07-24T21:57:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.173+0000 7f6d71794740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b01850 0x560a18b12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.173+0000 7f6d71794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.173+0000 7f6d71794740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b0e100 0x560a18b91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.173+0000 7f6d71794740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b2d70 con 0x560a18b0e100 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.173+0000 7f6d6d980640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b01850 0x560a18b12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d71794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b41e0 con 0x560a18b02170 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d71794740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681b3440 con 0x560a18b01850 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d6c97e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60920/0 (socket says 172.21.15.60:60920) 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1535915577 learned_addr learned my addr 172.21.15.60:0/1535915577 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d6d980640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b0e100 0x560a18b91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.174+0000 7f6d71794740 10 client.0 _ll_register_callbacks cb 0x7f6d58004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.176+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b01850 msgr2=0x560a18b12d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.176+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b01850 0x560a18b12d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.176+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1535915577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b0e100 msgr2=0x560a18b91d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.176+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b0e100 0x560a18b91d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.176+0000 7f6d6c97e640 1 -- 172.21.15.60:0/1535915577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a18b92410 con 0x560a18b02170 2024-07-24T21:57:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.177+0000 7f6d6c97e640 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005f30 tx=0x7f6d5800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcb3658e58e9512 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.177+0000 7f6d50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.178+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802f070 con 0x560a18b02170 2024-07-24T21:57:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.178+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d58015070 con 0x560a18b02170 2024-07-24T21:57:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.178+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f6d58005030 con 0x560a18b02170 2024-07-24T21:57:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.179+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6d58005230 con 0x560a18b02170 2024-07-24T21:57:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.179+0000 7f6d71794740 10 client.27131 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.179+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560a18b997b0 con 0x560a18b02170 2024-07-24T21:57:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.179+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58015070 con 0x560a18b02170 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6d58011440 con 0x560a18b02170 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 10 client.27131 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 10 client.27131 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 20 client.27131 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 4 client.27131 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:57:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.180+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a18a763a0 con 0x560a18b1e010 2024-07-24T21:57:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.181+0000 7f6d6d980640 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.187+0000 7f6d6d980640 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b9c000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40dc3040ec583d45 server_cookie=c999d8a78059b633 in_seq=0 out_seq=0 2024-07-24T21:57:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.188+0000 7f6d50ff9640 10 client.27131 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:57:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.191+0000 7f6d50ff9640 1 -- 172.21.15.60:0/1535915577 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a18a763a0 con 0x560a18b1e010 2024-07-24T21:57:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.191+0000 7f6d50ff9640 10 client.27131 handle_command_reply: tid=0 2024-07-24T21:57:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.191+0000 7f6d71794740 1 client.27131 shutdown 2024-07-24T21:57:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 msgr2=0x560a18b132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005f30 tx=0x7f6d5800b390 comp rx=0 tx=0).stop 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 shutdown_connections 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x560a18b1e010 0x560a18b9c000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a18b0e100 0x560a18b91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a18b02170 0x560a18b132c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 --2- 172.21.15.60:0/1535915577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a18b01850 0x560a18b12d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 >> 172.21.15.60:0/1535915577 conn(0x560a18aff680 msgr2=0x560a18b0ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.192+0000 7f6d50ff9640 0 client.27131 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:57:29.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.193+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 shutdown_connections 2024-07-24T21:57:29.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.193+0000 7f6d71794740 1 -- 172.21.15.60:0/1535915577 wait complete. 2024-07-24T21:57:29.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.193+0000 7f6d71794740 20 client.27131 trim_cache size 0 max 16384 2024-07-24T21:57:29.197 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.195+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 msgr2=0x7f6d540b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 0x7f6d540b6320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d64101d70 tx=0x7f6d64005c90 comp rx=0 tx=0).stop 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 msgr2=0x7f6d6819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6d640306b0 tx=0x7f6d64033e20 comp rx=0 tx=0).stop 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 shutdown_connections 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d540b3e60 0x7f6d540b6320 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d68113330 0x7f6d681a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d68075850 0x7f6d6819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 --2- 172.21.15.60:0/493596785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d68074f30 0x7f6d6819efe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.196+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 >> 172.21.15.60:0/493596785 conn(0x7f6d680fe060 msgr2=0x7f6d680fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.197+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 shutdown_connections 2024-07-24T21:57:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:29.197+0000 7f6d6e181640 1 -- 172.21.15.60:0/493596785 wait complete. 2024-07-24T21:57:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:29 smithi123 ceph-mon[37143]: pgmap v712: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-07-24T21:57:30.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:29 smithi060 ceph-mon[33000]: pgmap v712: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-07-24T21:57:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:29 smithi086 ceph-mon[37128]: pgmap v712: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-07-24T21:57:30.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:57:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:31 smithi123 ceph-mon[37143]: pgmap v713: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-07-24T21:57:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:31 smithi086 ceph-mon[37128]: pgmap v713: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-07-24T21:57:32.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:31 smithi060 ceph-mon[33000]: pgmap v713: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-07-24T21:57:32.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.496+0000 7f586a847640 1 -- 172.21.15.60:0/4198356608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5864106700 msgr2=0x7f5864106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.496+0000 7f586a847640 1 --2- 172.21.15.60:0/4198356608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5864106700 0x7f5864106b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f585c01acb0 tx=0x7f585c03b920 comp rx=0 tx=0).stop 2024-07-24T21:57:32.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 -- 172.21.15.60:0/4198356608 shutdown_connections 2024-07-24T21:57:32.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 --2- 172.21.15.60:0/4198356608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5864106700 0x7f5864106b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 --2- 172.21.15.60:0/4198356608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 0x7f58641060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 --2- 172.21.15.60:0/4198356608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58641012c0 0x7f58641016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 -- 172.21.15.60:0/4198356608 >> 172.21.15.60:0/4198356608 conn(0x7f58640fcf90 msgr2=0x7f58640ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.499+0000 7f586a847640 1 -- 172.21.15.60:0/4198356608 shutdown_connections 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.500+0000 7f586a847640 1 -- 172.21.15.60:0/4198356608 wait complete. 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.500+0000 7f586a847640 1 Processor -- start 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.500+0000 7f586a847640 1 -- start start 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 0x7f5864073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5864106700 0x7f5864115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5864109ea0 con 0x7f5864101be0 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5864109d20 con 0x7f58641012c0 2024-07-24T21:57:32.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a847640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f586410a020 con 0x7f5864106700 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60936/0 (socket says 172.21.15.60:60936) 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 -- 172.21.15.60:0/2503597148 learned_addr learned my addr 172.21.15.60:0/2503597148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f586a046640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5864106700 0x7f5864115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869044640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 0x7f5864073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 -- 172.21.15.60:0/2503597148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5864106700 msgr2=0x7f5864115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5864106700 0x7f5864115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 -- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 msgr2=0x7f5864073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 0x7f5864073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.501+0000 7f5869845640 1 -- 172.21.15.60:0/2503597148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586410d840 con 0x7f58641012c0 2024-07-24T21:57:32.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.502+0000 7f5869845640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f585400c890 tx=0x7f585400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.503+0000 7f585affd640 1 -- 172.21.15.60:0/2503597148 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854004220 con 0x7f58641012c0 2024-07-24T21:57:32.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.503+0000 7f585affd640 1 -- 172.21.15.60:0/2503597148 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58540043c0 con 0x7f58641012c0 2024-07-24T21:57:32.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.503+0000 7f585affd640 1 -- 172.21.15.60:0/2503597148 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854004a00 con 0x7f58641012c0 2024-07-24T21:57:32.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.504+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586410db30 con 0x7f58641012c0 2024-07-24T21:57:32.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.504+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586410e020 con 0x7f58641012c0 2024-07-24T21:57:32.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 Processor -- start 2024-07-24T21:57:32.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 -- start start 2024-07-24T21:57:32.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e045dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0469bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5629e03d49b0 con 0x5629e045d850 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629e03c3eb0 con 0x5629e046a100 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.508+0000 7f586de5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629e03c7ef0 con 0x5629e045e170 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60938/0 (socket says 172.21.15.60:60938) 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 -- 172.21.15.60:0/2735808802 learned_addr learned my addr 172.21.15.60:0/2735808802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 -- 172.21.15.60:0/2735808802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 msgr2=0x5629e0469bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0469bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 -- 172.21.15.60:0/2735808802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 msgr2=0x5629e045dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e045dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 -- 172.21.15.60:0/2735808802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e034d320 con 0x5629e046a100 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.509+0000 7f5869044640 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5860009c10 tx=0x7f5860011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5fa08258c8918c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.510+0000 7f584ffff640 1 -- 172.21.15.60:0/2735808802 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004020 con 0x5629e046a100 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.510+0000 7f584ffff640 1 -- 172.21.15.60:0/2735808802 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600054d0 con 0x5629e046a100 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.510+0000 7f584ffff640 1 -- 172.21.15.60:0/2735808802 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004cb0 con 0x5629e046a100 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.511+0000 7f585affd640 1 -- 172.21.15.60:0/2503597148 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f58540079b0 con 0x7f58641012c0 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.511+0000 7f586de5a740 1 -- 172.21.15.60:0/2735808802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 msgr2=0x5629e046c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.511+0000 7f586de5a740 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5860009c10 tx=0x7f5860011810 comp rx=0 tx=0).stop 2024-07-24T21:57:32.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.512+0000 7f585affd640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 0x7f58500b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.512+0000 7f585affd640 1 -- 172.21.15.60:0/2503597148 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f58540e3fe0 con 0x7f58641012c0 2024-07-24T21:57:32.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.513+0000 7f586de5a740 1 -- 172.21.15.60:0/2735808802 shutdown_connections 2024-07-24T21:57:32.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.513+0000 7f586de5a740 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e046c4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.513+0000 7f586de5a740 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0469bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.513+0000 7f586de5a740 1 --2- 172.21.15.60:0/2735808802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e045dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.513+0000 7f586de5a740 1 -- 172.21.15.60:0/2735808802 >> 172.21.15.60:0/2735808802 conn(0x5629e045b680 msgr2=0x5629e045c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.514+0000 7f586de5a740 1 -- 172.21.15.60:0/2735808802 shutdown_connections 2024-07-24T21:57:32.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.514+0000 7f586de5a740 1 -- 172.21.15.60:0/2735808802 wait complete. 2024-07-24T21:57:32.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.514+0000 7f586a046640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 0x7f58500b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.515+0000 7f586de5a740 1 -- start start 2024-07-24T21:57:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.515+0000 7f586a046640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 0x7f58500b64d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f585c006010 tx=0x7f585c01a810 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.515+0000 7f586de5a740 1 Processor -- start 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e04670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0467610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5629e0227700 con 0x5629e045d850 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629e03665f0 con 0x5629e046a100 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629e046cd80 con 0x5629e045e170 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586a046640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586a046640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60946/0 (socket says 172.21.15.60:60946) 2024-07-24T21:57:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586a046640 1 -- 172.21.15.60:0/2168916851 learned_addr learned my addr 172.21.15.60:0/2168916851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f5869044640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0467610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f5869845640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e04670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586de5a740 10 client.0 _ll_register_callbacks cb 0x5629e0386960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586a046640 1 -- 172.21.15.60:0/2168916851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 msgr2=0x5629e0467610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.516+0000 7f586a046640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0467610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f586a046640 1 -- 172.21.15.60:0/2168916851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 msgr2=0x5629e04670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f586a046640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e04670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f586a046640 1 -- 172.21.15.60:0/2168916851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e0359680 con 0x5629e046a100 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f5869044640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0467610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f586a046640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f585c003dc0 tx=0x7f585c03b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ff73c6be19677d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f584d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c04e070 con 0x5629e046a100 2024-07-24T21:57:32.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.517+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f585c041620 con 0x5629e046a100 2024-07-24T21:57:32.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.521+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58600054d0 con 0x5629e046a100 2024-07-24T21:57:32.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.523+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f585c058050 con 0x5629e046a100 2024-07-24T21:57:32.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.524+0000 7f586de5a740 10 client.27143 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:32.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.524+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629e0124650 con 0x5629e046a100 2024-07-24T21:57:32.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600054d0 con 0x5629e046a100 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f585c0043e0 con 0x5629e046a100 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 10 client.27143 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 10 client.27143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 20 client.27143 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e0479d30 0x5629e047c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 4 client.27143 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:32.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629e03d6900 con 0x5629e0479d30 2024-07-24T21:57:32.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.525+0000 7f5869845640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e0479d30 0x5629e047c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.526+0000 7f584d7fa640 10 client.27143 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.526+0000 7f5869845640 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e0479d30 0x5629e047c140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc84a77c600fee0d server_cookie=549b5c28f512fc3c in_seq=0 out_seq=0 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f584d7fa640 1 -- 172.21.15.60:0/2168916851 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629e03d6900 con 0x5629e0479d30 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f584d7fa640 10 client.27143 handle_command_reply: tid=0 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 client.27143 shutdown 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 msgr2=0x5629e0467b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f585c003dc0 tx=0x7f585c03b920 comp rx=0 tx=0).stop 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 shutdown_connections 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e0479d30 0x5629e047c140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e046a100 0x5629e0467b50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045e170 0x5629e0467610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 --2- 172.21.15.60:0/2168916851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e045d850 0x5629e04670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 >> 172.21.15.60:0/2168916851 conn(0x5629e045b680 msgr2=0x5629e0461220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f584d7fa640 0 client.27143 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.527+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 shutdown_connections 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.528+0000 7f586de5a740 1 -- 172.21.15.60:0/2168916851 wait complete. 2024-07-24T21:57:32.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.528+0000 7f586de5a740 20 client.27143 trim_cache size 0 max 16384 2024-07-24T21:57:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.550+0000 7f586de5a740 1 Processor -- start 2024-07-24T21:57:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.550+0000 7f586de5a740 1 -- start start 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045d850 0x5629e04565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e045e170 0x5629e0481a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d9000 con 0x5629e046a100 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d6810 con 0x5629e045e170 2024-07-24T21:57:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586de5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d4680 con 0x5629e045d850 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586a046640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586a046640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33858/0 (socket says 172.21.15.60:33858) 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.551+0000 7f586a046640 1 -- 172.21.15.60:0/1508464097 learned_addr learned my addr 172.21.15.60:0/1508464097 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 -- 172.21.15.60:0/1508464097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045d850 msgr2=0x5629e04565d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045d850 0x5629e04565d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 -- 172.21.15.60:0/1508464097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e045e170 msgr2=0x5629e0481a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e045e170 0x5629e0481a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 -- 172.21.15.60:0/1508464097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e04655d0 con 0x5629e046a100 2024-07-24T21:57:32.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.552+0000 7f586a046640 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f585c004820 tx=0x7f585c04a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24854b6b442036b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.553+0000 7f584f7fe640 1 -- 172.21.15.60:0/1508464097 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c04e040 con 0x5629e046a100 2024-07-24T21:57:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.553+0000 7f584f7fe640 1 -- 172.21.15.60:0/1508464097 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c04a800 con 0x5629e046a100 2024-07-24T21:57:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.553+0000 7f584f7fe640 1 -- 172.21.15.60:0/1508464097 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c045050 con 0x5629e046a100 2024-07-24T21:57:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.554+0000 7f586de5a740 1 -- 172.21.15.60:0/1508464097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 msgr2=0x5629e0481f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.554+0000 7f586de5a740 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f585c004820 tx=0x7f585c04a760 comp rx=0 tx=0).stop 2024-07-24T21:57:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.555+0000 7f586de5a740 1 -- 172.21.15.60:0/1508464097 shutdown_connections 2024-07-24T21:57:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.555+0000 7f586de5a740 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0481f90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.555+0000 7f586de5a740 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e045e170 0x5629e0481a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.555+0000 7f586de5a740 1 --2- 172.21.15.60:0/1508464097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e045d850 0x5629e04565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.555+0000 7f586de5a740 1 -- 172.21.15.60:0/1508464097 >> 172.21.15.60:0/1508464097 conn(0x5629e045b680 msgr2=0x5629e0468f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.557+0000 7f586de5a740 1 -- 172.21.15.60:0/1508464097 shutdown_connections 2024-07-24T21:57:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.557+0000 7f586de5a740 1 -- 172.21.15.60:0/1508464097 wait complete. 2024-07-24T21:57:32.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.557+0000 7f586de5a740 1 -- start start 2024-07-24T21:57:32.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.558+0000 7f586de5a740 1 Processor -- start 2024-07-24T21:57:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0472840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e04649a0 0x5629e04e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d4cd0 con 0x5629e046a100 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d6140 con 0x5629e04649a0 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f5869044640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f5869044640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52360/0 (socket says 172.21.15.60:52360) 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f5869044640 1 -- 172.21.15.60:0/1073082385 learned_addr learned my addr 172.21.15.60:0/1073082385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586a046640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e04649a0 0x5629e04e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58641d53a0 con 0x5629e0463fd0 2024-07-24T21:57:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.559+0000 7f586de5a740 10 client.0 _ll_register_callbacks cb 0x7f585c012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.561+0000 7f5869044640 1 -- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e04649a0 msgr2=0x5629e04e3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.561+0000 7f5869044640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e04649a0 0x5629e04e3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.561+0000 7f5869044640 1 -- 172.21.15.60:0/1073082385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 msgr2=0x5629e0472840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.561+0000 7f5869044640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0472840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.561+0000 7f5869044640 1 -- 172.21.15.60:0/1073082385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e04e4060 con 0x5629e0463fd0 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.562+0000 7f5869044640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5860011d10 tx=0x7f5860019c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6830c45312b92ef server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.562+0000 7f584d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:57:32.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.562+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860027020 con 0x5629e0463fd0 2024-07-24T21:57:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.563+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f5860019d80 con 0x5629e0463fd0 2024-07-24T21:57:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.563+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c04a800 con 0x5629e0463fd0 2024-07-24T21:57:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.564+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f586001d040 con 0x5629e0463fd0 2024-07-24T21:57:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.564+0000 7f586de5a740 10 client.27118 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.564+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629e04e53c0 con 0x5629e0463fd0 2024-07-24T21:57:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.564+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c04a800 con 0x5629e0463fd0 2024-07-24T21:57:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.565+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5860004a60 con 0x5629e0463fd0 2024-07-24T21:57:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.565+0000 7f586de5a740 10 client.27118 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.565+0000 7f586de5a740 10 client.27118 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:57:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.565+0000 7f586de5a740 20 client.27118 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e04e59a0 0x5629e04e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f586de5a740 4 client.27118 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629e0456da0 con 0x5629e04e59a0 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f5869845640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e04e59a0 0x5629e04e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f5869845640 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e04e59a0 0x5629e04e7d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e849d3bfecb635e8 server_cookie=9c48f1ed00e4a15c in_seq=0 out_seq=0 2024-07-24T21:57:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.566+0000 7f584d7fa640 10 client.27118 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:57:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.567+0000 7f584d7fa640 1 -- 172.21.15.60:0/1073082385 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629e0456da0 con 0x5629e04e59a0 2024-07-24T21:57:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.567+0000 7f584d7fa640 10 client.27118 handle_command_reply: tid=0 2024-07-24T21:57:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.567+0000 7f586de5a740 1 client.27118 shutdown 2024-07-24T21:57:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 msgr2=0x5629e0464410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5860011d10 tx=0x7f5860019c10 comp rx=0 tx=0).stop 2024-07-24T21:57:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 shutdown_connections 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5629e04e59a0 0x5629e04e7d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629e04649a0 0x5629e04e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.568+0000 7f584d7fa640 0 client.27118 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629e0463fd0 0x5629e0464410 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 1 --2- 172.21.15.60:0/1073082385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629e046a100 0x5629e0472840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 >> 172.21.15.60:0/1073082385 conn(0x5629e045b680 msgr2=0x5629e0459eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 shutdown_connections 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 1 -- 172.21.15.60:0/1073082385 wait complete. 2024-07-24T21:57:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.569+0000 7f586de5a740 20 client.27118 trim_cache size 0 max 16384 2024-07-24T21:57:32.572 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:57:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.570+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 msgr2=0x7f58500b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.570+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 0x7f58500b64d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f585c006010 tx=0x7f585c01a810 comp rx=0 tx=0).stop 2024-07-24T21:57:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.570+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 msgr2=0x7f58640735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.570+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f585400c890 tx=0x7f585400cd50 comp rx=0 tx=0).stop 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 shutdown_connections 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f58500b4010 0x7f58500b64d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5864106700 0x7f5864115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5864101be0 0x7f5864073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 --2- 172.21.15.60:0/2503597148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58641012c0 0x7f58640735a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 >> 172.21.15.60:0/2503597148 conn(0x7f58640fcf90 msgr2=0x7f5864104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.571+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 shutdown_connections 2024-07-24T21:57:32.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:32.572+0000 7f586a847640 1 -- 172.21.15.60:0/2503597148 wait complete. 2024-07-24T21:57:33.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:57:33.706 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:33.990 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:33 smithi060 ceph-mon[33000]: pgmap v714: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 331 op/s 2024-07-24T21:57:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:33 smithi123 ceph-mon[37143]: pgmap v714: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 331 op/s 2024-07-24T21:57:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:33 smithi086 ceph-mon[37128]: pgmap v714: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 331 op/s 2024-07-24T21:57:35.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 -- 172.21.15.60:0/2794845756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04113330 msgr2=0x7fdf04115720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 --2- 172.21.15.60:0/2794845756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04113330 0x7fdf04115720 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdef401a910 tx=0x7fdef403b430 comp rx=0 tx=0).stop 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 -- 172.21.15.60:0/2794845756 shutdown_connections 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 --2- 172.21.15.60:0/2794845756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04113330 0x7fdf04115720 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 --2- 172.21.15.60:0/2794845756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04075850 0x7fdf04112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 --2- 172.21.15.60:0/2794845756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf04075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.460+0000 7fdf0359e640 1 -- 172.21.15.60:0/2794845756 >> 172.21.15.60:0/2794845756 conn(0x7fdf040fe060 msgr2=0x7fdf04100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.461+0000 7fdf0359e640 1 -- 172.21.15.60:0/2794845756 shutdown_connections 2024-07-24T21:57:35.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.461+0000 7fdf0359e640 1 -- 172.21.15.60:0/2794845756 wait complete. 2024-07-24T21:57:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 Processor -- start 2024-07-24T21:57:35.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 -- start start 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 0x7fdf0410eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04113330 0x7fdf0419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04108e00 con 0x7fdf04075850 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04108c80 con 0x7fdf04113330 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.462+0000 7fdf0359e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04108f80 con 0x7fdf04074f30 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52376/0 (socket says 172.21.15.60:52376) 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 -- 172.21.15.60:0/4086572576 learned_addr learned my addr 172.21.15.60:0/4086572576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04113330 0x7fdf0419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 0x7fdf0410eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 -- 172.21.15.60:0/4086572576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04113330 msgr2=0x7fdf0419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04113330 0x7fdf0419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 -- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 msgr2=0x7fdf0410eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 0x7fdf0410eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf0259c640 1 -- 172.21.15.60:0/4086572576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0419c8d0 con 0x7fdf04074f30 2024-07-24T21:57:35.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.463+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 0x7fdf0410eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.464+0000 7fdf0259c640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00cce0 tx=0x7fdefc00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.465+0000 7fdef37fe640 1 -- 172.21.15.60:0/4086572576 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc004b20 con 0x7fdf04074f30 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.465+0000 7fdef37fe640 1 -- 172.21.15.60:0/4086572576 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc004cc0 con 0x7fdf04074f30 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.465+0000 7fdef37fe640 1 -- 172.21.15.60:0/4086572576 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00f6b0 con 0x7fdf04074f30 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.465+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0419cbc0 con 0x7fdf04074f30 2024-07-24T21:57:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.465+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0406c560 con 0x7fdf04074f30 2024-07-24T21:57:35.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.469+0000 7fdf0ad71740 1 Processor -- start 2024-07-24T21:57:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 -- start start 2024-07-24T21:57:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba05efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555ba055a9b0 con 0x555ba05f0100 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ba0549eb0 con 0x555ba05e4170 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ba054def0 con 0x555ba05e3850 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52386/0 (socket says 172.21.15.60:52386) 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.470+0000 7fdf02d9d640 1 -- 172.21.15.60:0/1481477117 learned_addr learned my addr 172.21.15.60:0/1481477117 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.471+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.471+0000 7fdef37fe640 1 -- 172.21.15.60:0/4086572576 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fdefc00f850 con 0x7fdf04074f30 2024-07-24T21:57:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.471+0000 7fdf0259c640 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba05efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 -- 172.21.15.60:0/1481477117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 msgr2=0x555ba05efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba05efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 -- 172.21.15.60:0/1481477117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 msgr2=0x555ba05f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 -- 172.21.15.60:0/1481477117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba04d3320 con 0x555ba05e3850 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdef401a910 tx=0x7fdef4004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd141088ac4655dd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.472+0000 7fdef37fe640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 0x7fdeec0b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.473+0000 7fdef0ff9640 1 -- 172.21.15.60:0/1481477117 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4046030 con 0x555ba05e3850 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.473+0000 7fdf0259c640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 0x7fdeec0b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.473+0000 7fdef37fe640 1 -- 172.21.15.60:0/4086572576 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7fdefc0e3ff0 con 0x7fdf04074f30 2024-07-24T21:57:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.473+0000 7fdef0ff9640 1 -- 172.21.15.60:0/1481477117 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef4004b00 con 0x555ba05e3850 2024-07-24T21:57:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.473+0000 7fdef0ff9640 1 -- 172.21.15.60:0/1481477117 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4040b70 con 0x555ba05e3850 2024-07-24T21:57:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.474+0000 7fdf0259c640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 0x7fdeec0b64d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555ba05e2560 tx=0x7fdefc0cd040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.476+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1481477117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 msgr2=0x555ba05e3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.476+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdef401a910 tx=0x7fdef4004990 comp rx=0 tx=0).stop 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1481477117 shutdown_connections 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba05efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1481477117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e3850 0x555ba05e3c30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1481477117 >> 172.21.15.60:0/1481477117 conn(0x555ba05e1680 msgr2=0x555ba05e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1481477117 shutdown_connections 2024-07-24T21:57:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.477+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1481477117 wait complete. 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 -- start start 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 Processor -- start 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 0x555ba05ed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555ba03ad700 con 0x555ba05e3850 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ba04ec5f0 con 0x555ba05f0100 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ba05f2d20 con 0x555ba05e4170 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.478+0000 7fdf0ad71740 10 client.0 _ll_register_callbacks cb 0x555ba050c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.479+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.479+0000 7fdf01d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.479+0000 7fdf01d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05ed040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52390/0 (socket says 172.21.15.60:52390) 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.479+0000 7fdf01d9b640 1 -- 172.21.15.60:0/2203568796 learned_addr learned my addr 172.21.15.60:0/2203568796 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.479+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 0x555ba05ed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 msgr2=0x555ba05ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2203568796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 msgr2=0x555ba05efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2203568796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba04df680 con 0x555ba05f0100 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 0x555ba05ed7d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdef401a6d0 tx=0x7fdef4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76e67f4c51fd94a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.480+0000 7fdee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.481+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4046070 con 0x555ba05f0100 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.481+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7fdef4019040 con 0x555ba05f0100 2024-07-24T21:57:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.481+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef4004b00 con 0x555ba05f0100 2024-07-24T21:57:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.483+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdef4040b60 con 0x555ba05f0100 2024-07-24T21:57:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.483+0000 7fdf0ad71740 10 client.27158 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.483+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555ba02aa650 con 0x555ba05f0100 2024-07-24T21:57:35.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.483+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef4004b00 con 0x555ba05f0100 2024-07-24T21:57:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fdef404d560 con 0x555ba05f0100 2024-07-24T21:57:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 10 client.27158 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 20 client.27158 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba05ffb10 0x555ba0601f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 4 client.27158 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.484+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ba055c900 con 0x555ba05ffb10 2024-07-24T21:57:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.486+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba05ffb10 0x555ba0601f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.488+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba05ffb10 0x555ba0601f00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b232c2a4fc4976ac server_cookie=5a49dc4365119939 in_seq=0 out_seq=0 2024-07-24T21:57:35.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.488+0000 7fdee9ffb640 10 client.27158 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.490+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2203568796 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ba055c900 con 0x555ba05ffb10 2024-07-24T21:57:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.490+0000 7fdee9ffb640 10 client.27158 handle_command_reply: tid=0 2024-07-24T21:57:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.490+0000 7fdf0ad71740 1 client.27158 shutdown 2024-07-24T21:57:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 msgr2=0x555ba05ed7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 0x555ba05ed7d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdef401a6d0 tx=0x7fdef4005fd0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 shutdown_connections 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba05ffb10 0x555ba0601f00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05f0100 0x555ba05ed7d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2203568796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 >> 172.21.15.60:0/2203568796 conn(0x555ba05e1680 msgr2=0x555ba05e7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdee9ffb640 0 client.27158 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:35.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.491+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 shutdown_connections 2024-07-24T21:57:35.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.492+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2203568796 wait complete. 2024-07-24T21:57:35.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.492+0000 7fdf0ad71740 20 client.27158 trim_cache size 0 max 16384 2024-07-24T21:57:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.519+0000 7fdf0ad71740 1 Processor -- start 2024-07-24T21:57:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.520+0000 7fdf0ad71740 1 -- start start 2024-07-24T21:57:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba0607810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05f0100 0x555ba0607d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b6e40 con 0x555ba05e3850 2024-07-24T21:57:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b4650 con 0x555ba05e4170 2024-07-24T21:57:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04075350 con 0x555ba05f0100 2024-07-24T21:57:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33900/0 (socket says 172.21.15.60:33900) 2024-07-24T21:57:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf02d9d640 1 -- 172.21.15.60:0/1222542693 learned_addr learned my addr 172.21.15.60:0/1222542693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05f0100 0x555ba0607d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba0607810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 -- 172.21.15.60:0/1222542693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05f0100 msgr2=0x555ba0607d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05f0100 0x555ba0607d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 -- 172.21.15.60:0/1222542693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 msgr2=0x555ba05e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.521+0000 7fdf01d9b640 1 -- 172.21.15.60:0/1222542693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba05ed3b0 con 0x555ba05e4170 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.522+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba0607810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x555ba05ef8f0 tx=0x7fdef8014840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2466a3920b5fa6d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.522+0000 7fdeebfff640 1 -- 172.21.15.60:0/1222542693 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004030 con 0x555ba05e4170 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.522+0000 7fdeebfff640 1 -- 172.21.15.60:0/1222542693 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef80035f0 con 0x555ba05e4170 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.522+0000 7fdeebfff640 1 -- 172.21.15.60:0/1222542693 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef801d6c0 con 0x555ba05e4170 2024-07-24T21:57:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1222542693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 msgr2=0x555ba0607810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba0607810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x555ba05ef8f0 tx=0x7fdef8014840 comp rx=0 tx=0).stop 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1222542693 shutdown_connections 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05f0100 0x555ba0607d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e4170 0x555ba0607810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/1222542693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05e3850 0x555ba05e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.523+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1222542693 >> 172.21.15.60:0/1222542693 conn(0x555ba05e1680 msgr2=0x555ba05eee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.524+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1222542693 shutdown_connections 2024-07-24T21:57:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.524+0000 7fdf0ad71740 1 -- 172.21.15.60:0/1222542693 wait complete. 2024-07-24T21:57:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.525+0000 7fdf0ad71740 1 -- start start 2024-07-24T21:57:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.525+0000 7fdf0ad71740 1 Processor -- start 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04073620 con 0x555ba05f0100 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04074a70 con 0x555ba05e3850 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04073cd0 con 0x555ba05e4170 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf0ad71740 10 client.0 _ll_register_callbacks cb 0x7fdef4004d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf02d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:32804/0 (socket says 172.21.15.60:32804) 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2474355019 learned_addr learned my addr 172.21.15.60:0/2474355019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.527+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 msgr2=0x555ba05f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2474355019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 msgr2=0x555ba05f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 -- 172.21.15.60:0/2474355019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba05f1b60 con 0x555ba05e3850 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf02d9d640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdef403b470 tx=0x7fdef40406b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd382aaff0173628 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4046030 con 0x555ba05e3850 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7fdef4019040 con 0x555ba05e3850 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.528+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef80035f0 con 0x555ba05e3850 2024-07-24T21:57:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdef404dd70 con 0x555ba05e3850 2024-07-24T21:57:35.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdf0ad71740 10 client.27164 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:35.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555ba066a7e0 con 0x555ba05e3850 2024-07-24T21:57:35.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef80035f0 con 0x555ba05e3850 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fdef406f440 con 0x555ba05e3850 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdf0ad71740 10 client.27164 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.529+0000 7fdf0ad71740 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.530+0000 7fdf0ad71740 20 client.27164 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.530+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba066aae0 0x555ba066cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.530+0000 7fdf0ad71740 4 client.27164 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.530+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ba05583a0 con 0x555ba066aae0 2024-07-24T21:57:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.531+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba066aae0 0x555ba066cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.533+0000 7fdf01d9b640 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba066aae0 0x555ba066cf00 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4994d003eb777f3b server_cookie=3281dda20df4adef in_seq=0 out_seq=0 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.533+0000 7fdee9ffb640 10 client.27164 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.533+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2474355019 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ba05583a0 con 0x555ba066aae0 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.533+0000 7fdee9ffb640 10 client.27164 handle_command_reply: tid=0 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.533+0000 7fdf0ad71740 1 client.27164 shutdown 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 msgr2=0x555ba05f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdef403b470 tx=0x7fdef40406b0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 shutdown_connections 2024-07-24T21:57:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555ba066aae0 0x555ba066cf00 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ba05f0100 0x555ba05f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba05e4170 0x555ba05f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 --2- 172.21.15.60:0/2474355019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba05e3850 0x555ba05f84d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 >> 172.21.15.60:0/2474355019 conn(0x555ba05e1680 msgr2=0x555ba05e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdee9ffb640 0 client.27164 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 shutdown_connections 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 1 -- 172.21.15.60:0/2474355019 wait complete. 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.534+0000 7fdf0ad71740 20 client.27164 trim_cache size 0 max 16384 2024-07-24T21:57:35.537 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.535+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 msgr2=0x7fdeec0b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.535+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 0x7fdeec0b64d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555ba05e2560 tx=0x7fdefc0cd040 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.535+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 msgr2=0x7fdf0410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.535+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00cce0 tx=0x7fdefc00a620 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 shutdown_connections 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdeec0b4010 0x7fdeec0b64d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf04113330 0x7fdf0419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf04075850 0x7fdf0410eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 --2- 172.21.15.60:0/4086572576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf04074f30 0x7fdf0410eab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 >> 172.21.15.60:0/4086572576 conn(0x7fdf040fe060 msgr2=0x7fdf040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 shutdown_connections 2024-07-24T21:57:35.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:35.536+0000 7fdf0359e640 1 -- 172.21.15.60:0/4086572576 wait complete. 2024-07-24T21:57:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:35 smithi060 ceph-mon[33000]: pgmap v715: 97 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 322 op/s 2024-07-24T21:57:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:35 smithi123 ceph-mon[37143]: pgmap v715: 97 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 322 op/s 2024-07-24T21:57:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:35 smithi086 ceph-mon[37128]: pgmap v715: 97 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 322 op/s 2024-07-24T21:57:37.320 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:57:37.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:57:37.688 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:37.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:37 smithi060 ceph-mon[33000]: pgmap v716: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-07-24T21:57:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:37 smithi123 ceph-mon[37143]: pgmap v716: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-07-24T21:57:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:37 smithi086 ceph-mon[37128]: pgmap v716: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.424+0000 7f429812c640 1 -- 172.21.15.60:0/1910222674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 msgr2=0x7f429010c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.424+0000 7f429812c640 1 --2- 172.21.15.60:0/1910222674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f429010c450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4280009900 tx=0x7f428002a3f0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 -- 172.21.15.60:0/1910222674 shutdown_connections 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 --2- 172.21.15.60:0/1910222674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 0x7f429010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 --2- 172.21.15.60:0/1910222674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f429010c450 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 --2- 172.21.15.60:0/1910222674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f4290103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 -- 172.21.15.60:0/1910222674 >> 172.21.15.60:0/1910222674 conn(0x7f42900782e0 msgr2=0x7f42900ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 -- 172.21.15.60:0/1910222674 shutdown_connections 2024-07-24T21:57:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.425+0000 7f429812c640 1 -- 172.21.15.60:0/1910222674 wait complete. 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 Processor -- start 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 -- start start 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f42901012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 0x7f4290101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42901126f0 con 0x7f4290103e70 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4290112570 con 0x7f4290103470 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.426+0000 7f429812c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4290112870 con 0x7f429010ca20 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58756/0 (socket says 172.21.15.60:58756) 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f4295ec8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f42901012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 -- 172.21.15.60:0/2588212893 learned_addr learned my addr 172.21.15.60:0/2588212893 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42966c9640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 0x7f4290101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 -- 172.21.15.60:0/2588212893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 msgr2=0x7f4290101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 0x7f4290101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 -- 172.21.15.60:0/2588212893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 msgr2=0x7f42901012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f42901012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f42956c7640 1 -- 172.21.15.60:0/2588212893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42901a7020 con 0x7f4290103e70 2024-07-24T21:57:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.427+0000 7f4295ec8640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f42901012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f42956c7640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4280005ab0 tx=0x7f4280003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f4286ffd640 1 -- 172.21.15.60:0/2588212893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428003f070 con 0x7f4290103e70 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f4286ffd640 1 -- 172.21.15.60:0/2588212893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4280031d60 con 0x7f4290103e70 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42901a72b0 con 0x7f4290103e70 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f4286ffd640 1 -- 172.21.15.60:0/2588212893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280035030 con 0x7f4290103e70 2024-07-24T21:57:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.428+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42901a7870 con 0x7f4290103e70 2024-07-24T21:57:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.431+0000 7f429ae81740 1 Processor -- start 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 -- start start 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def881e0 0x55e2def9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2def9ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e2deefd280 con 0x55e2def878c0 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2deeec780 con 0x55e2def881e0 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f429ae81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2deef07c0 con 0x55e2def9c980 2024-07-24T21:57:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f42966c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f42966c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58762/0 (socket says 172.21.15.60:58762) 2024-07-24T21:57:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f42966c9640 1 -- 172.21.15.60:0/3165031484 learned_addr learned my addr 172.21.15.60:0/3165031484 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.432+0000 7f4295ec8640 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def881e0 0x55e2def9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 -- 172.21.15.60:0/3165031484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 msgr2=0x55e2def9ed70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2def9ed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 -- 172.21.15.60:0/3165031484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def881e0 msgr2=0x55e2def9c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def881e0 0x55e2def9c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 -- 172.21.15.60:0/3165031484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2dec50790 con 0x55e2def878c0 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f42966c9640 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f428800a430 tx=0x7f4288009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b451747fce2bf327 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f427bfff640 1 -- 172.21.15.60:0/3165031484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288013790 con 0x55e2def878c0 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f4286ffd640 1 -- 172.21.15.60:0/2588212893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4280042070 con 0x7f4290103e70 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f427bfff640 1 -- 172.21.15.60:0/3165031484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288019070 con 0x55e2def878c0 2024-07-24T21:57:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.433+0000 7f427bfff640 1 -- 172.21.15.60:0/3165031484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288014d50 con 0x55e2def878c0 2024-07-24T21:57:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.434+0000 7f429ae81740 1 -- 172.21.15.60:0/3165031484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 msgr2=0x55e2def87ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.434+0000 7f429ae81740 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f428800a430 tx=0x7f4288009650 comp rx=0 tx=0).stop 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f4286ffd640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 0x7f427c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 -- 172.21.15.60:0/3165031484 shutdown_connections 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2def9ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def881e0 0x55e2def9c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 --2- 172.21.15.60:0/3165031484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def87ca0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 -- 172.21.15.60:0/3165031484 >> 172.21.15.60:0/3165031484 conn(0x55e2def856f0 msgr2=0x55e2def86890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f42966c9640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 0x7f427c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f4286ffd640 1 -- 172.21.15.60:0/2588212893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f42801047c0 con 0x7f4290103e70 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 -- 172.21.15.60:0/3165031484 shutdown_connections 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.435+0000 7f429ae81740 1 -- 172.21.15.60:0/3165031484 wait complete. 2024-07-24T21:57:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.436+0000 7f42966c9640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 0x7f427c0b6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f428800a400 tx=0x7f428800af90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.436+0000 7f429ae81740 1 -- start start 2024-07-24T21:57:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.436+0000 7f429ae81740 1 Processor -- start 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.436+0000 7f429ae81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2def91be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def9be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e2dee26ab0 con 0x55e2def881e0 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2dee749e0 con 0x55e2def9c980 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2def8dfa0 con 0x55e2def878c0 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f429ae81740 10 client.0 _ll_register_callbacks cb 0x55e2debfca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52218/0 (socket says 172.21.15.60:52218) 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 -- 172.21.15.60:0/3632709593 learned_addr learned my addr 172.21.15.60:0/3632709593 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f42966c9640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def9be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f42956c7640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2def91be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 -- 172.21.15.60:0/3632709593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 msgr2=0x55e2def9be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def9be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 -- 172.21.15.60:0/3632709593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 msgr2=0x55e2def91be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2def91be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 -- 172.21.15.60:0/3632709593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2decc9620 con 0x55e2def878c0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f4295ec8640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f428c012d20 tx=0x7f428c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b18f58fbae0197c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.437+0000 7f42797fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c012110 con 0x55e2def878c0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f428c004050 con 0x55e2def878c0 2024-07-24T21:57:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2dee9bce0 con 0x55e2def878c0 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f428c0122b0 con 0x55e2def878c0 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f429ae81740 10 client.27136 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.438+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e2def979a0 con 0x55e2def878c0 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e2dee9bce0 con 0x55e2def878c0 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f428c010430 con 0x55e2def878c0 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 10 client.27136 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 10 client.27136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 20 client.27136 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2defb4b00 0x55e2defb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 4 client.27136 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.439+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2deeff1d0 con 0x55e2defb4b00 2024-07-24T21:57:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.440+0000 7f42956c7640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2defb4b00 0x55e2defb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.441+0000 7f42797fa640 10 client.27136 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.441+0000 7f42956c7640 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2defb4b00 0x55e2defb6f60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3471230e0da50523 server_cookie=4e7dada2002f6df0 in_seq=0 out_seq=0 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f42797fa640 1 -- 172.21.15.60:0/3632709593 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2deeff1d0 con 0x55e2defb4b00 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f42797fa640 10 client.27136 handle_command_reply: tid=0 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 client.27136 shutdown 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 msgr2=0x55e2def91660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f428c012d20 tx=0x7f428c00f790 comp rx=0 tx=0).stop 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 shutdown_connections 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2defb4b00 0x55e2defb6f60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def9be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f42797fa640 0 client.27136 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2def91be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 --2- 172.21.15.60:0/3632709593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def878c0 0x55e2def91660 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 >> 172.21.15.60:0/3632709593 conn(0x55e2def856f0 msgr2=0x55e2def8b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 shutdown_connections 2024-07-24T21:57:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.442+0000 7f429ae81740 1 -- 172.21.15.60:0/3632709593 wait complete. 2024-07-24T21:57:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.443+0000 7f429ae81740 20 client.27136 trim_cache size 0 max 16384 2024-07-24T21:57:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.475+0000 7f429ae81740 1 Processor -- start 2024-07-24T21:57:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.475+0000 7f429ae81740 1 -- start start 2024-07-24T21:57:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def881e0 0x55e2def90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42901aea20 con 0x55e2def878c0 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ac230 con 0x55e2def9c980 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f4295ec8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f42966c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f4295ec8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58774/0 (socket says 172.21.15.60:58774) 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f4295ec8640 1 -- 172.21.15.60:0/2559343232 learned_addr learned my addr 172.21.15.60:0/2559343232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.476+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42901aa0c0 con 0x55e2def881e0 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 -- 172.21.15.60:0/2559343232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def881e0 msgr2=0x55e2def90ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def881e0 0x55e2def90ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 -- 172.21.15.60:0/2559343232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 msgr2=0x55e2def86e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 -- 172.21.15.60:0/2559343232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2def88010 con 0x55e2def9c980 2024-07-24T21:57:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f4295ec8640 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.477+0000 7f42966c9640 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def8d120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4288014830 tx=0x7f4288014930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517f0a0a90dda0ba server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.478+0000 7f427b7fe640 1 -- 172.21.15.60:0/2559343232 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288013eb0 con 0x55e2def9c980 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.478+0000 7f427b7fe640 1 -- 172.21.15.60:0/2559343232 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288019040 con 0x55e2def9c980 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.478+0000 7f427b7fe640 1 -- 172.21.15.60:0/2559343232 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428802cc60 con 0x55e2def9c980 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 msgr2=0x55e2def8d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def8d120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4288014830 tx=0x7f4288014930 comp rx=0 tx=0).stop 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 shutdown_connections 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def9c980 0x55e2def8d120 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def881e0 0x55e2def90ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 --2- 172.21.15.60:0/2559343232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def878c0 0x55e2def86e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 >> 172.21.15.60:0/2559343232 conn(0x55e2def856f0 msgr2=0x55e2def8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 shutdown_connections 2024-07-24T21:57:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.479+0000 7f429ae81740 1 -- 172.21.15.60:0/2559343232 wait complete. 2024-07-24T21:57:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.480+0000 7f429ae81740 1 -- start start 2024-07-24T21:57:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.480+0000 7f429ae81740 1 Processor -- start 2024-07-24T21:57:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f429ae81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def878c0 0x55e2defc5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f429ae81740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f429ae81740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2df01d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f429ae81740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42901aa710 con 0x55e2def881e0 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f429ae81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42901abb60 con 0x55e2def878c0 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f42956c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f42956c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58786/0 (socket says 172.21.15.60:58786) 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.481+0000 7f42956c7640 1 -- 172.21.15.60:0/287334214 learned_addr learned my addr 172.21.15.60:0/287334214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42901aadc0 con 0x55e2def9c980 2024-07-24T21:57:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f429ae81740 10 client.0 _ll_register_callbacks cb 0x7f428c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f42956c7640 1 -- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 msgr2=0x55e2df01d670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f42956c7640 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2df01d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f42956c7640 1 -- 172.21.15.60:0/287334214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def878c0 msgr2=0x55e2defc5d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f42956c7640 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def878c0 0x55e2defc5d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.482+0000 7f42956c7640 1 -- 172.21.15.60:0/287334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2df01dcb0 con 0x55e2def881e0 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f42956c7640 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f42800cb350 tx=0x7f4280003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5213b850d1061534 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f42797fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42800e5060 con 0x55e2def881e0 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f42801020d0 con 0x55e2def881e0 2024-07-24T21:57:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4288019040 con 0x55e2def881e0 2024-07-24T21:57:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.483+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f42800ea050 con 0x55e2def881e0 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.484+0000 7f429ae81740 10 client.17760 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.484+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e2df01f020 con 0x55e2def881e0 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.485+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42800e5e50 con 0x55e2def881e0 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.485+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f42800d3f30 con 0x55e2def881e0 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.485+0000 7f429ae81740 10 client.17760 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.485+0000 7f429ae81740 10 client.17760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.485+0000 7f429ae81740 20 client.17760 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.486+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2df01f710 0x55e2df021b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.486+0000 7f429ae81740 4 client.17760 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:57:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.486+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4288019040 con 0x55e2df01f710 2024-07-24T21:57:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.486+0000 7f4295ec8640 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2df01f710 0x55e2df021b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.488+0000 7f4295ec8640 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2df01f710 0x55e2df021b30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ea1cdb627371bd server_cookie=54bd43e8d87739df in_seq=0 out_seq=0 2024-07-24T21:57:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.488+0000 7f42797fa640 10 client.17760 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.517+0000 7f42797fa640 1 -- 172.21.15.60:0/287334214 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4288019040 con 0x55e2df01f710 2024-07-24T21:57:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.517+0000 7f42797fa640 10 client.17760 handle_command_reply: tid=0 2024-07-24T21:57:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.517+0000 7f429ae81740 1 client.17760 shutdown 2024-07-24T21:57:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.517+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 msgr2=0x55e2defc6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.517+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f42800cb350 tx=0x7f4280003f90 comp rx=0 tx=0).stop 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 shutdown_connections 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e2df01f710 0x55e2df021b30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2def9c980 0x55e2df01d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e2def881e0 0x55e2defc6260 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 --2- 172.21.15.60:0/287334214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2def878c0 0x55e2defc5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 >> 172.21.15.60:0/287334214 conn(0x55e2def856f0 msgr2=0x55e2def8d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f42797fa640 0 client.17760 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 shutdown_connections 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.518+0000 7f429ae81740 1 -- 172.21.15.60:0/287334214 wait complete. 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.519+0000 7f429ae81740 20 client.17760 trim_cache size 0 max 16384 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:57:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:57:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.521+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 msgr2=0x7f427c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.521+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 0x7f427c0b6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f428800a400 tx=0x7f428800af90 comp rx=0 tx=0).stop 2024-07-24T21:57:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.521+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 msgr2=0x7f4290101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.521+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4280005ab0 tx=0x7f4280003c80 comp rx=0 tx=0).stop 2024-07-24T21:57:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 shutdown_connections 2024-07-24T21:57:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f427c0b3e40 0x7f427c0b6300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f429010ca20 0x7f4290101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4290103e70 0x7f4290101830 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 --2- 172.21.15.60:0/2588212893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290103470 0x7f42901012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.522+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 >> 172.21.15.60:0/2588212893 conn(0x7f42900782e0 msgr2=0x7f42900ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.524+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 shutdown_connections 2024-07-24T21:57:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:38.524+0000 7f429812c640 1 -- 172.21.15.60:0/2588212893 wait complete. 2024-07-24T21:57:39.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:39 smithi060 ceph-mon[33000]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- 172.21.15.60:0/3041860001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc108180 msgr2=0x7f5efc1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 --2- 172.21.15.60:0/3041860001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc108180 0x7f5efc1085c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009900 tx=0x7f5ef002a410 comp rx=0 tx=0).stop 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- 172.21.15.60:0/3041860001 shutdown_connections 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 --2- 172.21.15.60:0/3041860001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 --2- 172.21.15.60:0/3041860001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc108180 0x7f5efc1085c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 --2- 172.21.15.60:0/3041860001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc102770 0x7f5efc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- 172.21.15.60:0/3041860001 >> 172.21.15.60:0/3041860001 conn(0x7f5efc0fe1c0 msgr2=0x7f5efc1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- 172.21.15.60:0/3041860001 shutdown_connections 2024-07-24T21:57:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- 172.21.15.60:0/3041860001 wait complete. 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 Processor -- start 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.771+0000 7f5f041bf640 1 -- start start 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc108180 0x7f5efc074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112700 con 0x7f5efc102770 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112580 con 0x7f5efc108b00 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f041bf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112880 con 0x7f5efc108180 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58808/0 (socket says 172.21.15.60:58808) 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 -- 172.21.15.60:0/1330521049 learned_addr learned my addr 172.21.15.60:0/1330521049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f0175a640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc108180 0x7f5efc074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 -- 172.21.15.60:0/1330521049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc108180 msgr2=0x7f5efc074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc108180 0x7f5efc074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 -- 172.21.15.60:0/1330521049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 msgr2=0x7f5efc0754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc0754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 -- 172.21.15.60:0/1330521049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc1ab5b0 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc0754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.772+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9f0 tx=0x7f5ef400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.773+0000 7f5efaffd640 1 -- 172.21.15.60:0/1330521049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400cd80 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.773+0000 7f5efaffd640 1 -- 172.21.15.60:0/1330521049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef40108e0 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.773+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1ab780 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.773+0000 7f5efaffd640 1 -- 172.21.15.60:0/1330521049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4018980 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.773+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1abc90 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.775+0000 7f5efaffd640 1 -- 172.21.15.60:0/1330521049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5ef4010430 con 0x7f5efc102770 2024-07-24T21:57:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 Processor -- start 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 -- start start 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d98a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d98cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56463d8f59f0 con 0x56463d97df20 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56463d8e4ef0 con 0x56463d98a7d0 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f06570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56463d8e8f30 con 0x56463d97e840 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f01f5b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d98a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58820/0 (socket says 172.21.15.60:58820) 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f0275c640 1 -- 172.21.15.60:0/3760588691 learned_addr learned my addr 172.21.15.60:0/3760588691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.776+0000 7f5f0175a640 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d98cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 -- 172.21.15.60:0/3760588691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 msgr2=0x56463d98a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d98a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 -- 172.21.15.60:0/3760588691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 msgr2=0x56463d98cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d98cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 -- 172.21.15.60:0/3760588691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d645930 con 0x56463d97df20 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d98a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5efaffd640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 0x7f5ee40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009ea0 tx=0x7f5eec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb13fbec8de9e1a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 0x7f5ee40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5ee3fff640 1 -- 172.21.15.60:0/3760588691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec004d20 con 0x56463d97df20 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5ee3fff640 1 -- 172.21.15.60:0/3760588691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec0054d0 con 0x56463d97df20 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5ee3fff640 1 -- 172.21.15.60:0/3760588691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec011860 con 0x56463d97df20 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.777+0000 7f5efaffd640 1 -- 172.21.15.60:0/1330521049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f5ef40e4410 con 0x7f5efc102770 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 -- 172.21.15.60:0/3760588691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 msgr2=0x56463d97e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f01f5b640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 0x7f5ee40b6200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40e1bb0 tx=0x7f5ef40e7f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009ea0 tx=0x7f5eec009f90 comp rx=0 tx=0).stop 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 -- 172.21.15.60:0/3760588691 shutdown_connections 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d98cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d98a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 --2- 172.21.15.60:0/3760588691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97e300 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 -- 172.21.15.60:0/3760588691 >> 172.21.15.60:0/3760588691 conn(0x56463d97bd50 msgr2=0x56463d97cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 -- 172.21.15.60:0/3760588691 shutdown_connections 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.778+0000 7f5f06570740 1 -- 172.21.15.60:0/3760588691 wait complete. 2024-07-24T21:57:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 -- start start 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 Processor -- start 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d987d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d988240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56463d748890 con 0x56463d97df20 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56463d887390 con 0x56463d98a7d0 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56463d98d450 con 0x56463d97e840 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f06570740 10 client.0 _ll_register_callbacks cb 0x56463d891500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58832/0 (socket says 172.21.15.60:58832) 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f0175a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d987d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f0275c640 1 -- 172.21.15.60:0/2516793760 learned_addr learned my addr 172.21.15.60:0/2516793760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.779+0000 7f5f0275c640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d988240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 -- 172.21.15.60:0/2516793760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 msgr2=0x56463d988240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d988240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 -- 172.21.15.60:0/2516793760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 msgr2=0x56463d9877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 -- 172.21.15.60:0/2516793760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d881d60 con 0x56463d97e840 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0275c640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f0175a640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d987d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5efc104e30 tx=0x7f5ef0032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4af1077b00d2fb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0035040 con 0x56463d97e840 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f5ef0031860 con 0x56463d97e840 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.780+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec0054d0 con 0x56463d97e840 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5ef0004040 con 0x56463d97e840 2024-07-24T21:57:39.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 10 client.27148 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56463d9993c0 con 0x56463d97e840 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec0054d0 con 0x56463d97e840 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5ef0004750 con 0x56463d97e840 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 10 client.27148 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 10 client.27148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 20 client.27148 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463d99a880 0x56463d99cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 4 client.27148 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56463d8f7940 con 0x56463d99a880 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.781+0000 7f5f0275c640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463d99a880 0x56463d99cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.782+0000 7f5ee17fa640 10 client.27148 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.782+0000 7f5f0275c640 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463d99a880 0x56463d99cc70 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5bcf378d93865a6 server_cookie=eb150221b42d8574 in_seq=0 out_seq=0 2024-07-24T21:57:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5ee17fa640 1 -- 172.21.15.60:0/2516793760 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56463d8f7940 con 0x56463d99a880 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5ee17fa640 10 client.27148 handle_command_reply: tid=0 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 client.27148 shutdown 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 msgr2=0x56463d987d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d987d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5efc104e30 tx=0x7f5ef0032f20 comp rx=0 tx=0).stop 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 shutdown_connections 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463d99a880 0x56463d99cc70 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d98a7d0 0x56463d988240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d97e840 0x56463d987d00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5ee17fa640 0 client.27148 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 --2- 172.21.15.60:0/2516793760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d9877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 >> 172.21.15.60:0/2516793760 conn(0x56463d97bd50 msgr2=0x56463d9818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 shutdown_connections 2024-07-24T21:57:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 1 -- 172.21.15.60:0/2516793760 wait complete. 2024-07-24T21:57:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.783+0000 7f5f06570740 20 client.27148 trim_cache size 0 max 16384 2024-07-24T21:57:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 Processor -- start 2024-07-24T21:57:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 -- start start 2024-07-24T21:57:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97e840 0x56463d9a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d9a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b7240 con 0x56463d97df20 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b4a50 con 0x56463d97e840 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f06570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b28c0 con 0x56463d98a7d0 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58840/0 (socket says 172.21.15.60:58840) 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 -- 172.21.15.60:0/1313380782 learned_addr learned my addr 172.21.15.60:0/1313380782 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0175a640 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97e840 0x56463d9a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d9a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 -- 172.21.15.60:0/1313380782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 msgr2=0x56463d9a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d9a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 -- 172.21.15.60:0/1313380782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97e840 msgr2=0x56463d9a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97e840 0x56463d9a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.803+0000 7f5f0275c640 1 -- 172.21.15.60:0/1313380782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d99d2f0 con 0x56463d97df20 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f0275c640 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009ea0 tx=0x7f5eec009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3405224fabec25 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5ee37fe640 1 -- 172.21.15.60:0/1313380782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec021040 con 0x56463d97df20 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5ee37fe640 1 -- 172.21.15.60:0/1313380782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec004ea0 con 0x56463d97df20 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5ee37fe640 1 -- 172.21.15.60:0/1313380782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec011a80 con 0x56463d97df20 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 -- 172.21.15.60:0/1313380782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 msgr2=0x56463d97ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009ea0 tx=0x7f5eec009ad0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 -- 172.21.15.60:0/1313380782 shutdown_connections 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d9a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97e840 0x56463d9a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 --2- 172.21.15.60:0/1313380782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97df20 0x56463d97ad00 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.804+0000 7f5f06570740 1 -- 172.21.15.60:0/1313380782 >> 172.21.15.60:0/1313380782 conn(0x56463d97bd50 msgr2=0x56463d97a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.805+0000 7f5f06570740 1 -- 172.21.15.60:0/1313380782 shutdown_connections 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.805+0000 7f5f06570740 1 -- 172.21.15.60:0/1313380782 wait complete. 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.805+0000 7f5f06570740 1 -- start start 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.805+0000 7f5f06570740 1 Processor -- start 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97df20 0x56463d992c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d98f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b2f10 con 0x56463d97e840 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b4380 con 0x56463d97df20 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b35e0 con 0x56463d98a7d0 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58856/0 (socket says 172.21.15.60:58856) 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 -- 172.21.15.60:0/3085212569 learned_addr learned my addr 172.21.15.60:0/3085212569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 10 client.0 _ll_register_callbacks cb 0x7f5ef0032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97df20 0x56463d992c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d98f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 -- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 msgr2=0x56463d98f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d98f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 -- 172.21.15.60:0/3085212569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97df20 msgr2=0x56463d992c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97df20 0x56463d992c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 -- 172.21.15.60:0/3085212569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d97bb80 con 0x56463d97e840 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f0175a640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0004ba0 tx=0x7f5ef0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42b0e8fbce8fdb0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0035070 con 0x56463d97e840 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8609+0+0 (secure 0 0 0) 0x7f5ef0003980 con 0x56463d97e840 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.806+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec004ea0 con 0x56463d97e840 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.807+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5ef0052040 con 0x56463d97e840 2024-07-24T21:57:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.807+0000 7f5f06570740 10 client.17778 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:39.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.807+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56463da06310 con 0x56463d97e840 2024-07-24T21:57:39.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.807+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec004ea0 con 0x56463d97e840 2024-07-24T21:57:39.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5ef0031880 con 0x56463d97e840 2024-07-24T21:57:39.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 10 client.17778 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:39.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:39.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 20 client.17778 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463da06b20 0x56463da08f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 4 client.17778 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56463d8f33e0 con 0x56463da06b20 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.808+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463da06b20 0x56463da08f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.809+0000 7f5f0275c640 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463da06b20 0x56463da08f40 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6baf6197dfdff17 server_cookie=fdc31e21b2d5f243 in_seq=0 out_seq=0 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.809+0000 7f5ee17fa640 10 client.17778 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.809+0000 7f5ee17fa640 1 -- 172.21.15.60:0/3085212569 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56463d8f33e0 con 0x56463da06b20 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.809+0000 7f5ee17fa640 10 client.17778 handle_command_reply: tid=0 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.809+0000 7f5f06570740 1 client.17778 shutdown 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 msgr2=0x56463d98f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0004ba0 tx=0x7f5ef0005900 comp rx=0 tx=0).stop 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 shutdown_connections 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56463da06b20 0x56463da08f40 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5ee17fa640 0 client.17778 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56463d98a7d0 0x56463d98f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56463d97e840 0x56463d98f130 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 --2- 172.21.15.60:0/3085212569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463d97df20 0x56463d992c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 >> 172.21.15.60:0/3085212569 conn(0x56463d97bd50 msgr2=0x56463d97ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 shutdown_connections 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 1 -- 172.21.15.60:0/3085212569 wait complete. 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.810+0000 7f5f06570740 20 client.17778 trim_cache size 0 max 16384 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:57:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b97f4c9a-80b3-4998-8291-1fcfc59a4767", 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 msgr2=0x7f5ee40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 0x7f5ee40b6200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40e1bb0 tx=0x7f5ef40e7f60 comp rx=0 tx=0).stop 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 msgr2=0x7f5efc076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9f0 tx=0x7f5ef400eeb0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 shutdown_connections 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ee40b3d40 0x7f5ee40b6200 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5efc108b00 0x7f5efc0754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5efc108180 0x7f5efc074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 --2- 172.21.15.60:0/1330521049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5efc102770 0x7f5efc076940 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 >> 172.21.15.60:0/1330521049 conn(0x7f5efc0fe1c0 msgr2=0x7f5efc1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.811+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 shutdown_connections 2024-07-24T21:57:39.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:39.812+0000 7f5f041bf640 1 -- 172.21.15.60:0/1330521049 wait complete. 2024-07-24T21:57:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:39 smithi123 ceph-mon[37143]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-07-24T21:57:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:39 smithi086 ceph-mon[37128]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-07-24T21:57:40.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:57:40.793 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:40.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:57:40.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:57:40.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:57:40.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:40.794 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:40 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:57:40.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:40 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:57:41.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:41.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:40 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:57:41.182 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:41 smithi123 ceph-mon[37143]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2024-07-24T21:57:42.004 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-07-24T21:57:42.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:57:42.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:41 smithi060 ceph-mon[33000]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2024-07-24T21:57:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:41 smithi086 ceph-mon[37128]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2024-07-24T21:57:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:42.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:43.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:43.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:43 smithi060 ceph-mon[33000]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 677 op/s 2024-07-24T21:57:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:43 smithi060 ceph-mon[33000]: osdmap e99: 12 total, 12 up, 12 in 2024-07-24T21:57:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:43 smithi123 ceph-mon[37143]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 677 op/s 2024-07-24T21:57:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:43 smithi123 ceph-mon[37143]: osdmap e99: 12 total, 12 up, 12 in 2024-07-24T21:57:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:43 smithi086 ceph-mon[37128]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 677 op/s 2024-07-24T21:57:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:43 smithi086 ceph-mon[37128]: osdmap e99: 12 total, 12 up, 12 in 2024-07-24T21:57:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:44 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:44.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:44 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:44 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:45.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 -- 172.21.15.60:0/3016125950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 msgr2=0x7f4c0810f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 --2- 172.21.15.60:0/3016125950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c0810f340 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009be0 tx=0x7f4c0002a6c0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 -- 172.21.15.60:0/3016125950 shutdown_connections 2024-07-24T21:57:45.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 --2- 172.21.15.60:0/3016125950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c0810f340 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 --2- 172.21.15.60:0/3016125950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081081c0 0x7f4c08108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 --2- 172.21.15.60:0/3016125950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081027b0 0x7f4c08102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 -- 172.21.15.60:0/3016125950 >> 172.21.15.60:0/3016125950 conn(0x7f4c080fe200 msgr2=0x7f4c08100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 -- 172.21.15.60:0/3016125950 shutdown_connections 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.531+0000 7f4c0e655640 1 -- 172.21.15.60:0/3016125950 wait complete. 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 Processor -- start 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 -- start start 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081081c0 0x7f4c0819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c081a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08076c40 con 0x7f4c08108b40 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08076ac0 con 0x7f4c081027b0 2024-07-24T21:57:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0e655640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08076dc0 con 0x7f4c081081c0 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c081a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59964/0 (socket says 172.21.15.60:59964) 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c081a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58878/0 (socket says 172.21.15.60:58878) 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c07fff640 1 -- 172.21.15.60:0/302506415 learned_addr learned my addr 172.21.15.60:0/302506415 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c077fe640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081081c0 0x7f4c0819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.532+0000 7f4c07fff640 1 -- 172.21.15.60:0/302506415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081081c0 msgr2=0x7f4c0819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c07fff640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081081c0 0x7f4c0819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c07fff640 1 -- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 msgr2=0x7f4c081a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c07fff640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c081a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c07fff640 1 -- 172.21.15.60:0/302506415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c081a0e80 con 0x7f4c081027b0 2024-07-24T21:57:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c07fff640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c057fa640 1 -- 172.21.15.60:0/302506415 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800ccc0 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c057fa640 1 -- 172.21.15.60:0/302506415 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf80054d0 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c057fa640 1 -- 172.21.15.60:0/302506415 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8010430 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c081a1170 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.533+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a7830 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.535+0000 7f4c10a06740 1 Processor -- start 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 -- start start 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c057fa640 1 -- 172.21.15.60:0/302506415 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4bf8004f80 con 0x7f4c081027b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d68fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68ffd0 0x55d57d69ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d57d606990 con 0x55d57d68ffd0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d57d5f5e90 con 0x55d57d68f6b0 2024-07-24T21:57:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c10a06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d57d5f9ed0 con 0x55d57d69bf60 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d68fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68ffd0 0x55d57d69ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:52352/0 (socket says 172.21.15.60:52352) 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c077fe640 1 -- 172.21.15.60:0/2768073187 learned_addr learned my addr 172.21.15.60:0/2768073187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/2768073187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 msgr2=0x55d57d69e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/2768073187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68ffd0 msgr2=0x55d57d69ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68ffd0 0x55d57d69ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.536+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/2768073187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d57d57f300 con 0x55d57d68f6b0 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4c077fe640 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d68fa90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002abe0 tx=0x7f4c00005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed06fbf116f4ebee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4c057fa640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 0x7f4bf00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4bee7fc640 1 -- 172.21.15.60:0/2768073187 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00041030 con 0x55d57d68f6b0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4bee7fc640 1 -- 172.21.15.60:0/2768073187 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00003e20 con 0x55d57d68f6b0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4bee7fc640 1 -- 172.21.15.60:0/2768073187 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00036040 con 0x55d57d68f6b0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4c07fff640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 0x7f4bf00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.537+0000 7f4c057fa640 1 -- 172.21.15.60:0/302506415 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f4bf80e4410 con 0x7f4c081027b0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- 172.21.15.60:0/2768073187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 msgr2=0x55d57d68fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c07fff640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 0x7f4bf00b6200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80e1be0 tx=0x7f4bf80e9040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d68fa90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002abe0 tx=0x7f4c00005ec0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- 172.21.15.60:0/2768073187 shutdown_connections 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d69bf60 0x55d57d69e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68ffd0 0x55d57d69ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2768073187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d68fa90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- 172.21.15.60:0/2768073187 >> 172.21.15.60:0/2768073187 conn(0x55d57d68d4e0 msgr2=0x55d57d68e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- 172.21.15.60:0/2768073187 shutdown_connections 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- 172.21.15.60:0/2768073187 wait complete. 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.538+0000 7f4c10a06740 1 -- start start 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 Processor -- start 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 0x55d57d698ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d699940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d57d459700 con 0x55d57d69bf60 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d57d5985d0 con 0x55d57d68ffd0 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d57d69eb80 con 0x55d57d68f6b0 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c10a06740 10 client.0 _ll_register_callbacks cb 0x55d57d5b8940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 0x55d57d698ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59986/0 (socket says 172.21.15.60:59986) 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 -- 172.21.15.60:0/2122761030 learned_addr learned my addr 172.21.15.60:0/2122761030 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d699940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 -- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 msgr2=0x55d57d698ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 0x55d57d698ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 -- 172.21.15.60:0/2122761030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 msgr2=0x55d57d699940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.539+0000 7f4c077fe640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d699940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c077fe640 1 -- 172.21.15.60:0/2122761030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d57d58b660 con 0x55d57d68ffd0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 0x55d57d698ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c077fe640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc012870 tx=0x7f4bfc012d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da95059d70f6a8ba server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc004020 con 0x55d57d68ffd0 2024-07-24T21:57:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f4bfc0041c0 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c00003e20 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4bfc0043c0 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c10a06740 10 client.27206 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d57d356650 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00003e20 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.540+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4bfc023070 con 0x55d57d68ffd0 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 10 client.27206 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 10 client.27206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 20 client.27206 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d6abab0 0x55d57d6aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 4 client.27206 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d57d6088e0 con 0x55d57d6abab0 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d6abab0 0x55d57d6aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d6abab0 0x55d57d6aded0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12557a958a70442c server_cookie=d0a213b53ef2f976 in_seq=0 out_seq=0 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.541+0000 7f4be77fe640 10 client.27206 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4be77fe640 1 -- 172.21.15.60:0/2122761030 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d57d6088e0 con 0x55d57d6abab0 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4be77fe640 10 client.27206 handle_command_reply: tid=0 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 client.27206 shutdown 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 msgr2=0x55d57d699400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc012870 tx=0x7f4bfc012d30 comp rx=0 tx=0).stop 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 shutdown_connections 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d6abab0 0x55d57d6aded0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d699940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68ffd0 0x55d57d699400 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 --2- 172.21.15.60:0/2122761030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68f6b0 0x55d57d698ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4be77fe640 0 client.27206 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 >> 172.21.15.60:0/2122761030 conn(0x55d57d68d4e0 msgr2=0x55d57d693080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 shutdown_connections 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 1 -- 172.21.15.60:0/2122761030 wait complete. 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.542+0000 7f4c10a06740 20 client.27206 trim_cache size 0 max 16384 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 Processor -- start 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 -- start start 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d6b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d697740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081b2cd0 con 0x55d57d69bf60 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081b04e0 con 0x55d57d68f6b0 2024-07-24T21:57:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c10a06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081ae350 con 0x55d57d68ffd0 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:59996/0 (socket says 172.21.15.60:59996) 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/3087514742 learned_addr learned my addr 172.21.15.60:0/3087514742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c077fe640 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d6b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d697740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/3087514742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 msgr2=0x55d57d6b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d6b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/3087514742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 msgr2=0x55d57d697740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d697740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.563+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/3087514742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d57d69e6f0 con 0x55d57d68f6b0 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.564+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002ac80 tx=0x7f4c0002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=943ddaa780c74e85 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.564+0000 7f4bedffb640 1 -- 172.21.15.60:0/3087514742 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00036040 con 0x55d57d68f6b0 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.564+0000 7f4bedffb640 1 -- 172.21.15.60:0/3087514742 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00003e20 con 0x55d57d68f6b0 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.564+0000 7f4bedffb640 1 -- 172.21.15.60:0/3087514742 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004610 con 0x55d57d68f6b0 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- 172.21.15.60:0/3087514742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 msgr2=0x55d57d6b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002ac80 tx=0x7f4c0002a6c0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- 172.21.15.60:0/3087514742 shutdown_connections 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d69bf60 0x55d57d697740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d6b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 --2- 172.21.15.60:0/3087514742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d68f6b0 0x55d57d6b37a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- 172.21.15.60:0/3087514742 >> 172.21.15.60:0/3087514742 conn(0x55d57d68d4e0 msgr2=0x55d57d69ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- 172.21.15.60:0/3087514742 shutdown_connections 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- 172.21.15.60:0/3087514742 wait complete. 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 -- start start 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.565+0000 7f4c10a06740 1 Processor -- start 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d69f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d69bf60 0x55d57d69ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081ae9a0 con 0x55d57d68f6b0 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081afe10 con 0x55d57d69bf60 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081af070 con 0x55d57d68ffd0 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c10a06740 10 client.0 _ll_register_callbacks cb 0x7f4bfc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d69f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58916/0 (socket says 172.21.15.60:58916) 2024-07-24T21:57:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/1787458839 learned_addr learned my addr 172.21.15.60:0/1787458839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d69bf60 0x55d57d69ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 msgr2=0x55d57d69f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d69f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/1787458839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d69bf60 msgr2=0x55d57d69ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d69bf60 0x55d57d69ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 -- 172.21.15.60:0/1787458839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d57d6a0770 con 0x55d57d68f6b0 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c077fe640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d69f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4c0cbf2640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009560 tx=0x7f4c00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ffebb19f97b1f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.566+0000 7f4be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00036070 con 0x55d57d68f6b0 2024-07-24T21:57:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c00003e20 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f4c00003970 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4c00003b70 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4c10a06740 10 client.17796 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d57d716820 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.567+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00003e20 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4c0005c070 con 0x55d57d68f6b0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 10 client.17796 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 20 client.17796 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d716ab0 0x55d57d718ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 4 client.17796 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d57d604380 con 0x55d57d716ab0 2024-07-24T21:57:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c077fe640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d716ab0 0x55d57d718ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c077fe640 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d716ab0 0x55d57d718ed0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1e76d4eb9e0b2d server_cookie=a69a775233b82bc9 in_seq=0 out_seq=0 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4be77fe640 10 client.17796 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4be77fe640 1 -- 172.21.15.60:0/1787458839 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d57d604380 con 0x55d57d716ab0 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4be77fe640 10 client.17796 handle_command_reply: tid=0 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.568+0000 7f4c10a06740 1 client.17796 shutdown 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 msgr2=0x55d57d6a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009560 tx=0x7f4c00009840 comp rx=0 tx=0).stop 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 shutdown_connections 2024-07-24T21:57:45.268 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "b97f4c9a-80b3-4998-8291-1fcfc59a4767": { 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b97f4c9a-80b3-4998-8291-1fcfc59a4767", 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d57d716ab0 0x55d57d718ed0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d57d69bf60 0x55d57d69ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d57d68ffd0 0x55d57d69f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 --2- 172.21.15.60:0/1787458839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d57d68f6b0 0x55d57d6a44f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 >> 172.21.15.60:0/1787458839 conn(0x55d57d68d4e0 msgr2=0x55d57d6927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4be77fe640 0 client.17796 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 shutdown_connections 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 1 -- 172.21.15.60:0/1787458839 wait complete. 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.569+0000 7f4c10a06740 20 client.17796 trim_cache size 0 max 16384 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 msgr2=0x7f4bf00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 0x7f4bf00b6200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80e1be0 tx=0x7f4bf80e9040 comp rx=0 tx=0).stop 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 msgr2=0x7f4c0819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf800ee80 comp rx=0 tx=0).stop 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 shutdown_connections 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4bf00b3d40 0x7f4bf00b6200 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08108b40 0x7f4c081a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c081081c0 0x7f4c0819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 --2- 172.21.15.60:0/302506415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c081027b0 0x7f4c0819add0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:45.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 >> 172.21.15.60:0/302506415 conn(0x7f4c080fe200 msgr2=0x7f4c081005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:45.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 shutdown_connections 2024-07-24T21:57:45.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:44.570+0000 7f4c0e655640 1 -- 172.21.15.60:0/302506415 wait complete. 2024-07-24T21:57:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:45 smithi123 ceph-mon[37143]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 1.62k op/s 2024-07-24T21:57:46.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:45 smithi086 ceph-mon[37128]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 1.62k op/s 2024-07-24T21:57:46.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:45 smithi060 ceph-mon[33000]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 1.62k op/s 2024-07-24T21:57:47.928 INFO:tasks.cephfs.filesystem:scrub status for tag:b97f4c9a-80b3-4998-8291-1fcfc59a4767 - {'path': '/', 'tag': 'b97f4c9a-80b3-4998-8291-1fcfc59a4767', 'options': 'recursive,force'} 2024-07-24T21:57:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:47 smithi123 ceph-mon[37143]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 3.67k op/s 2024-07-24T21:57:48.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:47 smithi060 ceph-mon[33000]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 3.67k op/s 2024-07-24T21:57:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:47 smithi086 ceph-mon[37128]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.7 MiB/s wr, 3.67k op/s 2024-07-24T21:57:48.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- 172.21.15.60:0/1749096801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 msgr2=0x7f72e010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 --2- 172.21.15.60:0/1749096801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e010ef20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009ca0 tx=0x7f72d002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- 172.21.15.60:0/1749096801 shutdown_connections 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 --2- 172.21.15.60:0/1749096801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e010ef20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 --2- 172.21.15.60:0/1749096801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103f80 0x7f72e010c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 --2- 172.21.15.60:0/1749096801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103660 0x7f72e0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- 172.21.15.60:0/1749096801 >> 172.21.15.60:0/1749096801 conn(0x7f72e00fd0c0 msgr2=0x7f72e00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- 172.21.15.60:0/1749096801 shutdown_connections 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- 172.21.15.60:0/1749096801 wait complete. 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 Processor -- start 2024-07-24T21:57:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.337+0000 7f72e5782640 1 -- start start 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0112760 con 0x7f72e010cac0 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01125e0 con 0x7f72e0103660 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72e5782640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01128e0 con 0x7f72e0103f80 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72df7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72de7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72de7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:45966/0 (socket says 172.21.15.60:45966) 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72de7fc640 1 -- 172.21.15.60:0/3346312903 learned_addr learned my addr 172.21.15.60:0/3346312903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55804/0 (socket says 172.21.15.60:55804) 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 -- 172.21.15.60:0/3346312903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 msgr2=0x7f72e019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 -- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 msgr2=0x7f72e01a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e01a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 -- 172.21.15.60:0/3346312903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e01a0ee0 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72de7fc640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.338+0000 7f72deffd640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72d800e9f0 tx=0x7f72d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.339+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ccc0 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.339+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80108e0 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.339+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e01a11d0 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.339+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8018880 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.339+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01a7890 con 0x7f72e0103660 2024-07-24T21:57:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.340+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e0103a40 con 0x7f72e0103660 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.341+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f72d8010430 con 0x7f72e0103660 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.342+0000 7f72d7fff640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 0x7f72cc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.342+0000 7f72de7fc640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 0x7f72cc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.342+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f72d80e4810 con 0x7f72e0103660 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.342+0000 7f72de7fc640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 0x7f72cc0b6200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f72e010f120 tx=0x7f72c8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.345+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f72d80ab440 con 0x7f72e0103660 2024-07-24T21:57:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.647+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e01a7c50 con 0x7f72e0103660 2024-07-24T21:57:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.648+0000 7f72d7fff640 1 -- 172.21.15.60:0/3346312903 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f72d80af990 con 0x7f72e0103660 2024-07-24T21:57:48.650 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:57:48.650 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:57:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 msgr2=0x7f72cc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 0x7f72cc0b6200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f72e010f120 tx=0x7f72c8005f90 comp rx=0 tx=0).stop 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 msgr2=0x7f72e019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72d800e9f0 tx=0x7f72d800eeb0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 shutdown_connections 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72cc0b3d40 0x7f72cc0b6200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72e010cac0 0x7f72e01a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e0103f80 0x7f72e019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 --2- 172.21.15.60:0/3346312903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72e0103660 0x7f72e019ae30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:48.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.650+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 >> 172.21.15.60:0/3346312903 conn(0x7f72e00fd0c0 msgr2=0x7f72e00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:48.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.651+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 shutdown_connections 2024-07-24T21:57:48.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:48.651+0000 7f72e5782640 1 -- 172.21.15.60:0/3346312903 wait complete. 2024-07-24T21:57:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:48 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:48 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3346312903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:49.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:48 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:49.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:48 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3346312903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:49.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:48 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:57:49.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:48 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3346312903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:57:49.484 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-07-24T21:57:49.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-07-24T21:57:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:49.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:49 smithi060 ceph-mon[33000]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 4.48k op/s 2024-07-24T21:57:49.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:49 smithi123 ceph-mon[37143]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 4.48k op/s 2024-07-24T21:57:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:49 smithi086 ceph-mon[37128]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 4.48k op/s 2024-07-24T21:57:51.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 -- 172.21.15.60:0/3216811873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78410cac0 msgr2=0x7fa78410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 --2- 172.21.15.60:0/3216811873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78410cac0 0x7fa78410ef20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009900 tx=0x7fa77c02a410 comp rx=0 tx=0).stop 2024-07-24T21:57:51.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 -- 172.21.15.60:0/3216811873 shutdown_connections 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 --2- 172.21.15.60:0/3216811873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78410cac0 0x7fa78410ef20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 --2- 172.21.15.60:0/3216811873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa784103f80 0x7fa78410c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 --2- 172.21.15.60:0/3216811873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103660 0x7fa784103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 -- 172.21.15.60:0/3216811873 >> 172.21.15.60:0/3216811873 conn(0x7fa7840fd0c0 msgr2=0x7fa7840ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 -- 172.21.15.60:0/3216811873 shutdown_connections 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.366+0000 7fa789a20640 1 -- 172.21.15.60:0/3216811873 wait complete. 2024-07-24T21:57:51.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 Processor -- start 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 -- start start 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 0x7fa78419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 0x7fa7841a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa784112720 con 0x7fa784103660 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841125a0 con 0x7fa78410cac0 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa789a20640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841128a0 con 0x7fa784103f80 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.367+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 0x7fa78419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57400/0 (socket says 172.21.15.60:57400) 2024-07-24T21:57:51.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 -- 172.21.15.60:0/2540301546 learned_addr learned my addr 172.21.15.60:0/2540301546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 0x7fa7841a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 -- 172.21.15.60:0/2540301546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 msgr2=0x7fa78419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 0x7fa78419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 -- 172.21.15.60:0/2540301546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 msgr2=0x7fa7841a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 0x7fa7841a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 -- 172.21.15.60:0/2540301546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7841a0ef0 con 0x7fa784103660 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 0x7fa78419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 0x7fa7841a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa77400eb90 tx=0x7fa77400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa788a1e640 1 -- 172.21.15.60:0/2540301546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774004d20 con 0x7fa784103660 2024-07-24T21:57:51.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7841a11e0 con 0x7fa784103660 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa788a1e640 1 -- 172.21.15.60:0/2540301546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa77400ee90 con 0x7fa784103660 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa788a1e640 1 -- 172.21.15.60:0/2540301546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774010670 con 0x7fa784103660 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.368+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7841a78a0 con 0x7fa784103660 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 Processor -- start 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa788a1e640 1 -- 172.21.15.60:0/2540301546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa774005020 con 0x7fa784103660 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 -- start start 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d9197260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 0x55d2d9197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d2d910d6a0 con 0x55d2d91977a0 2024-07-24T21:57:51.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2d90fcba0 con 0x55d2d91a35d0 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9100be0 con 0x55d2d9198d00 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 0x55d2d9197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55828/0 (socket says 172.21.15.60:55828) 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa7827fc640 1 -- 172.21.15.60:0/2853595773 learned_addr learned my addr 172.21.15.60:0/2853595773 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 0x55d2d9197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57412/0 (socket says 172.21.15.60:57412) 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.371+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d9197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 -- 172.21.15.60:0/2853595773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 msgr2=0x55d2d9197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d9197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 -- 172.21.15.60:0/2853595773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 msgr2=0x55d2d9197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 0x55d2d9197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 -- 172.21.15.60:0/2853595773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9094d60 con 0x55d2d91a35d0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d9197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7780114f0 tx=0x7fa778009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238784f89e3b969 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa788a1e640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 0x7fa76c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa769ffb640 1 -- 172.21.15.60:0/2853595773 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778005020 con 0x55d2d91a35d0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 0x7fa76c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa769ffb640 1 -- 172.21.15.60:0/2853595773 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7780047a0 con 0x55d2d91a35d0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa769ffb640 1 -- 172.21.15.60:0/2853595773 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800f8c0 con 0x55d2d91a35d0 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.372+0000 7fa788a1e640 1 -- 172.21.15.60:0/2540301546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fa7740e4660 con 0x7fa784103660 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2853595773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 msgr2=0x55d2d91a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 0x7fa76c0b6200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006010 tx=0x7fa77c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7780114f0 tx=0x7fa778009f90 comp rx=0 tx=0).stop 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2853595773 shutdown_connections 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91a59c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91977a0 0x55d2d9197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2853595773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d9197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2853595773 >> 172.21.15.60:0/2853595773 conn(0x55d2d9161750 msgr2=0x55d2d9195090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2853595773 shutdown_connections 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2853595773 wait complete. 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.373+0000 7fa78bdd1740 1 -- start start 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 Processor -- start 2024-07-24T21:57:51.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91a35d0 0x55d2d91a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8f607a0 con 0x55d2d9198d00 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2d909f020 con 0x55d2d91977a0 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d2d91a6490 con 0x55d2d91a35d0 2024-07-24T21:57:51.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa78bdd1740 10 client.0 _ll_register_callbacks cb 0x55d2d8e0a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:51.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57426/0 (socket says 172.21.15.60:57426) 2024-07-24T21:57:51.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa7827fc640 1 -- 172.21.15.60:0/2491921469 learned_addr learned my addr 172.21.15.60:0/2491921469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa7837fe640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91a35d0 0x55d2d91a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 -- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91a35d0 msgr2=0x55d2d91a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91a35d0 0x55d2d91a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 -- 172.21.15.60:0/2491921469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 msgr2=0x55d2d91a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa782ffd640 1 -- 172.21.15.60:0/2491921469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d90a30d0 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.374+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa782ffd640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91a3000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7740e1d80 tx=0x7fa774004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900513533a5d6e43 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740aab90 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fa7740b4070 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7780047a0 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa7740aa9b0 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 10 client.27227 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d2d8e6cb90 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7780047a0 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa7740e4b10 con 0x55d2d91977a0 2024-07-24T21:57:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 10 client.27227 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 10 client.27227 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 20 client.27227 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d91b3680 0x55d2d91b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 4 client.27227 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.375+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d910f5f0 con 0x55d2d91b3680 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.376+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d91b3680 0x55d2d91b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.376+0000 7fa7827fc640 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d91b3680 0x55d2d91b5b20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1230c856b89ec276 server_cookie=4d79db35c0671408 in_seq=0 out_seq=0 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.376+0000 7fa762ffd640 10 client.27227 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa762ffd640 1 -- 172.21.15.60:0/2491921469 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2d910f5f0 con 0x55d2d91b3680 2024-07-24T21:57:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa762ffd640 10 client.27227 handle_command_reply: tid=0 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 client.27227 shutdown 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 msgr2=0x55d2d91a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91a3000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7740e1d80 tx=0x7fa774004ec0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 shutdown_connections 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d91b3680 0x55d2d91b5b20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91a35d0 0x55d2d91a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa762ffd640 0 client.27227 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/2491921469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91a3000 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 >> 172.21.15.60:0/2491921469 conn(0x55d2d9161750 msgr2=0x55d2d919ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 shutdown_connections 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 1 -- 172.21.15.60:0/2491921469 wait complete. 2024-07-24T21:57:51.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.377+0000 7fa78bdd1740 20 client.27227 trim_cache size 0 max 16384 2024-07-24T21:57:51.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.395+0000 7fa78bdd1740 1 Processor -- start 2024-07-24T21:57:51.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.395+0000 7fa78bdd1740 1 -- start start 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.395+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91977a0 0x55d2d9196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.395+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.395+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b2db0 con 0x55d2d9198d00 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b05c0 con 0x55d2d91a35d0 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841ae430 con 0x55d2d91977a0 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57438/0 (socket says 172.21.15.60:57438) 2024-07-24T21:57:51.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 -- 172.21.15.60:0/336740387 learned_addr learned my addr 172.21.15.60:0/336740387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7837fe640 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa782ffd640 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91977a0 0x55d2d9196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 -- 172.21.15.60:0/336740387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91977a0 msgr2=0x55d2d9196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91977a0 0x55d2d9196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 -- 172.21.15.60:0/336740387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 msgr2=0x55d2d91bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 -- 172.21.15.60:0/336740387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d91907a0 con 0x55d2d9198d00 2024-07-24T21:57:51.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7827fc640 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa778005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab10ad987ba9f43e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.396+0000 7fa7697fa640 1 -- 172.21.15.60:0/336740387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778011030 con 0x55d2d9198d00 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa7697fa640 1 -- 172.21.15.60:0/336740387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa778005560 con 0x55d2d9198d00 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa7697fa640 1 -- 172.21.15.60:0/336740387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778004530 con 0x55d2d9198d00 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 -- 172.21.15.60:0/336740387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 msgr2=0x55d2d91a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa778005da0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 -- 172.21.15.60:0/336740387 shutdown_connections 2024-07-24T21:57:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91a35d0 0x55d2d91bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d9198d00 0x55d2d91a2190 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/336740387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d91977a0 0x55d2d9196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 -- 172.21.15.60:0/336740387 >> 172.21.15.60:0/336740387 conn(0x55d2d9161750 msgr2=0x55d2d91a1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 -- 172.21.15.60:0/336740387 shutdown_connections 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.397+0000 7fa78bdd1740 1 -- 172.21.15.60:0/336740387 wait complete. 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 -- start start 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 Processor -- start 2024-07-24T21:57:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d91abc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91a35d0 0x55d2d91a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841aea80 con 0x55d2d91a35d0 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841afef0 con 0x55d2d91977a0 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841af150 con 0x55d2d9198d00 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa78bdd1740 10 client.0 _ll_register_callbacks cb 0x7fa774004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:51.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91a35d0 0x55d2d91a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.398+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d91abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55858/0 (socket says 172.21.15.60:55858) 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa7827fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d91abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46024/0 (socket says 172.21.15.60:46024) 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 -- 172.21.15.60:0/1797848772 learned_addr learned my addr 172.21.15.60:0/1797848772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 -- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 msgr2=0x55d2d91abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d91abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 -- 172.21.15.60:0/1797848772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91a35d0 msgr2=0x55d2d91a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91a35d0 0x55d2d91a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 -- 172.21.15.60:0/1797848772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d91a0f00 con 0x55d2d91977a0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa782ffd640 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7740a72c0 tx=0x7fa7740a7170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a9355299335bb0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740aab90 con 0x55d2d91977a0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fa7740a7830 con 0x55d2d91977a0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa778005560 con 0x55d2d91977a0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa7740aad50 con 0x55d2d91977a0 2024-07-24T21:57:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa78bdd1740 10 client.27233 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.399+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d2d9227180 con 0x55d2d91977a0 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa778005560 con 0x55d2d91977a0 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa7740cc300 con 0x55d2d91977a0 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 10 client.27233 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 10 client.27233 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 20 client.27233 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d92277b0 0x55d2d9229bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 4 client.27233 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-07-24T21:57:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55d2d910b090 con 0x55d2d92277b0 2024-07-24T21:57:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa7827fc640 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d92277b0 0x55d2d9229bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa7827fc640 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d92277b0 0x55d2d9229bd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfeb66322f927e0 server_cookie=4acb2d4996e90710 in_seq=0 out_seq=0 2024-07-24T21:57:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:51.400+0000 7fa762ffd640 10 client.27233 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 -- 172.21.15.60:0/533077289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108140 msgr2=0x7ff4bc108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/533077289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108140 0x7ff4bc108580 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009900 tx=0x7ff4b002a410 comp rx=0 tx=0).stop 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 -- 172.21.15.60:0/533077289 shutdown_connections 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/533077289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/533077289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108140 0x7ff4bc108580 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/533077289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc102740 0x7ff4bc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 -- 172.21.15.60:0/533077289 >> 172.21.15.60:0/533077289 conn(0x7ff4bc0fe180 msgr2=0x7ff4bc1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 -- 172.21.15.60:0/533077289 shutdown_connections 2024-07-24T21:57:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 -- 172.21.15.60:0/533077289 wait complete. 2024-07-24T21:57:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.620+0000 7ff4c1abe640 1 Processor -- start 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 -- start start 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc102740 0x7ff4bc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 0x7ff4bc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc112710 con 0x7ff4bc108ac0 2024-07-24T21:57:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc112590 con 0x7ff4bc108140 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4c1abe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc112890 con 0x7ff4bc102740 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 0x7ff4bc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57474/0 (socket says 172.21.15.60:57474) 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 -- 172.21.15.60:0/1121338805 learned_addr learned my addr 172.21.15.60:0/1121338805 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc102740 0x7ff4bc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 -- 172.21.15.60:0/1121338805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc102740 msgr2=0x7ff4bc198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc102740 0x7ff4bc198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 -- 172.21.15.60:0/1121338805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 msgr2=0x7ff4bc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 0x7ff4bc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 -- 172.21.15.60:0/1121338805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc19ecd0 con 0x7ff4bc108ac0 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4baffd640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 0x7ff4bc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.621+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400eb10 tx=0x7ff4b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.622+0000 7ff4b8ff9640 1 -- 172.21.15.60:0/1121338805 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004b20 con 0x7ff4bc108ac0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.622+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc19efc0 con 0x7ff4bc108ac0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.622+0000 7ff4b8ff9640 1 -- 172.21.15.60:0/1121338805 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b4004cc0 con 0x7ff4bc108ac0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.622+0000 7ff4b8ff9640 1 -- 172.21.15.60:0/1121338805 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4010670 con 0x7ff4bc108ac0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.622+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc1a5680 con 0x7ff4bc108ac0 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 Processor -- start 2024-07-24T21:57:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4b8ff9640 1 -- 172.21.15.60:0/1121338805 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff4b4005020 con 0x7ff4bc108ac0 2024-07-24T21:57:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 -- start start 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f3fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56255f362280 con 0x56255f3f9170 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56255f351780 con 0x56255f3ed1e0 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4c4813740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56255f3557c0 con 0x56255f3ec8c0 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f3fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55874/0 (socket says 172.21.15.60:55874) 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.625+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/2260642491 learned_addr learned my addr 172.21.15.60:0/2260642491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/2260642491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 msgr2=0x56255f3ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/2260642491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 msgr2=0x56255f3fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f3fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/2260642491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255f0b5790 con 0x56255f3ed1e0 2024-07-24T21:57:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4b8ff9640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 0x7ff4a40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4baffd640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 0x7ff4a40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009cf0 tx=0x7ff4ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebcdf3ac8d47bb5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4b8ff9640 1 -- 172.21.15.60:0/1121338805 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7ff4b40e45c0 con 0x7ff4bc108ac0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4a1ffb640 1 -- 172.21.15.60:0/2260642491 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01e070 con 0x56255f3ed1e0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4a1ffb640 1 -- 172.21.15.60:0/2260642491 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ac009de0 con 0x56255f3ed1e0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.626+0000 7ff4a1ffb640 1 -- 172.21.15.60:0/2260642491 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac019450 con 0x56255f3ed1e0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- 172.21.15.60:0/2260642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 msgr2=0x56255f3f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4baffd640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 0x7ff4a40b6200 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0031440 tx=0x7ff4b0005780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009cf0 tx=0x7ff4ac011810 comp rx=0 tx=0).stop 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- 172.21.15.60:0/2260642491 shutdown_connections 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f3fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f8c30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 --2- 172.21.15.60:0/2260642491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- 172.21.15.60:0/2260642491 >> 172.21.15.60:0/2260642491 conn(0x56255f3ea6f0 msgr2=0x56255f3eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- 172.21.15.60:0/2260642491 shutdown_connections 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- 172.21.15.60:0/2260642491 wait complete. 2024-07-24T21:57:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 -- start start 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.627+0000 7ff4c4813740 1 Processor -- start 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3f6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56255f28bab0 con 0x56255f3ed1e0 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56255f2d99e0 con 0x56255f3f9170 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56255f3fbe50 con 0x56255f3ec8c0 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 10 client.0 _ll_register_callbacks cb 0x56255f061a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57490/0 (socket says 172.21.15.60:57490) 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 -- 172.21.15.60:0/4116315443 learned_addr learned my addr 172.21.15.60:0/4116315443 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 -- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 msgr2=0x56255f3f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 -- 172.21.15.60:0/4116315443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 msgr2=0x56255f3f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 -- 172.21.15.60:0/4116315443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255f12e620 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4baffd640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0033000 tx=0x7ff4b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a20b9c16c8da05 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff49affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b004f070 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7ff4b00037f0 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.628+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56255f300ce0 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff4b0035040 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 10 client.17832 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56255f4075e0 con 0x56255f3ed1e0 2024-07-24T21:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56255f300ce0 con 0x56255f3ed1e0 2024-07-24T21:57:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff4b005b070 con 0x56255f3ed1e0 2024-07-24T21:57:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 10 client.17832 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 20 client.17832 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f408bb0 0x56255f40afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 4 client.17832 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56255f3641d0 con 0x56255f408bb0 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.629+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f408bb0 0x56255f40afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.630+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f408bb0 0x56255f40afd0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736010ac824ed5f4 server_cookie=efb4b56dc31783d0 in_seq=0 out_seq=0 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.630+0000 7ff49affd640 10 client.17832 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.630+0000 7ff49affd640 1 -- 172.21.15.60:0/4116315443 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56255f3641d0 con 0x56255f408bb0 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.630+0000 7ff49affd640 10 client.17832 handle_command_reply: tid=0 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.630+0000 7ff4c4813740 1 client.17832 shutdown 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 msgr2=0x56255f3f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0033000 tx=0x7ff4b002a410 comp rx=0 tx=0).stop 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 shutdown_connections 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f408bb0 0x56255f40afd0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ed1e0 0x56255f3f5d40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 --2- 172.21.15.60:0/4116315443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff49affd640 0 client.17832 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 >> 172.21.15.60:0/4116315443 conn(0x56255f3ea6f0 msgr2=0x56255f3f0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 shutdown_connections 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 1 -- 172.21.15.60:0/4116315443 wait complete. 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.631+0000 7ff4c4813740 20 client.17832 trim_cache size 0 max 16384 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 Processor -- start 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 -- start start 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b0b70 con 0x56255f3f9170 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1ae380 con 0x56255f3ed1e0 2024-07-24T21:57:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4c4813740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1ac1f0 con 0x56255f3ec8c0 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57498/0 (socket says 172.21.15.60:57498) 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 -- 172.21.15.60:0/542770881 learned_addr learned my addr 172.21.15.60:0/542770881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4baffd640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 -- 172.21.15.60:0/542770881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 msgr2=0x56255f3f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 -- 172.21.15.60:0/542770881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 msgr2=0x56255f3f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.646+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4bbfff640 1 -- 172.21.15.60:0/542770881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255f3ed010 con 0x56255f3f9170 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4baffd640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40049d0 tx=0x7ff4b40aaaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688980ba94a058e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4a17fa640 1 -- 172.21.15.60:0/542770881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40ab8a0 con 0x56255f3f9170 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4a17fa640 1 -- 172.21.15.60:0/542770881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b40ab1a0 con 0x56255f3f9170 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4a17fa640 1 -- 172.21.15.60:0/542770881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40b4020 con 0x56255f3f9170 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 -- 172.21.15.60:0/542770881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 msgr2=0x56255f4108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40049d0 tx=0x7ff4b40aaaf0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 -- 172.21.15.60:0/542770881 shutdown_connections 2024-07-24T21:57:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3f9170 0x56255f4108a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3ed1e0 0x56255f3f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 --2- 172.21.15.60:0/542770881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ec8c0 0x56255f3f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 -- 172.21.15.60:0/542770881 >> 172.21.15.60:0/542770881 conn(0x56255f3ea6f0 msgr2=0x56255f3f5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 -- 172.21.15.60:0/542770881 shutdown_connections 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.647+0000 7ff4c4813740 1 -- 172.21.15.60:0/542770881 wait complete. 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 -- start start 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 Processor -- start 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ed1e0 0x56255f3fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1ac840 con 0x56255f3ec8c0 2024-07-24T21:57:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1adcb0 con 0x56255f3f9170 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1acf10 con 0x56255f3ed1e0 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4c4813740 10 client.0 _ll_register_callbacks cb 0x7ff4b0041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ed1e0 0x56255f3fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57512/0 (socket says 172.21.15.60:57512) 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/3565857342 learned_addr learned my addr 172.21.15.60:0/3565857342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.648+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ed1e0 msgr2=0x56255f3fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ed1e0 0x56255f3fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/3565857342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 msgr2=0x56255f3fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:57:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 -- 172.21.15.60:0/3565857342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255f3fceb0 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bbfff640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4bb7fe640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x56255f3efa40 tx=0x7ff4ac009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b13cf84370da50 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01e040 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7ff4ac0036f0 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b40ab1a0 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff4ac036050 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4c4813740 10 client.17838 fetch_fsmap learned FSMap version 44 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56255f472940 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b40ab1a0 con 0x56255f3ec8c0 2024-07-24T21:57:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff4ac02a450 con 0x56255f3ec8c0 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.649+0000 7ff4c4813740 10 client.17838 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4c4813740 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4c4813740 20 client.17838 populate_metadata read hostname 'smithi060' 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f472f50 0x56255f475370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4c4813740 4 client.17838 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56255f35fc70 con 0x56255f472f50 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4baffd640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f472f50 0x56255f475370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff49affd640 10 client.17838 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:51.650+0000 7ff4baffd640 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f472f50 0x56255f475370 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d7dfb167dfecdf server_cookie=94c769078c6596d2 in_seq=0 out_seq=0 2024-07-24T21:57:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:51 smithi123 ceph-mon[37143]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 4.65k op/s 2024-07-24T21:57:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:51 smithi086 ceph-mon[37128]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 4.65k op/s 2024-07-24T21:57:52.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:51 smithi060 ceph-mon[33000]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 4.65k op/s 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff49affd640 1 -- 172.21.15.60:0/3565857342 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56255f35fc70 con 0x56255f472f50 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff49affd640 10 client.17838 handle_command_reply: tid=0 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff4c4813740 1 client.17838 shutdown 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 msgr2=0x56255f401820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x56255f3efa40 tx=0x7ff4ac009420 comp rx=0 tx=0).stop 2024-07-24T21:57:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 shutdown_connections 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.427+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56255f472f50 0x56255f475370 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56255f3f9170 0x56255f3fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56255f3ed1e0 0x56255f3fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff49affd640 0 client.17838 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 --2- 172.21.15.60:0/3565857342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56255f3ec8c0 0x56255f401820 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 >> 172.21.15.60:0/3565857342 conn(0x56255f3ea6f0 msgr2=0x56255f3f0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 shutdown_connections 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 1 -- 172.21.15.60:0/3565857342 wait complete. 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c4813740 20 client.17838 trim_cache size 0 max 16384 2024-07-24T21:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:57:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:57:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:57:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 msgr2=0x7ff4a40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 0x7ff4a40b6200 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0031440 tx=0x7ff4b0005780 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 msgr2=0x7ff4bc19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400eb10 tx=0x7ff4b400c2d0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 shutdown_connections 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff4a40b3d40 0x7ff4a40b6200 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4bc108ac0 0x7ff4bc19e500 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc108140 0x7ff4bc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 --2- 172.21.15.60:0/1121338805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4bc102740 0x7ff4bc198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 >> 172.21.15.60:0/1121338805 conn(0x7ff4bc0fe180 msgr2=0x7ff4bc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.428+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 shutdown_connections 2024-07-24T21:57:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:57:52.429+0000 7ff4c1abe640 1 -- 172.21.15.60:0/1121338805 wait complete. 2024-07-24T21:57:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:53 smithi123 ceph-mon[37143]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 4.24k op/s 2024-07-24T21:57:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:53 smithi086 ceph-mon[37128]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 4.24k op/s 2024-07-24T21:57:54.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:53 smithi060 ceph-mon[33000]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 4.24k op/s 2024-07-24T21:57:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:55 smithi123 ceph-mon[37143]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 3.41k op/s 2024-07-24T21:57:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:55 smithi086 ceph-mon[37128]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 3.41k op/s 2024-07-24T21:57:56.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:55 smithi060 ceph-mon[33000]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 3.41k op/s 2024-07-24T21:57:56.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:56 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:57:56.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:56 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:57:56.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:56 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:57:57.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:56 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:57:57.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:56 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:57:57.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:56 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:57:57.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:56 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:57:57.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:56 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:57:57.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:56 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 30, 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "85fbd568": { 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 30, 2024-07-24T21:57:57.766 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 130.0, 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:57.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:57.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:57.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:57.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:57:57.768 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:57:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa762ffd640 1 -- 172.21.15.60:0/1797848772 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2d910b090 con 0x55d2d92277b0 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa762ffd640 10 client.27233 handle_command_reply: tid=0 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 client.27233 shutdown 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 msgr2=0x55d2d91ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7740a72c0 tx=0x7fa7740a7170 comp rx=0 tx=0).stop 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 shutdown_connections 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d2d92277b0 0x55d2d9229bd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2d91a35d0 0x55d2d91a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2d9198d00 0x55d2d91abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 --2- 172.21.15.60:0/1797848772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2d91977a0 0x55d2d91ab730 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa762ffd640 0 client.27233 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 >> 172.21.15.60:0/1797848772 conn(0x55d2d9161750 msgr2=0x55d2d91a22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 shutdown_connections 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 1 -- 172.21.15.60:0/1797848772 wait complete. 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.763+0000 7fa78bdd1740 20 client.27233 trim_cache size 0 max 16384 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 msgr2=0x7fa76c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 0x7fa76c0b6200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006010 tx=0x7fa77c0058d0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 msgr2=0x7fa78419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa77400eb90 tx=0x7fa77400c6a0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 shutdown_connections 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa76c0b3d40 0x7fa76c0b6200 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78410cac0 0x7fa7841a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f80 0x7fa78419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 --2- 172.21.15.60:0/2540301546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa784103660 0x7fa78419ae40 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.764+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 >> 172.21.15.60:0/2540301546 conn(0x7fa7840fd0c0 msgr2=0x7fa7840fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:57:57.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.765+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 shutdown_connections 2024-07-24T21:57:57.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:57:57.765+0000 7fa789a20640 1 -- 172.21.15.60:0/2540301546 wait complete. 2024-07-24T21:57:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:57 smithi123 ceph-mon[37143]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 2.89k op/s 2024-07-24T21:57:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:57 smithi086 ceph-mon[37128]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 2.89k op/s 2024-07-24T21:57:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:57 smithi060 ceph-mon[33000]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 2.89k op/s 2024-07-24T21:57:58.705 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '85fbd568', quiesce took 16.7 seconds. Will release after: -6.699999999999999 2024-07-24T21:57:58.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:57:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:57:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:57:59 smithi123 ceph-mon[37143]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-07-24T21:58:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:57:59 smithi086 ceph-mon[37128]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-07-24T21:58:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:57:59 smithi060 ceph-mon[33000]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 1.13k op/s 2024-07-24T21:58:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 -- 172.21.15.60:0/4101801732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c108b20 msgr2=0x7f124c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 --2- 172.21.15.60:0/4101801732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c108b20 0x7f124c10f320 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1244009ca0 tx=0x7f124402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 -- 172.21.15.60:0/4101801732 shutdown_connections 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 --2- 172.21.15.60:0/4101801732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c108b20 0x7f124c10f320 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 --2- 172.21.15.60:0/4101801732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 0x7f124c1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 --2- 172.21.15.60:0/4101801732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c102790 0x7f124c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 -- 172.21.15.60:0/4101801732 >> 172.21.15.60:0/4101801732 conn(0x7f124c0fe200 msgr2=0x7f124c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 -- 172.21.15.60:0/4101801732 shutdown_connections 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.663+0000 7f1252775640 1 -- 172.21.15.60:0/4101801732 wait complete. 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 Processor -- start 2024-07-24T21:58:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 -- start start 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 0x7f124c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 0x7f124c075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f124c112790 con 0x7f124c102790 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f124c112610 con 0x7f124c1081a0 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f1252775640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f124c112910 con 0x7f124c108b20 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f124bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f124bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42054/0 (socket says 172.21.15.60:42054) 2024-07-24T21:58:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.664+0000 7f124bfff640 1 -- 172.21.15.60:0/425889353 learned_addr learned my addr 172.21.15.60:0/425889353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124b7fe640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 0x7f124c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f1250d12640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 0x7f124c075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 -- 172.21.15.60:0/425889353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 msgr2=0x7f124c075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 0x7f124c075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 -- 172.21.15.60:0/425889353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 msgr2=0x7f124c0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 0x7f124c0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 -- 172.21.15.60:0/425889353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c1a7140 con 0x7f124c102790 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f1250d12640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 0x7f124c075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f124bfff640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d770 tx=0x7f123c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c017070 con 0x7f124c102790 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c0054d0 con 0x7f124c102790 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a7430 con 0x7f124c102790 2024-07-24T21:58:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c004c30 con 0x7f124c102790 2024-07-24T21:58:00.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.665+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a7920 con 0x7f124c102790 2024-07-24T21:58:00.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.666+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c102b70 con 0x7f124c102790 2024-07-24T21:58:00.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.668+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f123c004dd0 con 0x7f124c102790 2024-07-24T21:58:00.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.668+0000 7f12497fa640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 0x7f12340b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:00.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.669+0000 7f124b7fe640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 0x7f12340b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:00.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.669+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f123c0e3ef0 con 0x7f124c102790 2024-07-24T21:58:00.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.669+0000 7f124b7fe640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 0x7f12340b6200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f124c076980 tx=0x7f124000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.671+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f123c0ab1d0 con 0x7f124c102790 2024-07-24T21:58:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.907+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f124c076190 con 0x7f124c102790 2024-07-24T21:58:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.908+0000 7f12497fa640 1 -- 172.21.15.60:0/425889353 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f123c0aaad0 con 0x7f124c102790 2024-07-24T21:58:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:58:00.910 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:58:00.910 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:58:00.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 msgr2=0x7f12340b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:00.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 0x7f12340b6200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f124c076980 tx=0x7f124000b390 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 msgr2=0x7f124c074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d770 tx=0x7f123c00dc30 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 shutdown_connections 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12340b3d40 0x7f12340b6200 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c108b20 0x7f124c075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f124c1081a0 0x7f124c0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 --2- 172.21.15.60:0/425889353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c102790 0x7f124c074e90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 >> 172.21.15.60:0/425889353 conn(0x7f124c0fe200 msgr2=0x7f124c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 shutdown_connections 2024-07-24T21:58:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:00.910+0000 7f1252775640 1 -- 172.21.15.60:0/425889353 wait complete. 2024-07-24T21:58:01.533 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 85fbd568 --release --await' 2024-07-24T21:58:01.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 85fbd568 --release --await 2024-07-24T21:58:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:01 smithi060 ceph-mon[33000]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 412 op/s 2024-07-24T21:58:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:01 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/425889353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:01 smithi123 ceph-mon[37143]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 412 op/s 2024-07-24T21:58:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:01 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/425889353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:01 smithi086 ceph-mon[37128]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 412 op/s 2024-07-24T21:58:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:01 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/425889353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:03.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 -- 172.21.15.60:0/322158260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108170 msgr2=0x7f11f01085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 --2- 172.21.15.60:0/322158260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108170 0x7f11f01085b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402a410 comp rx=0 tx=0).stop 2024-07-24T21:58:03.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 -- 172.21.15.60:0/322158260 shutdown_connections 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 --2- 172.21.15.60:0/322158260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108af0 0x7f11f010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 --2- 172.21.15.60:0/322158260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108170 0x7f11f01085b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 --2- 172.21.15.60:0/322158260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 0x7f11f0102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 -- 172.21.15.60:0/322158260 >> 172.21.15.60:0/322158260 conn(0x7f11f00fe210 msgr2=0x7f11f0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 -- 172.21.15.60:0/322158260 shutdown_connections 2024-07-24T21:58:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.452+0000 7f11f6887640 1 -- 172.21.15.60:0/322158260 wait complete. 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 Processor -- start 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 -- start start 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 0x7f11f0074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 0x7f11f00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0112710 con 0x7f11f0108af0 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0112590 con 0x7f11f0108170 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f6887640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0112890 con 0x7f11f0102760 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f4e24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 0x7f11f00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f4e24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42066/0 (socket says 172.21.15.60:42066) 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11f4e24640 1 -- 172.21.15.60:0/717003221 learned_addr learned my addr 172.21.15.60:0/717003221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.453+0000 7f11effff640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 0x7f11f0074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 -- 172.21.15.60:0/717003221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 msgr2=0x7f11f0074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 0x7f11f0074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 -- 172.21.15.60:0/717003221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 msgr2=0x7f11f00753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 0x7f11f00753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 -- 172.21.15.60:0/717003221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f01a7040 con 0x7f11f0108af0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 0x7f11f00753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f4e24640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f11e800cce0 tx=0x7f11e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11ed7fa640 1 -- 172.21.15.60:0/717003221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e800f450 con 0x7f11f0108af0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f01a7330 con 0x7f11f0108af0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11ed7fa640 1 -- 172.21.15.60:0/717003221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e800faa0 con 0x7f11f0108af0 2024-07-24T21:58:03.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11ed7fa640 1 -- 172.21.15.60:0/717003221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8018b70 con 0x7f11f0108af0 2024-07-24T21:58:03.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.454+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01a77f0 con 0x7f11f0108af0 2024-07-24T21:58:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.457+0000 7f11ed7fa640 1 -- 172.21.15.60:0/717003221 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f11e8018d10 con 0x7f11f0108af0 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.457+0000 7f11f8c38740 1 Processor -- start 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.457+0000 7f11f8c38740 1 -- start start 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.457+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42bd2e0 0x561ff42bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11f8c38740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561ff4227dd0 con 0x561ff42b1350 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11f8c38740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ff42172d0 con 0x561ff42bd2e0 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11f8c38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ff421b310 con 0x561ff42b0a30 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42070/0 (socket says 172.21.15.60:42070) 2024-07-24T21:58:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43528/0 (socket says 172.21.15.60:43528) 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 -- 172.21.15.60:0/2734040643 learned_addr learned my addr 172.21.15.60:0/2734040643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42bd2e0 0x561ff42bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 -- 172.21.15.60:0/2734040643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 msgr2=0x561ff42b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 -- 172.21.15.60:0/2734040643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42bd2e0 msgr2=0x561ff42bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42bd2e0 0x561ff42bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11effff640 1 -- 172.21.15.60:0/2734040643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff3f77b00 con 0x561ff42b1350 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ed7fa640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 0x7f11d80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.458+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 0x7f11d80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.459+0000 7f11ed7fa640 1 -- 172.21.15.60:0/717003221 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f11e80e4430 con 0x7f11f0108af0 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.459+0000 7f11effff640 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f11e000ac10 tx=0x7f11e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcca82c5f3789233 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.459+0000 7f11d67fc640 1 -- 172.21.15.60:0/2734040643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e001d070 con 0x561ff42b1350 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.459+0000 7f11d67fc640 1 -- 172.21.15.60:0/2734040643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e0018af0 con 0x561ff42b1350 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.459+0000 7f11d67fc640 1 -- 172.21.15.60:0/2734040643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0010a90 con 0x561ff42b1350 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- 172.21.15.60:0/2734040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 msgr2=0x561ff42bcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 0x7f11d80b6200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f11e4031440 tx=0x7f11e4005e60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f11e000ac10 tx=0x7f11e000f790 comp rx=0 tx=0).stop 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- 172.21.15.60:0/2734040643 shutdown_connections 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42bd2e0 0x561ff42bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b1350 0x561ff42bcda0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2734040643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- 172.21.15.60:0/2734040643 >> 172.21.15.60:0/2734040643 conn(0x561ff42ae860 msgr2=0x561ff42afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- 172.21.15.60:0/2734040643 shutdown_connections 2024-07-24T21:58:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- 172.21.15.60:0/2734040643 wait complete. 2024-07-24T21:58:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.460+0000 7f11f8c38740 1 -- start start 2024-07-24T21:58:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 Processor -- start 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 0x561ff42bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561ff407abb0 con 0x561ff42b0a30 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ff41b9900 con 0x561ff42b1350 2024-07-24T21:58:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ff42bff60 con 0x561ff42bd2e0 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f8c38740 10 client.0 _ll_register_callbacks cb 0x561ff41c3c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42084/0 (socket says 172.21.15.60:42084) 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 -- 172.21.15.60:0/1367243302 learned_addr learned my addr 172.21.15.60:0/1367243302 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11f4e24640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 0x561ff42bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 -- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 msgr2=0x561ff42bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 0x561ff42bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 -- 172.21.15.60:0/1367243302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 msgr2=0x561ff42ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.461+0000 7f11effff640 1 -- 172.21.15.60:0/1367243302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff3fee960 con 0x561ff42b0a30 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11f4e24640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 0x561ff42bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11effff640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f11e0018670 tx=0x7f11e00186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b12c11d8c56323 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0027020 con 0x561ff42b0a30 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f11e0005040 con 0x561ff42b0a30 2024-07-24T21:58:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e0018af0 con 0x561ff42b0a30 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f11e0032050 con 0x561ff42b0a30 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11f8c38740 10 client.17856 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561ff42cba70 con 0x561ff42b0a30 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.462+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e0018af0 con 0x561ff42b0a30 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f11e0010cb0 con 0x561ff42b0a30 2024-07-24T21:58:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 10 client.17856 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 10 client.17856 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 20 client.17856 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff42cce70 0x561ff42cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 4 client.17856 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff4229d20 con 0x561ff42cce70 2024-07-24T21:58:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff42cce70 0x561ff42cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff42cce70 0x561ff42cf260 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f29a9c95418a75 server_cookie=5224547055b754e0 in_seq=0 out_seq=0 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.463+0000 7f11cf7fe640 10 client.17856 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1367243302 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ff4229d20 con 0x561ff42cce70 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11cf7fe640 10 client.17856 handle_command_reply: tid=0 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11f8c38740 1 client.17856 shutdown 2024-07-24T21:58:03.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 msgr2=0x561ff42ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f11e0018670 tx=0x7f11e00186a0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 shutdown_connections 2024-07-24T21:58:03.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.464+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff42cce70 0x561ff42cf260 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42bd2e0 0x561ff42bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1367243302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42b0a30 0x561ff42ba200 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 >> 172.21.15.60:0/1367243302 conn(0x561ff42ae860 msgr2=0x561ff42b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11cf7fe640 0 client.17856 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 shutdown_connections 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 1 -- 172.21.15.60:0/1367243302 wait complete. 2024-07-24T21:58:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.465+0000 7f11f8c38740 20 client.17856 trim_cache size 0 max 16384 2024-07-24T21:58:03.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.483+0000 7f11f8c38740 1 Processor -- start 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.483+0000 7f11f8c38740 1 -- start start 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.483+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.483+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.483+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11f8c38740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01b2c80 con 0x561ff42bd2e0 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11f8c38740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01b0490 con 0x561ff42b1350 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11f8c38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01ae300 con 0x561ff42b0a30 2024-07-24T21:58:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43546/0 (socket says 172.21.15.60:43546) 2024-07-24T21:58:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 -- 172.21.15.60:0/2513968698 learned_addr learned my addr 172.21.15.60:0/2513968698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11f4e24640 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 -- 172.21.15.60:0/2513968698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 msgr2=0x561ff42b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 -- 172.21.15.60:0/2513968698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 msgr2=0x561ff42d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42d48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.484+0000 7f11effff640 1 -- 172.21.15.60:0/2513968698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff42b5b10 con 0x561ff42b0a30 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.485+0000 7f11effff640 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a6b0 tx=0x7f11e000af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f13a72d0a3c872 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.485+0000 7f11d5ffb640 1 -- 172.21.15.60:0/2513968698 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e003b070 con 0x561ff42b0a30 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.485+0000 7f11d5ffb640 1 -- 172.21.15.60:0/2513968698 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e00048d0 con 0x561ff42b0a30 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.485+0000 7f11d5ffb640 1 -- 172.21.15.60:0/2513968698 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0027070 con 0x561ff42b0a30 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- 172.21.15.60:0/2513968698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 msgr2=0x561ff42b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a6b0 tx=0x7f11e000af10 comp rx=0 tx=0).stop 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- 172.21.15.60:0/2513968698 shutdown_connections 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 --2- 172.21.15.60:0/2513968698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42b4b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- 172.21.15.60:0/2513968698 >> 172.21.15.60:0/2513968698 conn(0x561ff42ae860 msgr2=0x561ff42bc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- 172.21.15.60:0/2513968698 shutdown_connections 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- 172.21.15.60:0/2513968698 wait complete. 2024-07-24T21:58:03.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.486+0000 7f11f8c38740 1 -- start start 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 Processor -- start 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01ae950 con 0x561ff42bd2e0 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01afdc0 con 0x561ff42b1350 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01af020 con 0x561ff42b0a30 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f8c38740 10 client.0 _ll_register_callbacks cb 0x7f11e40314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f4e24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f4e24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42104/0 (socket says 172.21.15.60:42104) 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11f4e24640 1 -- 172.21.15.60:0/1565214171 learned_addr learned my addr 172.21.15.60:0/1565214171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.487+0000 7f11effff640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 -- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 msgr2=0x561ff42c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 -- 172.21.15.60:0/1565214171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 msgr2=0x561ff42b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 -- 172.21.15.60:0/1565214171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff42b9160 con 0x561ff42bd2e0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11ef7fe640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f4e24640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f11e80a8220 tx=0x7f11e80aa720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3d3b475f35649 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e80c4070 con 0x561ff42bd2e0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7f11e80b3de0 con 0x561ff42bd2e0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e00048d0 con 0x561ff42bd2e0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f11e80aafb0 con 0x561ff42bd2e0 2024-07-24T21:58:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f8c38740 10 client.17862 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561ff4337950 con 0x561ff42bd2e0 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.488+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e00048d0 con 0x561ff42bd2e0 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f11e80af0e0 con 0x561ff42bd2e0 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 10 client.17862 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 10 client.17862 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 20 client.17862 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff4338130 0x561ff433a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 4 client.17862 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "85fbd568", "release": true, "await": true}] 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "85fbd568", "release": true, "await": true}) -- 0x561ff42257c0 con 0x561ff4338130 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11effff640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff4338130 0x561ff433a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11effff640 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff4338130 0x561ff433a550 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7077ff9c6a7eb459 server_cookie=9573213d1d4c3779 in_seq=0 out_seq=0 2024-07-24T21:58:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:03.489+0000 7f11cf7fe640 10 client.17862 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:03 smithi060 ceph-mon[33000]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-07-24T21:58:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:03 smithi123 ceph-mon[37143]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-07-24T21:58:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:03 smithi086 ceph-mon[37128]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-07-24T21:58:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11cf7fe640 1 -- 172.21.15.60:0/1565214171 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561ff42257c0 con 0x561ff4338130 2024-07-24T21:58:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11cf7fe640 10 client.17862 handle_command_reply: tid=0 2024-07-24T21:58:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 client.17862 shutdown 2024-07-24T21:58:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 msgr2=0x561ff42b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f11e80a8220 tx=0x7f11e80aa720 comp rx=0 tx=0).stop 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 shutdown_connections 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ff4338130 0x561ff433a550 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ff42bd2e0 0x561ff42b77d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff42b1350 0x561ff42b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 --2- 172.21.15.60:0/1565214171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ff42b0a30 0x561ff42c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 >> 172.21.15.60:0/1565214171 conn(0x561ff42ae860 msgr2=0x561ff42bc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11cf7fe640 0 client.17862 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 shutdown_connections 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.565+0000 7f11f8c38740 1 -- 172.21.15.60:0/1565214171 wait complete. 2024-07-24T21:58:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f8c38740 20 client.17862 trim_cache size 0 max 16384 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 32, 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "85fbd568": { 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 32, 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:58:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 130.0, 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 msgr2=0x7f11d80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 0x7f11d80b6200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f11e4031440 tx=0x7f11e4005e60 comp rx=0 tx=0).stop 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 msgr2=0x7f11f0075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f11e800cce0 tx=0x7f11e800a620 comp rx=0 tx=0).stop 2024-07-24T21:58:04.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 shutdown_connections 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f11d80b3d40 0x7f11d80b6200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.566+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11f0108af0 0x7f11f0075910 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.567+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f0108170 0x7f11f00753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.567+0000 7f11f6887640 1 --2- 172.21.15.60:0/717003221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11f0102760 0x7f11f0074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.567+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 >> 172.21.15.60:0/717003221 conn(0x7f11f00fe210 msgr2=0x7f11f0100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.567+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 shutdown_connections 2024-07-24T21:58:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:04.567+0000 7f11f6887640 1 -- 172.21.15.60:0/717003221 wait complete. 2024-07-24T21:58:05.240 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '85fbd568', total seconds elapsed: 23.2 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 -- 172.21.15.60:0/3397904342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484075850 msgr2=0x7fb484112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 --2- 172.21.15.60:0/3397904342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484075850 0x7fb484112ec0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb478009900 tx=0x7fb47802a410 comp rx=0 tx=0).stop 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 -- 172.21.15.60:0/3397904342 shutdown_connections 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 --2- 172.21.15.60:0/3397904342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484113400 0x7fb4841157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 --2- 172.21.15.60:0/3397904342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484075850 0x7fb484112ec0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 --2- 172.21.15.60:0/3397904342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb484074f30 0x7fb484075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 -- 172.21.15.60:0/3397904342 >> 172.21.15.60:0/3397904342 conn(0x7fb4840fe0f0 msgr2=0x7fb484100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 -- 172.21.15.60:0/3397904342 shutdown_connections 2024-07-24T21:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.563+0000 7fb489c8d640 1 -- 172.21.15.60:0/3397904342 wait complete. 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 Processor -- start 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 -- start start 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb484075850 0x7fb48419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 0x7fb4841a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb484118cf0 con 0x7fb484113400 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb484118b70 con 0x7fb484074f30 2024-07-24T21:58:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.564+0000 7fb489c8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb484118e70 con 0x7fb484075850 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46230/0 (socket says 172.21.15.60:46230) 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb483fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 0x7fb4841a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 -- 172.21.15.60:0/1307467035 learned_addr learned my addr 172.21.15.60:0/1307467035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 -- 172.21.15.60:0/1307467035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb484075850 msgr2=0x7fb48419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb484075850 0x7fb48419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 -- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 msgr2=0x7fb4841a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 0x7fb4841a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb4837fe640 1 -- 172.21.15.60:0/1307467035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4841a5370 con 0x7fb484074f30 2024-07-24T21:58:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.565+0000 7fb483fff640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 0x7fb4841a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb4837fe640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b4a0 tx=0x7fb47400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb480ff9640 1 -- 172.21.15.60:0/1307467035 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474014070 con 0x7fb484074f30 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb480ff9640 1 -- 172.21.15.60:0/1307467035 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474010ac0 con 0x7fb484074f30 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4841a5660 con 0x7fb484074f30 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb480ff9640 1 -- 172.21.15.60:0/1307467035 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474014070 con 0x7fb484074f30 2024-07-24T21:58:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.566+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841abd20 con 0x7fb484074f30 2024-07-24T21:58:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.569+0000 7fb480ff9640 1 -- 172.21.15.60:0/1307467035 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fb474017070 con 0x7fb484074f30 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 Processor -- start 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 -- start start 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0247ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0253c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0f01bd280 con 0x55b0f0254170 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0f01ac780 con 0x55b0f02481e0 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0f01b07c0 con 0x55b0f02478c0 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb480ff9640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 0x7fb46c0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0253c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb483fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0247ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42134/0 (socket says 172.21.15.60:42134) 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0253c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46234/0 (socket says 172.21.15.60:46234) 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb482ffd640 1 -- 172.21.15.60:0/3118235536 learned_addr learned my addr 172.21.15.60:0/3118235536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.570+0000 7fb483fff640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 0x7fb46c0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 -- 172.21.15.60:0/3118235536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 msgr2=0x55b0f0247ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0247ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 -- 172.21.15.60:0/3118235536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 msgr2=0x55b0f0253c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0253c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 -- 172.21.15.60:0/3118235536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0eff10790 con 0x55b0f0254170 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb480ff9640 1 -- 172.21.15.60:0/1307467035 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fb4740e32b0 con 0x7fb484074f30 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb483fff640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 0x7fb46c0b6400 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005c40 tx=0x7fb47c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.571+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb478009a20 tx=0x7fb478000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c676fbd2496ea server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.572+0000 7fb469ffb640 1 -- 172.21.15.60:0/3118235536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47803e030 con 0x55b0f0254170 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.572+0000 7fb469ffb640 1 -- 172.21.15.60:0/3118235536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478032d60 con 0x55b0f0254170 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.572+0000 7fb469ffb640 1 -- 172.21.15.60:0/3118235536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478035040 con 0x55b0f0254170 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.573+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3118235536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 msgr2=0x55b0f0256560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.573+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb478009a20 tx=0x7fb478000bc0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.573+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3118235536 shutdown_connections 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.573+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f0256560 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.573+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0253c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.574+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3118235536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0247ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.574+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3118235536 >> 172.21.15.60:0/3118235536 conn(0x55b0f02456f0 msgr2=0x55b0f0246890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.576+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3118235536 shutdown_connections 2024-07-24T21:58:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.576+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3118235536 wait complete. 2024-07-24T21:58:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.576+0000 7fb48c9e4740 1 -- start start 2024-07-24T21:58:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 Processor -- start 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f02538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f0254170 0x55b0f0251300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b0f00e6ab0 con 0x55b0f02478c0 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0f01349e0 con 0x55b0f0254170 2024-07-24T21:58:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0f0256df0 con 0x55b0f02481e0 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.577+0000 7fb48c9e4740 10 client.0 _ll_register_callbacks cb 0x55b0efebca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f02538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43582/0 (socket says 172.21.15.60:43582) 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 -- 172.21.15.60:0/3848329932 learned_addr learned my addr 172.21.15.60:0/3848329932 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f02538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42140/0 (socket says 172.21.15.60:42140) 2024-07-24T21:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb483fff640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f0254170 0x55b0f0251300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 -- 172.21.15.60:0/3848329932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f0254170 msgr2=0x55b0f0251300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f0254170 0x55b0f0251300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 -- 172.21.15.60:0/3848329932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 msgr2=0x55b0f02538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f02538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 -- 172.21.15.60:0/3848329932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0eff89620 con 0x55b0f02481e0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb482ffd640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb4780099f0 tx=0x7fb478009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7781704d47e3e1bb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.578+0000 7fb462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47803e040 con 0x55b0f02481e0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fb478031720 con 0x55b0f02481e0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0f015bce0 con 0x55b0f02481e0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb478055050 con 0x55b0f02481e0 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb48c9e4740 10 client.27238 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.579+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b0f02625a0 con 0x55b0f02481e0 2024-07-24T21:58:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b0f015bce0 con 0x55b0f02481e0 2024-07-24T21:58:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb478004bc0 con 0x55b0f02481e0 2024-07-24T21:58:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 10 client.27238 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 10 client.27238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 20 client.27238 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02638f0 0x55b0f0265ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 4 client.27238 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f01bf1d0 con 0x55b0f02638f0 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.580+0000 7fb4837fe640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02638f0 0x55b0f0265ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.581+0000 7fb4837fe640 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02638f0 0x55b0f0265ce0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d101cdd156db19f0 server_cookie=eef6d2ffe155c541 in_seq=0 out_seq=0 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.581+0000 7fb462ffd640 10 client.27238 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb462ffd640 1 -- 172.21.15.60:0/3848329932 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0f01bf1d0 con 0x55b0f02638f0 2024-07-24T21:58:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb462ffd640 10 client.27238 handle_command_reply: tid=0 2024-07-24T21:58:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 client.27238 shutdown 2024-07-24T21:58:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 msgr2=0x55b0f0250dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb4780099f0 tx=0x7fb478009540 comp rx=0 tx=0).stop 2024-07-24T21:58:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 shutdown_connections 2024-07-24T21:58:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02638f0 0x55b0f0265ce0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f0254170 0x55b0f0251300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb462ffd640 0 client.27238 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02481e0 0x55b0f0250dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.582+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/3848329932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f02538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.583+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 >> 172.21.15.60:0/3848329932 conn(0x55b0f02456f0 msgr2=0x55b0f024b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.583+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 shutdown_connections 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.583+0000 7fb48c9e4740 1 -- 172.21.15.60:0/3848329932 wait complete. 2024-07-24T21:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.583+0000 7fb48c9e4740 20 client.27238 trim_cache size 0 max 16384 2024-07-24T21:58:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 Processor -- start 2024-07-24T21:58:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 -- start start 2024-07-24T21:58:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0246ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f026b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b72f0 con 0x55b0f0254170 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b4b00 con 0x55b0f02481e0 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b2970 con 0x55b0f02478c0 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb482ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46254/0 (socket says 172.21.15.60:46254) 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb482ffd640 1 -- 172.21.15.60:0/245643030 learned_addr learned my addr 172.21.15.60:0/245643030 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.605+0000 7fb483fff640 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f026b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb4837fe640 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0246ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 -- 172.21.15.60:0/245643030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 msgr2=0x55b0f0246ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0246ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 -- 172.21.15.60:0/245643030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 msgr2=0x55b0f026b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f026b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 -- 172.21.15.60:0/245643030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f0266050 con 0x55b0f02481e0 2024-07-24T21:58:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb482ffd640 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb478009900 tx=0x7fb478032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa38054bdefa915 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb4697fa640 1 -- 172.21.15.60:0/245643030 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478052070 con 0x55b0f02481e0 2024-07-24T21:58:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb4697fa640 1 -- 172.21.15.60:0/245643030 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478049070 con 0x55b0f02481e0 2024-07-24T21:58:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.606+0000 7fb4697fa640 1 -- 172.21.15.60:0/245643030 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47803e070 con 0x55b0f02481e0 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 -- 172.21.15.60:0/245643030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 msgr2=0x55b0f026b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb478009900 tx=0x7fb478032f80 comp rx=0 tx=0).stop 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 -- 172.21.15.60:0/245643030 shutdown_connections 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f0254170 0x55b0f026b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f026b280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/245643030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f02478c0 0x55b0f0246ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 -- 172.21.15.60:0/245643030 >> 172.21.15.60:0/245643030 conn(0x55b0f02456f0 msgr2=0x55b0f0246790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 -- 172.21.15.60:0/245643030 shutdown_connections 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.607+0000 7fb48c9e4740 1 -- 172.21.15.60:0/245643030 wait complete. 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 -- start start 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 Processor -- start 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0259540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f0254170 0x55b0f0259a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b2fc0 con 0x55b0f02478c0 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.608+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b4430 con 0x55b0f02481e0 2024-07-24T21:58:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb48c9e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841b3690 con 0x55b0f0254170 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42160/0 (socket says 172.21.15.60:42160) 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 -- 172.21.15.60:0/2158747253 learned_addr learned my addr 172.21.15.60:0/2158747253 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb48c9e4740 10 client.0 _ll_register_callbacks cb 0x7fb4740a5c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb482ffd640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0259540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb483fff640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f0254170 0x55b0f0259a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 -- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f0254170 msgr2=0x55b0f0259a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f0254170 0x55b0f0259a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 -- 172.21.15.60:0/2158747253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 msgr2=0x55b0f0259540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0259540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 -- 172.21.15.60:0/2158747253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f02cc3d0 con 0x55b0f02478c0 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb482ffd640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0259540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb4837fe640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b470 tx=0x7fb474010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140a90f7b33d25e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb462ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47401e040 con 0x55b0f02478c0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8565+0+0 (secure 0 0 0) 0x7fb4740e0c50 con 0x55b0f02478c0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478049070 con 0x55b0f02478c0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb4740a9fa0 con 0x55b0f02478c0 2024-07-24T21:58:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.609+0000 7fb48c9e4740 10 client.17880 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b0f02cd740 con 0x55b0f02478c0 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4740ae5a0 con 0x55b0f02478c0 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb4740a9200 con 0x55b0f02478c0 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 10 client.17880 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 20 client.17880 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02cdf20 0x55b0f02d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 4 client.17880 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb478049070 con 0x55b0f02cdf20 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.610+0000 7fb482ffd640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02cdf20 0x55b0f02d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.611+0000 7fb462ffd640 10 client.17880 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.611+0000 7fb482ffd640 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02cdf20 0x55b0f02d0340 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3dc994e603a41e0 server_cookie=27f178141c41280 in_seq=0 out_seq=0 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb462ffd640 1 -- 172.21.15.60:0/2158747253 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb478049070 con 0x55b0f02cdf20 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb462ffd640 10 client.17880 handle_command_reply: tid=0 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 client.17880 shutdown 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 msgr2=0x55b0f0259000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b470 tx=0x7fb474010f50 comp rx=0 tx=0).stop 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 shutdown_connections 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b0f02cdf20 0x55b0f02d0340 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0f0254170 0x55b0f0259a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb462ffd640 0 client.17880 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f02481e0 0x55b0f0259540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 --2- 172.21.15.60:0/2158747253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0f02478c0 0x55b0f0259000 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 >> 172.21.15.60:0/2158747253 conn(0x55b0f02456f0 msgr2=0x55b0f0252230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 shutdown_connections 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 1 -- 172.21.15.60:0/2158747253 wait complete. 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.620+0000 7fb48c9e4740 20 client.17880 trim_cache size 0 max 16384 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:58:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:58:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:58:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 msgr2=0x7fb46c0b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 0x7fb46c0b6400 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005c40 tx=0x7fb47c009500 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 msgr2=0x7fb48419f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b4a0 tx=0x7fb47400b960 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 shutdown_connections 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb46c0b3f40 0x7fb46c0b6400 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb484113400 0x7fb4841a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb484075850 0x7fb48419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 --2- 172.21.15.60:0/1307467035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb484074f30 0x7fb48419f2d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 >> 172.21.15.60:0/1307467035 conn(0x7fb4840fe0f0 msgr2=0x7fb4840fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 shutdown_connections 2024-07-24T21:58:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:05.621+0000 7fb489c8d640 1 -- 172.21.15.60:0/1307467035 wait complete. 2024-07-24T21:58:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:05 smithi123 ceph-mon[37143]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 837 KiB/s wr, 103 op/s 2024-07-24T21:58:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:05 smithi086 ceph-mon[37128]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 837 KiB/s wr, 103 op/s 2024-07-24T21:58:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:05 smithi060 ceph-mon[33000]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 837 KiB/s wr, 103 op/s 2024-07-24T21:58:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:07 smithi123 ceph-mon[37143]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 99 op/s 2024-07-24T21:58:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:07 smithi123 ceph-mon[37143]: osdmap e100: 12 total, 12 up, 12 in 2024-07-24T21:58:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:07 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:58:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:07 smithi086 ceph-mon[37128]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 99 op/s 2024-07-24T21:58:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:07 smithi086 ceph-mon[37128]: osdmap e100: 12 total, 12 up, 12 in 2024-07-24T21:58:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:07 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:58:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:07 smithi060 ceph-mon[33000]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 925 KiB/s wr, 99 op/s 2024-07-24T21:58:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:07 smithi060 ceph-mon[33000]: osdmap e100: 12 total, 12 up, 12 in 2024-07-24T21:58:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:07 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:58:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:58:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:58:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:58:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:58:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:58:09.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:58:09.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:58:09.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:58:09.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:58:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:09 smithi123 ceph-mon[37143]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 676 KiB/s wr, 54 op/s 2024-07-24T21:58:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:09 smithi086 ceph-mon[37128]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 676 KiB/s wr, 54 op/s 2024-07-24T21:58:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:09 smithi060 ceph-mon[33000]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 676 KiB/s wr, 54 op/s 2024-07-24T21:58:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:11 smithi123 ceph-mon[37143]: pgmap v735: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 975 KiB/s wr, 105 op/s 2024-07-24T21:58:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:11 smithi086 ceph-mon[37128]: pgmap v735: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 975 KiB/s wr, 105 op/s 2024-07-24T21:58:12.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:11 smithi060 ceph-mon[33000]: pgmap v735: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 975 KiB/s wr, 105 op/s 2024-07-24T21:58:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:13 smithi086 ceph-mon[37128]: pgmap v736: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 978 KiB/s wr, 136 op/s 2024-07-24T21:58:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:13 smithi060 ceph-mon[33000]: pgmap v736: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 978 KiB/s wr, 136 op/s 2024-07-24T21:58:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:13 smithi123 ceph-mon[37143]: pgmap v736: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 978 KiB/s wr, 136 op/s 2024-07-24T21:58:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:15 smithi086 ceph-mon[37128]: pgmap v737: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-07-24T21:58:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:15 smithi060 ceph-mon[33000]: pgmap v737: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-07-24T21:58:16.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:15 smithi123 ceph-mon[37143]: pgmap v737: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-07-24T21:58:17.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:58:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:17 smithi086 ceph-mon[37128]: pgmap v738: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-07-24T21:58:18.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:17 smithi060 ceph-mon[33000]: pgmap v738: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-07-24T21:58:18.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:17 smithi123 ceph-mon[37143]: pgmap v738: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-07-24T21:58:18.384 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.780+0000 7fe508dd9640 1 -- 172.21.15.60:0/840560419 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40308b0 con 0x7fe504107760 2024-07-24T21:58:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.780+0000 7fe50addd640 1 -- 172.21.15.60:0/840560419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe504107760 msgr2=0x7fe50410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.780+0000 7fe50addd640 1 --2- 172.21.15.60:0/840560419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe504107760 0x7fe50410df60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009860 tx=0x7fe4f402a380 comp rx=0 tx=0).stop 2024-07-24T21:58:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.782+0000 7fe50addd640 1 -- 172.21.15.60:0/840560419 shutdown_connections 2024-07-24T21:58:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.782+0000 7fe50addd640 1 --2- 172.21.15.60:0/840560419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe504107760 0x7fe50410df60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.782+0000 7fe50addd640 1 --2- 172.21.15.60:0/840560419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504106de0 0x7fe504107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.782+0000 7fe50addd640 1 --2- 172.21.15.60:0/840560419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5041013e0 0x7fe5041017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.782+0000 7fe50addd640 1 -- 172.21.15.60:0/840560419 >> 172.21.15.60:0/840560419 conn(0x7fe5040fce00 msgr2=0x7fe5040ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.786+0000 7fe50addd640 1 -- 172.21.15.60:0/840560419 shutdown_connections 2024-07-24T21:58:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.786+0000 7fe50addd640 1 -- 172.21.15.60:0/840560419 wait complete. 2024-07-24T21:58:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.787+0000 7fe50addd640 1 Processor -- start 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 -- start start 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe504106de0 0x7fe50406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504107760 0x7fe504072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe50406ca10 con 0x7fe504106de0 2024-07-24T21:58:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe50406c890 con 0x7fe504107760 2024-07-24T21:58:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.789+0000 7fe50addd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe50406cb90 con 0x7fe5041013e0 2024-07-24T21:58:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.790+0000 7fe50a5dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504107760 0x7fe504072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.790+0000 7fe509ddb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.790+0000 7fe509ddb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42556/0 (socket says 172.21.15.60:42556) 2024-07-24T21:58:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.790+0000 7fe509ddb640 1 -- 172.21.15.60:0/2004997839 learned_addr learned my addr 172.21.15.60:0/2004997839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 -- 172.21.15.60:0/2004997839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504107760 msgr2=0x7fe504072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504107760 0x7fe504072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 -- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe504106de0 msgr2=0x7fe50406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe504106de0 0x7fe50406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 -- 172.21.15.60:0/2004997839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5040732e0 con 0x7fe5041013e0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe509ddb640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009fa0 tx=0x7fe4fc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe4faffd640 1 -- 172.21.15.60:0/2004997839 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc01b070 con 0x7fe5041013e0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5040735d0 con 0x7fe5041013e0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.791+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe504073e90 con 0x7fe5041013e0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.792+0000 7fe4faffd640 1 -- 172.21.15.60:0/2004997839 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc00ba20 con 0x7fe5041013e0 2024-07-24T21:58:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.792+0000 7fe4faffd640 1 -- 172.21.15.60:0/2004997839 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc016bf0 con 0x7fe5041013e0 2024-07-24T21:58:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.796+0000 7fe50ed94740 1 Processor -- start 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 -- start start 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9c9aa0 0x55696e9c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55696e93f280 con 0x55696e9ca3c0 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55696e92e780 con 0x55696e9c9aa0 2024-07-24T21:58:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.798+0000 7fe50ed94740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55696e9327c0 con 0x55696e9d6350 2024-07-24T21:58:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe50a5dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9c9aa0 0x55696e9c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42572/0 (socket says 172.21.15.60:42572) 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 -- 172.21.15.60:0/755439641 learned_addr learned my addr 172.21.15.60:0/755439641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 -- 172.21.15.60:0/755439641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9c9aa0 msgr2=0x55696e9c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9c9aa0 0x55696e9c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 -- 172.21.15.60:0/755439641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 msgr2=0x55696e9d5e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9d5e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 -- 172.21.15.60:0/755439641 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696e692790 con 0x55696e9d6350 2024-07-24T21:58:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.799+0000 7fe5095da640 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe50000a9c0 tx=0x7fe50000ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bca6a1a5ccefbca server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.801+0000 7fe4effff640 1 -- 172.21.15.60:0/755439641 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50000cde0 con 0x55696e9d6350 2024-07-24T21:58:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.801+0000 7fe4effff640 1 -- 172.21.15.60:0/755439641 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50000f900 con 0x55696e9d6350 2024-07-24T21:58:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 -- 172.21.15.60:0/755439641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 msgr2=0x55696e9d8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe50000a9c0 tx=0x7fe50000ae80 comp rx=0 tx=0).stop 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe4effff640 1 -- 172.21.15.60:0/755439641 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5000189e0 con 0x55696e9d6350 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 -- 172.21.15.60:0/755439641 shutdown_connections 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9d6350 0x55696e9d8740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 --2- 172.21.15.60:0/755439641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9c9aa0 0x55696e9c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.802+0000 7fe50ed94740 1 -- 172.21.15.60:0/755439641 >> 172.21.15.60:0/755439641 conn(0x55696e9c78d0 msgr2=0x55696e9c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.803+0000 7fe50ed94740 1 -- 172.21.15.60:0/755439641 shutdown_connections 2024-07-24T21:58:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.805+0000 7fe4faffd640 1 -- 172.21.15.60:0/2004997839 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe4fc01e070 con 0x7fe5041013e0 2024-07-24T21:58:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.806+0000 7fe50ed94740 1 -- 172.21.15.60:0/755439641 wait complete. 2024-07-24T21:58:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.806+0000 7fe4faffd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 0x7fe4f00b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.806+0000 7fe50ed94740 1 -- start start 2024-07-24T21:58:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.806+0000 7fe4faffd640 1 -- 172.21.15.60:0/2004997839 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fe4fc0e3f30 con 0x7fe5041013e0 2024-07-24T21:58:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.806+0000 7fe50ed94740 1 Processor -- start 2024-07-24T21:58:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe5095da640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 0x7fe4f00b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe50ed94740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55696e868ab0 con 0x55696e9c9aa0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.807+0000 7fe50ed94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55696e8b69e0 con 0x55696e9d6350 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe50ed94740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55696e9d9080 con 0x55696e9ca3c0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe50ed94740 10 client.0 _ll_register_callbacks cb 0x55696e63ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9d2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42588/0 (socket says 172.21.15.60:42588) 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe5095da640 1 -- 172.21.15.60:0/1247902563 learned_addr learned my addr 172.21.15.60:0/1247902563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe5095da640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 0x7fe4f00b61f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe50000a750 tx=0x7fe5000046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.808+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.809+0000 7fe509ddb640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.810+0000 7fe50a5dc640 1 -- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 msgr2=0x55696e9d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.810+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.810+0000 7fe50a5dc640 1 -- 172.21.15.60:0/1247902563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 msgr2=0x55696e9d5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.810+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9d5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.810+0000 7fe50a5dc640 1 -- 172.21.15.60:0/1247902563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696e70b620 con 0x55696e9d6350 2024-07-24T21:58:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.815+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9d3530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009780 tx=0x7fe4f4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9324ac2a4cb5e643 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.815+0000 7fe4ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.816+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4037040 con 0x55696e9d6350 2024-07-24T21:58:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.816+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fe4f4003da0 con 0x55696e9d6350 2024-07-24T21:58:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.817+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55696e8ddce0 con 0x55696e9d6350 2024-07-24T21:58:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.820+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe4f40324e0 con 0x55696e9d6350 2024-07-24T21:58:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.820+0000 7fe50ed94740 10 client.27299 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.820+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55696e9e4940 con 0x55696e9d6350 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55696e8ddce0 con 0x55696e9d6350 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe4f40306b0 con 0x55696e9d6350 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 10 client.27299 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 10 client.27299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 20 client.27299 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696e9e5f40 0x55696e9e8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 4 client.27299 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.821+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696e9411d0 con 0x55696e9e5f40 2024-07-24T21:58:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.824+0000 7fe509ddb640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696e9e5f40 0x55696e9e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.837+0000 7fe509ddb640 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696e9e5f40 0x55696e9e8380 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144e563674c325bc server_cookie=a0fb55742563c1d9 in_seq=0 out_seq=0 2024-07-24T21:58:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.837+0000 7fe4ed7fa640 10 client.27299 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.843+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/1247902563 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55696e9411d0 con 0x55696e9e5f40 2024-07-24T21:58:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.844+0000 7fe4ed7fa640 10 client.27299 handle_command_reply: tid=0 2024-07-24T21:58:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.844+0000 7fe50ed94740 1 client.27299 shutdown 2024-07-24T21:58:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.846+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 msgr2=0x55696e9d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.846+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9d3530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009780 tx=0x7fe4f4003820 comp rx=0 tx=0).stop 2024-07-24T21:58:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 shutdown_connections 2024-07-24T21:58:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696e9e5f40 0x55696e9e8380 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9d3530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 --2- 172.21.15.60:0/1247902563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9d5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 >> 172.21.15.60:0/1247902563 conn(0x55696e9c78d0 msgr2=0x55696e9cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.849+0000 7fe4ed7fa640 0 client.27299 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.851+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 shutdown_connections 2024-07-24T21:58:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.852+0000 7fe50ed94740 1 -- 172.21.15.60:0/1247902563 wait complete. 2024-07-24T21:58:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.852+0000 7fe50ed94740 20 client.27299 trim_cache size 0 max 16384 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.891+0000 7fe50ed94740 1 Processor -- start 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.891+0000 7fe50ed94740 1 -- start start 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9c9aa0 0x55696e9d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80113e0 con 0x55696e9ca3c0 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e800ebf0 con 0x55696e9d6350 2024-07-24T21:58:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50ed94740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e800ca60 con 0x55696e9c9aa0 2024-07-24T21:58:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe5095da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50a5dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50a5dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53644/0 (socket says 172.21.15.60:53644) 2024-07-24T21:58:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.892+0000 7fe50a5dc640 1 -- 172.21.15.60:0/4068161883 learned_addr learned my addr 172.21.15.60:0/4068161883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.893+0000 7fe509ddb640 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9c9aa0 0x55696e9d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 -- 172.21.15.60:0/4068161883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9c9aa0 msgr2=0x55696e9d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9c9aa0 0x55696e9d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 -- 172.21.15.60:0/4068161883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 msgr2=0x55696e9c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 -- 172.21.15.60:0/4068161883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696e9d3b10 con 0x55696e9d6350 2024-07-24T21:58:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.896+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a970 tx=0x7fe4f4003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db925d7b5e28685 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.898+0000 7fe4ef7fe640 1 -- 172.21.15.60:0/4068161883 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4037030 con 0x55696e9d6350 2024-07-24T21:58:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.898+0000 7fe4ef7fe640 1 -- 172.21.15.60:0/4068161883 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4038040 con 0x55696e9d6350 2024-07-24T21:58:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.898+0000 7fe4ef7fe640 1 -- 172.21.15.60:0/4068161883 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4005290 con 0x55696e9d6350 2024-07-24T21:58:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 -- 172.21.15.60:0/4068161883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 msgr2=0x55696e9edc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a970 tx=0x7fe4f4003890 comp rx=0 tx=0).stop 2024-07-24T21:58:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 -- 172.21.15.60:0/4068161883 shutdown_connections 2024-07-24T21:58:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9edc50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9ca3c0 0x55696e9c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 --2- 172.21.15.60:0/4068161883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9c9aa0 0x55696e9d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.899+0000 7fe50ed94740 1 -- 172.21.15.60:0/4068161883 >> 172.21.15.60:0/4068161883 conn(0x55696e9c78d0 msgr2=0x55696e9c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.900+0000 7fe50ed94740 1 -- 172.21.15.60:0/4068161883 shutdown_connections 2024-07-24T21:58:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.901+0000 7fe50ed94740 1 -- 172.21.15.60:0/4068161883 wait complete. 2024-07-24T21:58:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.901+0000 7fe50ed94740 1 -- start start 2024-07-24T21:58:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.901+0000 7fe50ed94740 1 Processor -- start 2024-07-24T21:58:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e800d0b0 con 0x55696e9c9aa0 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe5095da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9db3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42610/0 (socket says 172.21.15.60:42610) 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe5095da640 1 -- 172.21.15.60:0/598083226 learned_addr learned my addr 172.21.15.60:0/598083226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e800e520 con 0x55696e9d6350 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.902+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e800d780 con 0x55696e9ca3c0 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50ed94740 10 client.0 _ll_register_callbacks cb 0x7fe4fc009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50a5dc640 1 -- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 msgr2=0x55696e9db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50a5dc640 1 -- 172.21.15.60:0/598083226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 msgr2=0x55696e9de930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.903+0000 7fe50a5dc640 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696e9c7700 con 0x55696e9d6350 2024-07-24T21:58:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.904+0000 7fe50a5dc640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9db900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a970 tx=0x7fe4f4030720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5391180a673faeda server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.904+0000 7fe4ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.904+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4056070 con 0x55696e9d6350 2024-07-24T21:58:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.905+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4038040 con 0x55696e9d6350 2024-07-24T21:58:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.905+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fe4f4005440 con 0x55696e9d6350 2024-07-24T21:58:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.906+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe4f405f050 con 0x55696e9d6350 2024-07-24T21:58:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.906+0000 7fe50ed94740 10 client.27305 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.906+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55696ea4f8f0 con 0x55696e9d6350 2024-07-24T21:58:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.906+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4038040 con 0x55696e9d6350 2024-07-24T21:58:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe4f4059020 con 0x55696e9d6350 2024-07-24T21:58:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 10 client.27305 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 10 client.27305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 20 client.27305 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696ea4ff80 0x55696ea523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 4 client.27305 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:58:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.907+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55696e93cc70 con 0x55696ea4ff80 2024-07-24T21:58:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.913+0000 7fe509ddb640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696ea4ff80 0x55696ea523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.917+0000 7fe509ddb640 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696ea4ff80 0x55696ea523e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fcecae463c2ed3 server_cookie=24e393889e482818 in_seq=0 out_seq=0 2024-07-24T21:58:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:18.918+0000 7fe4ed7fa640 10 client.27305 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:20.020 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:19 smithi060 ceph-mon[33000]: pgmap v739: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-07-24T21:58:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:19 smithi086 ceph-mon[37128]: pgmap v739: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-07-24T21:58:20.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:19 smithi123 ceph-mon[37143]: pgmap v739: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-07-24T21:58:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.481+0000 7f4c600f6640 1 -- 172.21.15.60:0/2710902787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 msgr2=0x7f4c581085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.482+0000 7f4c600f6640 1 --2- 172.21.15.60:0/2710902787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c581085e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009900 tx=0x7f4c4802a3f0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 -- 172.21.15.60:0/2710902787 shutdown_connections 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 --2- 172.21.15.60:0/2710902787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58108b20 0x7f4c5810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 --2- 172.21.15.60:0/2710902787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c581085e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 --2- 172.21.15.60:0/2710902787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c58102790 0x7f4c58102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 -- 172.21.15.60:0/2710902787 >> 172.21.15.60:0/2710902787 conn(0x7f4c580fe200 msgr2=0x7f4c58100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 -- 172.21.15.60:0/2710902787 shutdown_connections 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.488+0000 7f4c600f6640 1 -- 172.21.15.60:0/2710902787 wait complete. 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 Processor -- start 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 -- start start 2024-07-24T21:58:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 0x7f4c58076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c58074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c58108b20 0x7f4c580754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c581127f0 con 0x7f4c581081a0 2024-07-24T21:58:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c58112670 con 0x7f4c58102790 2024-07-24T21:58:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.490+0000 7f4c600f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c58112970 con 0x7f4c58108b20 2024-07-24T21:58:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.493+0000 7f4c5d691640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c58074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.493+0000 7f4c5d691640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c58074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43798/0 (socket says 172.21.15.60:43798) 2024-07-24T21:58:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.493+0000 7f4c5d691640 1 -- 172.21.15.60:0/1119507454 learned_addr learned my addr 172.21.15.60:0/1119507454 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 0x7f4c58076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 -- 172.21.15.60:0/1119507454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c58108b20 msgr2=0x7f4c580754c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c58108b20 0x7f4c580754c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 -- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 msgr2=0x7f4c58074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c58074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 -- 172.21.15.60:0/1119507454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c581ab5e0 con 0x7f4c58102790 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.495+0000 7f4c5de92640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 0x7f4c58076940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c970 tx=0x7f4c5400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.496+0000 7f4c4effd640 1 -- 172.21.15.60:0/1119507454 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54007bb0 con 0x7f4c58102790 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.496+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c581ab7b0 con 0x7f4c58102790 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.496+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c581abd90 con 0x7f4c58102790 2024-07-24T21:58:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.496+0000 7f4c4effd640 1 -- 172.21.15.60:0/1119507454 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c54007d50 con 0x7f4c58102790 2024-07-24T21:58:20.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.496+0000 7f4c4effd640 1 -- 172.21.15.60:0/1119507454 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400f450 con 0x7f4c58102790 2024-07-24T21:58:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.506+0000 7f4c4effd640 1 -- 172.21.15.60:0/1119507454 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4c54004e70 con 0x7f4c58102790 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.508+0000 7f4c4effd640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 0x7f4c440b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.508+0000 7f4c4effd640 1 -- 172.21.15.60:0/1119507454 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f4c540a6c00 con 0x7f4c58102790 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.509+0000 7f4c624a7740 1 Processor -- start 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.509+0000 7f4c624a7740 1 -- start start 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de28a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de28fd0 0x55dc0de34a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dc0dd9f990 con 0x55dc0de34f60 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc0dd8ee90 con 0x55dc0de286b0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c624a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc0dd92ed0 con 0x55dc0de28fd0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43800/0 (socket says 172.21.15.60:43800) 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.510+0000 7f4c5e693640 1 -- 172.21.15.60:0/810425662 learned_addr learned my addr 172.21.15.60:0/810425662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.511+0000 7f4c5de92640 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de28a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.511+0000 7f4c5d691640 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de28fd0 0x55dc0de34a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.511+0000 7f4c5d691640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 0x7f4c440b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.513+0000 7f4c5e693640 1 -- 172.21.15.60:0/810425662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de28fd0 msgr2=0x55dc0de34a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.513+0000 7f4c5e693640 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de28fd0 0x55dc0de34a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.513+0000 7f4c5e693640 1 -- 172.21.15.60:0/810425662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 msgr2=0x55dc0de28a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.513+0000 7f4c5e693640 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de28a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.513+0000 7f4c5e693640 1 -- 172.21.15.60:0/810425662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0dd18300 con 0x55dc0de34f60 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.514+0000 7f4c5e693640 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d00 tx=0x7f4c5000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3135a196ab316ec7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.514+0000 7f4c43fff640 1 -- 172.21.15.60:0/810425662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000bd30 con 0x55dc0de34f60 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.515+0000 7f4c43fff640 1 -- 172.21.15.60:0/810425662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c50028070 con 0x55dc0de34f60 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.515+0000 7f4c43fff640 1 -- 172.21.15.60:0/810425662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50022600 con 0x55dc0de34f60 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 -- 172.21.15.60:0/810425662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 msgr2=0x55dc0de37350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d00 tx=0x7f4c5000b420 comp rx=0 tx=0).stop 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 -- 172.21.15.60:0/810425662 shutdown_connections 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37350 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de28fd0 0x55dc0de34a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 --2- 172.21.15.60:0/810425662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.516+0000 7f4c624a7740 1 -- 172.21.15.60:0/810425662 >> 172.21.15.60:0/810425662 conn(0x55dc0de264e0 msgr2=0x55dc0de27680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.519+0000 7f4c5d691640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 0x7f4c440b62b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4c5819cb50 tx=0x7f4c48005bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.520+0000 7f4c624a7740 1 -- 172.21.15.60:0/810425662 shutdown_connections 2024-07-24T21:58:20.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.520+0000 7f4c624a7740 1 -- 172.21.15.60:0/810425662 wait complete. 2024-07-24T21:58:20.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.521+0000 7f4c624a7740 1 -- start start 2024-07-24T21:58:20.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.521+0000 7f4c624a7740 1 Processor -- start 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de34910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de34f60 0x55dc0de32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dc0dbf2700 con 0x55dc0de28fd0 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc0dd315d0 con 0x55dc0de286b0 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc0de37c30 con 0x55dc0de34f60 2024-07-24T21:58:20.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c624a7740 10 client.0 _ll_register_callbacks cb 0x55dc0dd51940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:20.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c5d691640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c5d691640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43802/0 (socket says 172.21.15.60:43802) 2024-07-24T21:58:20.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.522+0000 7f4c5d691640 1 -- 172.21.15.60:0/298445580 learned_addr learned my addr 172.21.15.60:0/298445580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:20.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.524+0000 7f4c5e693640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de34f60 0x55dc0de32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 -- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de34f60 msgr2=0x55dc0de32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de34f60 0x55dc0de32430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 -- 172.21.15.60:0/298445580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 msgr2=0x55dc0de34910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:58:20.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de34910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 -- 172.21.15.60:0/298445580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0dd24660 con 0x55dc0de28fd0 2024-07-24T21:58:20.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c5d691640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006010 tx=0x7f4c48005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f53f02462385ce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.526+0000 7f4c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:20.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.527+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4804e070 con 0x55dc0de28fd0 2024-07-24T21:58:20.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.527+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f4c48031a30 con 0x55dc0de28fd0 2024-07-24T21:58:20.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.530+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c50028070 con 0x55dc0de28fd0 2024-07-24T21:58:20.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.531+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4c48058050 con 0x55dc0de28fd0 2024-07-24T21:58:20.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.531+0000 7f4c624a7740 10 client.17901 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:20.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.531+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc0daef650 con 0x55dc0de28fd0 2024-07-24T21:58:20.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.532+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c50028070 con 0x55dc0de28fd0 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.532+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4c4803c450 con 0x55dc0de28fd0 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 10 client.17901 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 20 client.17901 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0de44ee0 0x55dc0de473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 4 client.17901 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.533+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0dda18e0 con 0x55dc0de44ee0 2024-07-24T21:58:20.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.535+0000 7f4c5de92640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0de44ee0 0x55dc0de473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.536+0000 7f4c5de92640 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0de44ee0 0x55dc0de473a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b5d8ded48d3292 server_cookie=7ee37b1895c12aa1 in_seq=0 out_seq=0 2024-07-24T21:58:20.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.536+0000 7f4c417fa640 10 client.17901 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:20.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.537+0000 7f4c417fa640 1 -- 172.21.15.60:0/298445580 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc0dda18e0 con 0x55dc0de44ee0 2024-07-24T21:58:20.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.537+0000 7f4c417fa640 10 client.17901 handle_command_reply: tid=0 2024-07-24T21:58:20.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.537+0000 7f4c624a7740 1 client.17901 shutdown 2024-07-24T21:58:20.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.538+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 msgr2=0x55dc0de31d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.538+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4c48006010 tx=0x7f4c48005b50 comp rx=0 tx=0).stop 2024-07-24T21:58:20.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 shutdown_connections 2024-07-24T21:58:20.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0de44ee0 0x55dc0de473a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de34f60 0x55dc0de32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de31d60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 --2- 172.21.15.60:0/298445580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de286b0 0x55dc0de34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c417fa640 0 client.17901 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 >> 172.21.15.60:0/298445580 conn(0x55dc0de264e0 msgr2=0x55dc0de2c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 shutdown_connections 2024-07-24T21:58:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.539+0000 7f4c624a7740 1 -- 172.21.15.60:0/298445580 wait complete. 2024-07-24T21:58:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.540+0000 7f4c624a7740 20 client.17901 trim_cache size 0 max 16384 2024-07-24T21:58:20.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 Processor -- start 2024-07-24T21:58:20.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 -- start start 2024-07-24T21:58:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de27530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de40db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c0112d0 con 0x55dc0de28fd0 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c00eae0 con 0x55dc0de34f60 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.578+0000 7f4c624a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c00c950 con 0x55dc0de286b0 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:53712/0 (socket says 172.21.15.60:53712) 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 -- 172.21.15.60:0/2410277502 learned_addr learned my addr 172.21.15.60:0/2410277502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5de92640 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de27530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 -- 172.21.15.60:0/2410277502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 msgr2=0x55dc0de27530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de27530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 -- 172.21.15.60:0/2410277502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 msgr2=0x55dc0de40db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de40db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.579+0000 7f4c5e693640 1 -- 172.21.15.60:0/2410277502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0de27d00 con 0x55dc0de34f60 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.580+0000 7f4c5de92640 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de27530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:58:20.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.580+0000 7f4c5e693640 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000f80 tx=0x7f4c50009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d49f50e8fe9189 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.584+0000 7f4c437fe640 1 -- 172.21.15.60:0/2410277502 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000be70 con 0x55dc0de34f60 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.584+0000 7f4c437fe640 1 -- 172.21.15.60:0/2410277502 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c50028040 con 0x55dc0de34f60 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.585+0000 7f4c624a7740 1 -- 172.21.15.60:0/2410277502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 msgr2=0x55dc0de30cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.585+0000 7f4c624a7740 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000f80 tx=0x7f4c50009a90 comp rx=0 tx=0).stop 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.586+0000 7f4c624a7740 1 -- 172.21.15.60:0/2410277502 shutdown_connections 2024-07-24T21:58:20.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.586+0000 7f4c624a7740 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de34f60 0x55dc0de30cf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.586+0000 7f4c624a7740 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de28fd0 0x55dc0de40db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.586+0000 7f4c624a7740 1 --2- 172.21.15.60:0/2410277502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de27530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.586+0000 7f4c624a7740 1 -- 172.21.15.60:0/2410277502 >> 172.21.15.60:0/2410277502 conn(0x55dc0de264e0 msgr2=0x55dc0de305e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.588+0000 7f4c624a7740 1 -- 172.21.15.60:0/2410277502 shutdown_connections 2024-07-24T21:58:20.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.590+0000 7f4c624a7740 1 -- 172.21.15.60:0/2410277502 wait complete. 2024-07-24T21:58:20.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.590+0000 7f4c624a7740 1 -- start start 2024-07-24T21:58:20.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 Processor -- start 2024-07-24T21:58:20.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de28fd0 0x55dc0de3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c00cfa0 con 0x55dc0de34f60 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c00e410 con 0x55dc0de28fd0 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c00d670 con 0x55dc0de286b0 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.591+0000 7f4c624a7740 10 client.0 _ll_register_callbacks cb 0x7f4c48004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5e693640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43830/0 (socket says 172.21.15.60:43830) 2024-07-24T21:58:20.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5e693640 1 -- 172.21.15.60:0/4127671074 learned_addr learned my addr 172.21.15.60:0/4127671074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:20.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5de92640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5d691640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de28fd0 0x55dc0de3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5de92640 1 -- 172.21.15.60:0/4127671074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de28fd0 msgr2=0x55dc0de3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5de92640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de28fd0 0x55dc0de3d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5de92640 1 -- 172.21.15.60:0/4127671074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 msgr2=0x55dc0de37ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.592+0000 7f4c5de92640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.593+0000 7f4c5de92640 1 -- 172.21.15.60:0/4127671074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0de386f0 con 0x55dc0de286b0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.593+0000 7f4c5e693640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.593+0000 7f4c5de92640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de3d110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c540e1150 tx=0x7f4c54007f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd91dced103d9ab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.593+0000 7f4c417fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5401d040 con 0x55dc0de286b0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c50028040 con 0x55dc0de286b0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f4c540cd0b0 con 0x55dc0de286b0 2024-07-24T21:58:20.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4c540aa610 con 0x55dc0de286b0 2024-07-24T21:58:20.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c624a7740 10 client.27265 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:20.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.594+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc0dec1780 con 0x55dc0de286b0 2024-07-24T21:58:20.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.597+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c50028040 con 0x55dc0de286b0 2024-07-24T21:58:20.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.597+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4c540e13d0 con 0x55dc0de286b0 2024-07-24T21:58:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 10 client.27265 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 10 client.27265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 20 client.27265 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0dec1f30 0x55dc0dec4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:20.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 4 client.27265 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:58:20.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.598+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc0dd9d380 con 0x55dc0dec1f30 2024-07-24T21:58:20.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.603+0000 7f4c5d691640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0dec1f30 0x55dc0dec4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:20.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.605+0000 7f4c5d691640 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0dec1f30 0x55dc0dec4350 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32a3ff8a8f6106c server_cookie=d61305311d2aff3d in_seq=0 out_seq=0 2024-07-24T21:58:20.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.605+0000 7f4c417fa640 10 client.27265 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.608+0000 7f4c417fa640 1 -- 172.21.15.60:0/4127671074 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc0dd9d380 con 0x55dc0dec1f30 2024-07-24T21:58:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.608+0000 7f4c417fa640 10 client.27265 handle_command_reply: tid=0 2024-07-24T21:58:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.608+0000 7f4c624a7740 1 client.27265 shutdown 2024-07-24T21:58:20.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 msgr2=0x55dc0de3d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de3d110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c540e1150 tx=0x7f4c54007f00 comp rx=0 tx=0).stop 2024-07-24T21:58:20.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 shutdown_connections 2024-07-24T21:58:20.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc0dec1f30 0x55dc0dec4350 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc0de34f60 0x55dc0de37ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0de28fd0 0x55dc0de3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c417fa640 0 client.27265 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 --2- 172.21.15.60:0/4127671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0de286b0 0x55dc0de3d110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 >> 172.21.15.60:0/4127671074 conn(0x55dc0de264e0 msgr2=0x55dc0de2c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 shutdown_connections 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 1 -- 172.21.15.60:0/4127671074 wait complete. 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.610+0000 7f4c624a7740 20 client.27265 trim_cache size 0 max 16384 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:20.613 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:58:20.614 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:58:20.614 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 msgr2=0x7f4c440b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 0x7f4c440b62b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4c5819cb50 tx=0x7f4c48005bc0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 msgr2=0x7f4c58076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 0x7f4c58076940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c970 tx=0x7f4c5400ce30 comp rx=0 tx=0).stop 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 shutdown_connections 2024-07-24T21:58:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4c440b3df0 0x7f4c440b62b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c58108b20 0x7f4c580754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c581081a0 0x7f4c58074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 --2- 172.21.15.60:0/1119507454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c58102790 0x7f4c58076940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 >> 172.21.15.60:0/1119507454 conn(0x7f4c580fe200 msgr2=0x7f4c581005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.611+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 shutdown_connections 2024-07-24T21:58:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:20.612+0000 7f4c600f6640 1 -- 172.21.15.60:0/1119507454 wait complete. 2024-07-24T21:58:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:58:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:58:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:58:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:58:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe4ed7fa640 1 -- 172.21.15.60:0/598083226 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55696e93cc70 con 0x55696ea4ff80 2024-07-24T21:58:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe4ed7fa640 10 client.27305 handle_command_reply: tid=0 2024-07-24T21:58:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 client.27305 shutdown 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 msgr2=0x55696e9db900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9db900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a970 tx=0x7fe4f4030720 comp rx=0 tx=0).stop 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 shutdown_connections 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55696ea4ff80 0x55696ea523e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696e9d6350 0x55696e9db900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55696e9ca3c0 0x55696e9db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.444+0000 7fe50ed94740 1 --2- 172.21.15.60:0/598083226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55696e9c9aa0 0x55696e9de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.445+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 >> 172.21.15.60:0/598083226 conn(0x55696e9c78d0 msgr2=0x55696e9c6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.445+0000 7fe4ed7fa640 0 client.27305 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.445+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 shutdown_connections 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.445+0000 7fe50ed94740 1 -- 172.21.15.60:0/598083226 wait complete. 2024-07-24T21:58:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.445+0000 7fe50ed94740 20 client.27305 trim_cache size 0 max 16384 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 msgr2=0x7fe4f00b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 0x7fe4f00b61f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe50000a750 tx=0x7fe5000046c0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 msgr2=0x7fe50406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009fa0 tx=0x7fe4fc00c800 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 shutdown_connections 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe4f00b3d30 0x7fe4f00b61f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe504107760 0x7fe504072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe504106de0 0x7fe50406d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 --2- 172.21.15.60:0/2004997839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5041013e0 0x7fe50406d3b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 >> 172.21.15.60:0/2004997839 conn(0x7fe5040fce00 msgr2=0x7fe5040ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 shutdown_connections 2024-07-24T21:58:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:21.446+0000 7fe50addd640 1 -- 172.21.15.60:0/2004997839 wait complete. 2024-07-24T21:58:21.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:58:21.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:58:21.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:21 smithi060 ceph-mon[33000]: pgmap v740: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-07-24T21:58:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:21 smithi086 ceph-mon[37128]: pgmap v740: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-07-24T21:58:22.088 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:22.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:21 smithi123 ceph-mon[37143]: pgmap v740: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-07-24T21:58:23.812 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.5 2024-07-24T21:58:23.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:58:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:23 smithi086 ceph-mon[37128]: pgmap v741: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-07-24T21:58:24.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:23 smithi060 ceph-mon[33000]: pgmap v741: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-07-24T21:58:24.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:23 smithi123 ceph-mon[37143]: pgmap v741: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.523+0000 7f9fd2620640 1 -- 172.21.15.60:0/3772232318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 msgr2=0x7f9fcc101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.523+0000 7f9fd2620640 1 --2- 172.21.15.60:0/3772232318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc101830 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009900 tx=0x7f9fc002a440 comp rx=0 tx=0).stop 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.524+0000 7f9fd2620640 1 -- 172.21.15.60:0/3772232318 shutdown_connections 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.524+0000 7f9fd2620640 1 --2- 172.21.15.60:0/3772232318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc106890 0x7f9fcc106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.524+0000 7f9fd2620640 1 --2- 172.21.15.60:0/3772232318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc101d70 0x7f9fcc106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.524+0000 7f9fd2620640 1 --2- 172.21.15.60:0/3772232318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc101830 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.524+0000 7f9fd2620640 1 -- 172.21.15.60:0/3772232318 >> 172.21.15.60:0/3772232318 conn(0x7f9fcc0fd0c0 msgr2=0x7f9fcc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.525+0000 7f9fd2620640 1 -- 172.21.15.60:0/3772232318 shutdown_connections 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.525+0000 7f9fd2620640 1 -- 172.21.15.60:0/3772232318 wait complete. 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.526+0000 7f9fd2620640 1 Processor -- start 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.526+0000 7f9fd2620640 1 -- start start 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.526+0000 7f9fd2620640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.526+0000 7f9fd2620640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 0x7f9fcc115960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fd2620640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc106890 0x7f9fcc10e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fd2620640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc118e60 con 0x7f9fcc101450 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fd2620640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc118ce0 con 0x7f9fcc106890 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fd2620640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc118fe0 con 0x7f9fcc101d70 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 0x7f9fcc115960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 0x7f9fcc115960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42682/0 (socket says 172.21.15.60:42682) 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fcb7fe640 1 -- 172.21.15.60:0/2313535269 learned_addr learned my addr 172.21.15.60:0/2313535269 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.527+0000 7f9fcbfff640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 -- 172.21.15.60:0/2313535269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 msgr2=0x7f9fcc115960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 0x7f9fcc115960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 -- 172.21.15.60:0/2313535269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc106890 msgr2=0x7f9fcc10e720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc106890 0x7f9fcc10e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 -- 172.21.15.60:0/2313535269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc10efe0 con 0x7f9fcc101450 2024-07-24T21:58:24.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.528+0000 7f9fcbfff640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc115420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009230 tx=0x7f9fc0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:24.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.529+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc003f070 con 0x7f9fcc101450 2024-07-24T21:58:24.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.529+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc10f270 con 0x7f9fcc101450 2024-07-24T21:58:24.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.529+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1142e0 con 0x7f9fcc101450 2024-07-24T21:58:24.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.529+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc0005630 con 0x7f9fcc101450 2024-07-24T21:58:24.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.530+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0035030 con 0x7f9fcc101450 2024-07-24T21:58:24.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.534+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb80052c0 con 0x7f9fcc101450 2024-07-24T21:58:24.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.535+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9fc0042070 con 0x7f9fcc101450 2024-07-24T21:58:24.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.536+0000 7f9fc97fa640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 0x7f9fb40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:24.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.537+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f9fc0107120 con 0x7f9fcc101450 2024-07-24T21:58:24.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.537+0000 7f9fcb7fe640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 0x7f9fb40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:24.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.538+0000 7f9fcb7fe640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 0x7f9fb40b6390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc107180 tx=0x7f9fc400b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.544+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9fc00cba90 con 0x7f9fcc101450 2024-07-24T21:58:24.573 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:24.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.980+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fb80055b0 con 0x7f9fcc101450 2024-07-24T21:58:24.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.983+0000 7f9fc97fa640 1 -- 172.21.15.60:0/2313535269 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f9fc00cb390 con 0x7f9fcc101450 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 msgr2=0x7f9fb40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 0x7f9fb40b6390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc107180 tx=0x7f9fc400b040 comp rx=0 tx=0).stop 2024-07-24T21:58:24.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 msgr2=0x7f9fcc115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc115420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009230 tx=0x7f9fc0005230 comp rx=0 tx=0).stop 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 shutdown_connections 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.986+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9fb40b3ed0 0x7f9fb40b6390 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.987+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fcc106890 0x7f9fcc10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.987+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc101d70 0x7f9fcc115960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.987+0000 7f9fd2620640 1 --2- 172.21.15.60:0/2313535269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fcc101450 0x7f9fcc115420 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:24.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.987+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 >> 172.21.15.60:0/2313535269 conn(0x7f9fcc0fd0c0 msgr2=0x7f9fcc104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:24.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.987+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 shutdown_connections 2024-07-24T21:58:24.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:24.988+0000 7f9fd2620640 1 -- 172.21.15.60:0/2313535269 wait complete. 2024-07-24T21:58:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:25 smithi086 ceph-mon[37128]: pgmap v742: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-07-24T21:58:26.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:25 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2313535269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:26.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:25 smithi060 ceph-mon[33000]: pgmap v742: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-07-24T21:58:26.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:25 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2313535269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:26.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:25 smithi123 ceph-mon[37143]: pgmap v742: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-07-24T21:58:26.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:25 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2313535269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:26.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:58:27.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.374+0000 7fa922470640 1 -- 172.21.15.60:0/567291517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c109b40 msgr2=0x7fa91c109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:27.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.374+0000 7fa922470640 1 --2- 172.21.15.60:0/567291517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c109b40 0x7fa91c109f20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa914009ca0 tx=0x7fa91402a8f0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.375+0000 7fa922470640 1 -- 172.21.15.60:0/567291517 shutdown_connections 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.375+0000 7fa922470640 1 --2- 172.21.15.60:0/567291517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10f0d0 0x7fa91c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.375+0000 7fa922470640 1 --2- 172.21.15.60:0/567291517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c10a460 0x7fa91c10eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.375+0000 7fa922470640 1 --2- 172.21.15.60:0/567291517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c109b40 0x7fa91c109f20 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa914009ca0 tx=0x7fa91402a8f0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.375+0000 7fa922470640 1 -- 172.21.15.60:0/567291517 >> 172.21.15.60:0/567291517 conn(0x7fa91c06c8e0 msgr2=0x7fa91c06ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.377+0000 7fa922470640 1 -- 172.21.15.60:0/567291517 shutdown_connections 2024-07-24T21:58:27.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.377+0000 7fa922470640 1 -- 172.21.15.60:0/567291517 wait complete. 2024-07-24T21:58:27.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.378+0000 7fa922470640 1 Processor -- start 2024-07-24T21:58:27.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.378+0000 7fa922470640 1 -- start start 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.378+0000 7fa922470640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10a460 0x7fa91c115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.378+0000 7fa922470640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.378+0000 7fa922470640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 0x7fa91c1b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.379+0000 7fa922470640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118b80 con 0x7fa91c1168d0 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.379+0000 7fa922470640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118a00 con 0x7fa91c10f0d0 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.379+0000 7fa922470640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118d00 con 0x7fa91c10a460 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.380+0000 7fa920c6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.380+0000 7fa920c6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55426/0 (socket says 172.21.15.60:55426) 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.380+0000 7fa920c6d640 1 -- 172.21.15.60:0/1510255596 learned_addr learned my addr 172.21.15.60:0/1510255596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.380+0000 7fa92146e640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10a460 0x7fa91c115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:27.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.380+0000 7fa921c6f640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 0x7fa91c1b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:27.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.386+0000 7fa920c6d640 1 -- 172.21.15.60:0/1510255596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10a460 msgr2=0x7fa91c115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:27.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.386+0000 7fa920c6d640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10a460 0x7fa91c115e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.386+0000 7fa920c6d640 1 -- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 msgr2=0x7fa91c1b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:27.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.386+0000 7fa920c6d640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 0x7fa91c1b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.386+0000 7fa920c6d640 1 -- 172.21.15.60:0/1510255596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1b3ef0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.387+0000 7fa921c6f640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 0x7fa91c1b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa920c6d640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02f960 tx=0x7fa90c02fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c039030 con 0x7fa91c10f0d0 2024-07-24T21:58:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c074dd0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c075360 con 0x7fa91c10f0d0 2024-07-24T21:58:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c004020 con 0x7fa91c10f0d0 2024-07-24T21:58:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.388+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0047a0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.390+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9000052c0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.392+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa90c0041d0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.394+0000 7fa9127fc640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 0x7fa9080b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:27.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.394+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fa90c105e90 con 0x7fa91c10f0d0 2024-07-24T21:58:27.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.399+0000 7fa92146e640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 0x7fa9080b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.401+0000 7fa92146e640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 0x7fa9080b6370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa91402ae60 tx=0x7fa914005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.404+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa90c0cd1c0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.457 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:27.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.856+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9000055b0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.858 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:27 smithi060 ceph-mon[33000]: pgmap v743: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-07-24T21:58:27.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.858+0000 7fa9127fc640 1 -- 172.21.15.60:0/1510255596 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fa90c0ccac0 con 0x7fa91c10f0d0 2024-07-24T21:58:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:58:27.860 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:58:27.860 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:58:27.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 msgr2=0x7fa9080b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:27.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 0x7fa9080b6370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa91402ae60 tx=0x7fa914005fd0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 msgr2=0x7fa91c116390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:27.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02f960 tx=0x7fa90c02fe20 comp rx=0 tx=0).stop 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 shutdown_connections 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa9080b3eb0 0x7fa9080b6370 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1168d0 0x7fa91c1b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c10f0d0 0x7fa91c116390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 --2- 172.21.15.60:0/1510255596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10a460 0x7fa91c115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.866+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 >> 172.21.15.60:0/1510255596 conn(0x7fa91c06c8e0 msgr2=0x7fa91c072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.867+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 shutdown_connections 2024-07-24T21:58:27.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:27.868+0000 7fa922470640 1 -- 172.21.15.60:0/1510255596 wait complete. 2024-07-24T21:58:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:27 smithi086 ceph-mon[37128]: pgmap v743: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-07-24T21:58:28.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:27 smithi123 ceph-mon[37143]: pgmap v743: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-07-24T21:58:28.965 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:28 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1510255596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:29.014 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 147.5 --await-for 45.0' 2024-07-24T21:58:29.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 147.5 --await-for 45.0 2024-07-24T21:58:29.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:28 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1510255596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:29.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:28 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1510255596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:29.610 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:29.988 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:29 smithi060 ceph-mon[33000]: pgmap v744: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-07-24T21:58:30.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:29 smithi123 ceph-mon[37143]: pgmap v744: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-07-24T21:58:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 -- 172.21.15.60:0/643468189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3878109da0 msgr2=0x7f387810a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 --2- 172.21.15.60:0/643468189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3878109da0 0x7f387810a180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f386c009900 tx=0x7f386c02a440 comp rx=0 tx=0).stop 2024-07-24T21:58:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 -- 172.21.15.60:0/643468189 shutdown_connections 2024-07-24T21:58:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 --2- 172.21.15.60:0/643468189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810f610 0x7f3878115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 --2- 172.21.15.60:0/643468189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f387810ec70 0x7f387810f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 --2- 172.21.15.60:0/643468189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3878109da0 0x7f387810a180 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.270+0000 7f387e321640 1 -- 172.21.15.60:0/643468189 >> 172.21.15.60:0/643468189 conn(0x7f387806dca0 msgr2=0x7f387806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.271+0000 7f387e321640 1 -- 172.21.15.60:0/643468189 shutdown_connections 2024-07-24T21:58:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.271+0000 7f387e321640 1 -- 172.21.15.60:0/643468189 wait complete. 2024-07-24T21:58:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.271+0000 7f387e321640 1 Processor -- start 2024-07-24T21:58:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 -- start start 2024-07-24T21:58:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810ec70 0x7f387811ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f387810f610 0x7f387811ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3878118e20 con 0x7f387810f610 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3878118ca0 con 0x7f3878109da0 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387e321640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3878118fa0 con 0x7f387810ec70 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f3877fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f387c8be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f387810f610 0x7f387811ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f3877fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55438/0 (socket says 172.21.15.60:55438) 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f3877fff640 1 -- 172.21.15.60:0/3279828719 learned_addr learned my addr 172.21.15.60:0/3279828719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.272+0000 7f38777fe640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810ec70 0x7f387811ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 -- 172.21.15.60:0/3279828719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810ec70 msgr2=0x7f387811ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810ec70 0x7f387811ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 -- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f387810f610 msgr2=0x7f387811ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f387810f610 0x7f387811ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 -- 172.21.15.60:0/3279828719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3878120610 con 0x7f3878109da0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f3877fff640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c0052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f38757fa640 1 -- 172.21.15.60:0/3279828719 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c03f070 con 0x7f3878109da0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38781208a0 con 0x7f3878109da0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f38757fa640 1 -- 172.21.15.60:0/3279828719 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386c003ca0 con 0x7f3878109da0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.273+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38781bcea0 con 0x7f3878109da0 2024-07-24T21:58:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.274+0000 7f38757fa640 1 -- 172.21.15.60:0/3279828719 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c035030 con 0x7f3878109da0 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.277+0000 7f38806d2740 1 Processor -- start 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.277+0000 7f38757fa640 1 -- 172.21.15.60:0/3279828719 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f386c042070 con 0x7f3878109da0 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.278+0000 7f38757fa640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 0x7f38600b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.278+0000 7f38757fa640 1 -- 172.21.15.60:0/3279828719 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f386c1048c0 con 0x7f3878109da0 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 -- start start 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de025fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de026bf80 0x557de026e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557de01d69b0 con 0x557de026bf80 2024-07-24T21:58:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557de01c5eb0 con 0x557de025fff0 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f38806d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557de01c9ef0 con 0x557de025f6d0 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.279+0000 7f387c8be640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 0x7f38600b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55446/0 (socket says 172.21.15.60:55446) 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 -- 172.21.15.60:0/2594241155 learned_addr learned my addr 172.21.15.60:0/2594241155 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 -- 172.21.15.60:0/2594241155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 msgr2=0x557de025fab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de025fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 -- 172.21.15.60:0/2594241155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de026bf80 msgr2=0x557de026e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de026bf80 0x557de026e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.280+0000 7f38777fe640 1 -- 172.21.15.60:0/2594241155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de014f320 con 0x557de025fff0 2024-07-24T21:58:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.281+0000 7f387c8be640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 0x7f38600b6300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f387000b600 tx=0x7f3870009990 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.282+0000 7f38777fe640 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f386800b7f0 tx=0x7f386800bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73354c6793b821c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.282+0000 7f385e7fc640 1 -- 172.21.15.60:0/2594241155 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386800d950 con 0x557de025fff0 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.282+0000 7f385e7fc640 1 -- 172.21.15.60:0/2594241155 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3868018070 con 0x557de025fff0 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 -- 172.21.15.60:0/2594241155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 msgr2=0x557de026ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f386800b7f0 tx=0x7f386800bcb0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 -- 172.21.15.60:0/2594241155 shutdown_connections 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de026bf80 0x557de026e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de025fff0 0x557de026ba40 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f386800b7f0 tx=0x7f386800bcb0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 --2- 172.21.15.60:0/2594241155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de025fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.283+0000 7f38806d2740 1 -- 172.21.15.60:0/2594241155 >> 172.21.15.60:0/2594241155 conn(0x557de025d500 msgr2=0x557de025e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.285+0000 7f38806d2740 1 -- 172.21.15.60:0/2594241155 shutdown_connections 2024-07-24T21:58:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.285+0000 7f38806d2740 1 -- 172.21.15.60:0/2594241155 wait complete. 2024-07-24T21:58:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.285+0000 7f38806d2740 1 -- start start 2024-07-24T21:58:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 Processor -- start 2024-07-24T21:58:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de0269040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557de0029700 con 0x557de025f6d0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557de01685f0 con 0x557de026bf80 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557de026eca0 con 0x557de0269ac0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38806d2740 10 client.0 _ll_register_callbacks cb 0x557de0188960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f387c8be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f3877fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de0269040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f387c8be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60274/0 (socket says 172.21.15.60:60274) 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f387c8be640 1 -- 172.21.15.60:0/422873087 learned_addr learned my addr 172.21.15.60:0/422873087 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.286+0000 7f38777fe640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 -- 172.21.15.60:0/422873087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 msgr2=0x557de0269580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 -- 172.21.15.60:0/422873087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 msgr2=0x557de0269040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de0269040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 -- 172.21.15.60:0/422873087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de015b680 con 0x557de0269ac0 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f387c8be640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3870008dd0 tx=0x7f3870004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cd0bd3d854ab70 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.287+0000 7f38577fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:58:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.289+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700175c0 con 0x557de0269ac0 2024-07-24T21:58:30.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.289+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f3870017760 con 0x557de0269ac0 2024-07-24T21:58:30.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.289+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3868018070 con 0x557de0269ac0 2024-07-24T21:58:30.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.290+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f387002e050 con 0x557de0269ac0 2024-07-24T21:58:30.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.290+0000 7f38806d2740 10 client.27280 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:30.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.290+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557ddff26650 con 0x557de0269ac0 2024-07-24T21:58:30.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.290+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3868018070 con 0x557de0269ac0 2024-07-24T21:58:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3870029030 con 0x557de0269ac0 2024-07-24T21:58:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 10 client.27280 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 10 client.27280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 20 client.27280 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de027c680 0x557de027eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 4 client.27280 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de01d8900 con 0x557de027c680 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.291+0000 7f3877fff640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de027c680 0x557de027eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.292+0000 7f38577fe640 10 client.27280 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:58:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.292+0000 7f3877fff640 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de027c680 0x557de027eac0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d1bd46f27c1dcd6 server_cookie=eca0bce427230ac3 in_seq=0 out_seq=0 2024-07-24T21:58:30.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:29 smithi086 ceph-mon[37128]: pgmap v744: 97 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-07-24T21:58:30.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.293+0000 7f38577fe640 1 -- 172.21.15.60:0/422873087 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557de01d8900 con 0x557de027c680 2024-07-24T21:58:30.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.293+0000 7f38577fe640 10 client.27280 handle_command_reply: tid=0 2024-07-24T21:58:30.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.293+0000 7f38806d2740 1 client.27280 shutdown 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.293+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 msgr2=0x557de02780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.293+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3870008dd0 tx=0x7f3870004b50 comp rx=0 tx=0).stop 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 shutdown_connections 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de027c680 0x557de027eac0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de02780b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 --2- 172.21.15.60:0/422873087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de0269040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 >> 172.21.15.60:0/422873087 conn(0x557de025d500 msgr2=0x557de02630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.294+0000 7f38577fe640 0 client.27280 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.296+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 shutdown_connections 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.296+0000 7f38806d2740 1 -- 172.21.15.60:0/422873087 wait complete. 2024-07-24T21:58:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.296+0000 7f38806d2740 20 client.27280 trim_cache size 0 max 16384 2024-07-24T21:58:30.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 Processor -- start 2024-07-24T21:58:30.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 -- start start 2024-07-24T21:58:30.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de0277ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de0269ac0 0x557de0268e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3864011310 con 0x557de0269ac0 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f386400eb20 con 0x557de026bf80 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38806d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f386400c990 con 0x557de025f6d0 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55470/0 (socket says 172.21.15.60:55470) 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f38777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de0269ac0 0x557de0268e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 -- 172.21.15.60:0/799329295 learned_addr learned my addr 172.21.15.60:0/799329295 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f3877fff640 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de0277ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 -- 172.21.15.60:0/799329295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 msgr2=0x557de0277ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de0277ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 -- 172.21.15.60:0/799329295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de0269ac0 msgr2=0x557de0268e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de0269ac0 0x557de0268e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.323+0000 7f387c8be640 1 -- 172.21.15.60:0/799329295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de025fe20 con 0x557de026bf80 2024-07-24T21:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.324+0000 7f387c8be640 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f387000be90 tx=0x7f3870009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3c03b3b13e3b9bf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.324+0000 7f385dffb640 1 -- 172.21.15.60:0/799329295 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700213f0 con 0x557de026bf80 2024-07-24T21:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.324+0000 7f385dffb640 1 -- 172.21.15.60:0/799329295 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3870021590 con 0x557de026bf80 2024-07-24T21:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.324+0000 7f385dffb640 1 -- 172.21.15.60:0/799329295 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870032430 con 0x557de026bf80 2024-07-24T21:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.325+0000 7f38806d2740 1 -- 172.21.15.60:0/799329295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 msgr2=0x557de0269350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.325+0000 7f38806d2740 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f387000be90 tx=0x7f3870009f40 comp rx=0 tx=0).stop 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.325+0000 7f38806d2740 1 -- 172.21.15.60:0/799329295 shutdown_connections 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de0269350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de0269ac0 0x557de0268e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 --2- 172.21.15.60:0/799329295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de025f6d0 0x557de0277ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 -- 172.21.15.60:0/799329295 >> 172.21.15.60:0/799329295 conn(0x557de025d500 msgr2=0x557de0264220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 -- 172.21.15.60:0/799329295 shutdown_connections 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.326+0000 7f38806d2740 1 -- 172.21.15.60:0/799329295 wait complete. 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.327+0000 7f38806d2740 1 -- start start 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.327+0000 7f38806d2740 1 Processor -- start 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de026f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de026fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f386400cfe0 con 0x557de025f6d0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f387c8be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f387c8be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55484/0 (socket says 172.21.15.60:55484) 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f387c8be640 1 -- 172.21.15.60:0/3500404934 learned_addr learned my addr 172.21.15.60:0/3500404934 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f386400e450 con 0x557de026bf80 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.328+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f386400d6b0 con 0x557de0269ac0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f38777fe640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de026fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f38806d2740 10 client.0 _ll_register_callbacks cb 0x7f38781270d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f3877fff640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de026f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 -- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 msgr2=0x557de026fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de026fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 -- 172.21.15.60:0/3500404934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 msgr2=0x557de026f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de026f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de02e5930 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f387c8be640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3870021a10 tx=0x7f3870004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caadebed960d154 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.329+0000 7f38577fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870040020 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f3870017d20 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3870021590 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f38700213f0 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38806d2740 10 client.27335 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.330+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557de02e6c10 con 0x557de026bf80 2024-07-24T21:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.331+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3870032430 con 0x557de026bf80 2024-07-24T21:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f387004d020 con 0x557de026bf80 2024-07-24T21:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 10 client.27335 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 10 client.27335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 20 client.27335 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:30.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de02e72f0 0x557de02e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:30.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 4 client.27335 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:58:30.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.332+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3870021590 con 0x557de02e72f0 2024-07-24T21:58:30.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.337+0000 7f3877fff640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de02e72f0 0x557de02e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:30.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.337+0000 7f3877fff640 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de02e72f0 0x557de02e9710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9b2b2907f0b511f server_cookie=60e39ef27c237394 in_seq=0 out_seq=0 2024-07-24T21:58:30.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.337+0000 7f38577fe640 10 client.27335 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:58:30.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.337+0000 7f38577fe640 1 -- 172.21.15.60:0/3500404934 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3870021590 con 0x557de02e72f0 2024-07-24T21:58:30.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38577fe640 10 client.27335 handle_command_reply: tid=0 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 client.27335 shutdown 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 msgr2=0x557de02e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3870021a10 tx=0x7f3870004b50 comp rx=0 tx=0).stop 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 shutdown_connections 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x557de02e72f0 0x557de02e9710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38577fe640 0 client.27335 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de026bf80 0x557de02e5350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557de0269ac0 0x557de026fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 --2- 172.21.15.60:0/3500404934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557de025f6d0 0x557de026f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 >> 172.21.15.60:0/3500404934 conn(0x557de025d500 msgr2=0x557de0268e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.338+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 shutdown_connections 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.339+0000 7f38806d2740 1 -- 172.21.15.60:0/3500404934 wait complete. 2024-07-24T21:58:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.339+0000 7f38806d2740 20 client.27335 trim_cache size 0 max 16384 2024-07-24T21:58:30.342 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:58:30.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 msgr2=0x7f38600b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 0x7f38600b6300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f387000b600 tx=0x7f3870009990 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 msgr2=0x7f387811a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c0052a0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 shutdown_connections 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38600b3e40 0x7f38600b6300 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f387810f610 0x7f387811ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f387810ec70 0x7f387811ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 --2- 172.21.15.60:0/3279828719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3878109da0 0x7f387811a650 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 >> 172.21.15.60:0/3279828719 conn(0x7f387806dca0 msgr2=0x7f3878072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.340+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 shutdown_connections 2024-07-24T21:58:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:30.341+0000 7f387e321640 1 -- 172.21.15.60:0/3279828719 wait complete. 2024-07-24T21:58:32.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:31 smithi060 ceph-mon[33000]: pgmap v745: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-07-24T21:58:32.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:31 smithi123 ceph-mon[37143]: pgmap v745: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-07-24T21:58:32.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:58:32.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:31 smithi086 ceph-mon[37128]: pgmap v745: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-07-24T21:58:33.714 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:33.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 -- 172.21.15.60:0/24705626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc84010cac0 msgr2=0x7fc84010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 --2- 172.21.15.60:0/24705626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc84010cac0 0x7fc84010ef20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc838009ca0 tx=0x7fc83802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 -- 172.21.15.60:0/24705626 shutdown_connections 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 --2- 172.21.15.60:0/24705626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc84010cac0 0x7fc84010ef20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 --2- 172.21.15.60:0/24705626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840103f80 0x7fc84010c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 --2- 172.21.15.60:0/24705626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103660 0x7fc840103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 -- 172.21.15.60:0/24705626 >> 172.21.15.60:0/24705626 conn(0x7fc8400fd0c0 msgr2=0x7fc8400ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 -- 172.21.15.60:0/24705626 shutdown_connections 2024-07-24T21:58:33.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.716+0000 7fc846182640 1 -- 172.21.15.60:0/24705626 wait complete. 2024-07-24T21:58:33.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.718+0000 7fc846182640 1 Processor -- start 2024-07-24T21:58:33.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.718+0000 7fc846182640 1 -- start start 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.718+0000 7fc846182640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 0x7fc840198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.718+0000 7fc846182640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 0x7fc840198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 0x7fc840198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35748/0 (socket says 172.21.15.60:35748) 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55514/0 (socket says 172.21.15.60:55514) 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc846182640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84010cac0 0x7fc84019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83f7fe640 1 -- 172.21.15.60:0/3280621451 learned_addr learned my addr 172.21.15.60:0/3280621451 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc840112720 con 0x7fc840103660 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401125a0 con 0x7fc840103f80 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401128a0 con 0x7fc84010cac0 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83ffff640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84010cac0 0x7fc84019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.719+0000 7fc83effd640 1 -- 172.21.15.60:0/3280621451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84010cac0 msgr2=0x7fc84019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83effd640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84010cac0 0x7fc84019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83effd640 1 -- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 msgr2=0x7fc840198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83effd640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 0x7fc840198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83effd640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84019eb80 con 0x7fc840103f80 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83effd640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc834009b20 tx=0x7fc834009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83cff9640 1 -- 172.21.15.60:0/3280621451 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83401e070 con 0x7fc840103f80 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84019ee70 con 0x7fc840103f80 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83cff9640 1 -- 172.21.15.60:0/3280621451 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83400bc30 con 0x7fc840103f80 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc83cff9640 1 -- 172.21.15.60:0/3280621451 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834003e00 con 0x7fc840103f80 2024-07-24T21:58:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.720+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401011f0 con 0x7fc840103f80 2024-07-24T21:58:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.724+0000 7fc83cff9640 1 -- 172.21.15.60:0/3280621451 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc834021070 con 0x7fc840103f80 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.724+0000 7fc848533740 1 Processor -- start 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.724+0000 7fc848533740 1 -- start start 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 0x55e9c5f583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f63db0 0x55e9c5f661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5ec5670 con 0x55e9c5f63db0 2024-07-24T21:58:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5eb4b70 con 0x55e9c5f57f80 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc848533740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5eb8bb0 con 0x55e9c5f594e0 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 0x55e9c5f583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60318/0 (socket says 172.21.15.60:60318) 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83ffff640 1 -- 172.21.15.60:0/1959846903 learned_addr learned my addr 172.21.15.60:0/1959846903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83cff9640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 0x7fc8280b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 -- 172.21.15.60:0/1959846903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 msgr2=0x55e9c5f57a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 -- 172.21.15.60:0/1959846903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f63db0 msgr2=0x55e9c5f661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f63db0 0x55e9c5f661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83f7fe640 1 -- 172.21.15.60:0/1959846903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c5e4cd30 con 0x55e9c5f57f80 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83ffff640 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.725+0000 7fc83cff9640 1 -- 172.21.15.60:0/3280621451 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fc8340e78d0 con 0x7fc840103f80 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.726+0000 7fc83ffff640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 0x7fc8280b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.727+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 0x55e9c5f583a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c80 tx=0x7fc830011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1cb5862f33c1ba server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.727+0000 7fc825ffb640 1 -- 172.21.15.60:0/1959846903 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830004230 con 0x55e9c5f57f80 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.727+0000 7fc83ffff640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 0x7fc8280b6280 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc838009620 tx=0x7fc838005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.727+0000 7fc825ffb640 1 -- 172.21.15.60:0/1959846903 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8300043d0 con 0x55e9c5f57f80 2024-07-24T21:58:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.727+0000 7fc825ffb640 1 -- 172.21.15.60:0/1959846903 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830004c70 con 0x55e9c5f57f80 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 -- 172.21.15.60:0/1959846903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 msgr2=0x55e9c5f583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 0x55e9c5f583a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c80 tx=0x7fc830011810 comp rx=0 tx=0).stop 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 -- 172.21.15.60:0/1959846903 shutdown_connections 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f63db0 0x55e9c5f661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f57f80 0x55e9c5f583a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 --2- 172.21.15.60:0/1959846903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 -- 172.21.15.60:0/1959846903 >> 172.21.15.60:0/1959846903 conn(0x55e9c5f19720 msgr2=0x55e9c5f55870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.728+0000 7fc848533740 1 -- 172.21.15.60:0/1959846903 shutdown_connections 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.729+0000 7fc848533740 1 -- 172.21.15.60:0/1959846903 wait complete. 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.729+0000 7fc848533740 1 -- start start 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.729+0000 7fc848533740 1 Processor -- start 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.729+0000 7fc848533740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f60d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5d187a0 con 0x55e9c5f57f80 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5e56ff0 con 0x55e9c5f63db0 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5f66cc0 con 0x55e9c5f594e0 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f60d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60328/0 (socket says 172.21.15.60:60328) 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83effd640 1 -- 172.21.15.60:0/655016406 learned_addr learned my addr 172.21.15.60:0/655016406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55532/0 (socket says 172.21.15.60:55532) 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc848533740 10 client.0 _ll_register_callbacks cb 0x55e9c5bc2090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 -- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 msgr2=0x55e9c5f61270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 -- 172.21.15.60:0/655016406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 msgr2=0x55e9c5f60d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f60d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83ffff640 1 -- 172.21.15.60:0/655016406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c5e5b0a0 con 0x55e9c5f63db0 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.730+0000 7fc83effd640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc83ffff640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc8380011d0 tx=0x7fc83802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5399913fbfb918da server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc81effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838031920 con 0x55e9c5f63db0 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fc8380047a0 con 0x55e9c5f63db0 2024-07-24T21:58:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8300043d0 con 0x55e9c5f63db0 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc838031e90 con 0x55e9c5f63db0 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc848533740 10 client.27347 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e9c5c24b90 con 0x55e9c5f63db0 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.731+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83804d030 con 0x55e9c5f63db0 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc838030890 con 0x55e9c5f63db0 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 10 client.27347 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 10 client.27347 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 20 client.27347 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5f762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 4 client.27347 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.732+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8300043d0 con 0x55e9c5f73ed0 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.734+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5f762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.735+0000 7fc81effd640 10 client.27347 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.735+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5f762f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7525da613bb2c518 server_cookie=100f0b545727600 in_seq=0 out_seq=0 2024-07-24T21:58:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc81effd640 1 -- 172.21.15.60:0/655016406 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc8300043d0 con 0x55e9c5f73ed0 2024-07-24T21:58:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc81effd640 10 client.27347 handle_command_reply: tid=0 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 client.27347 shutdown 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 msgr2=0x55e9c5f617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc8380011d0 tx=0x7fc83802a8c0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 shutdown_connections 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5f762f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f617b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc81effd640 0 client.27347 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f61270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 --2- 172.21.15.60:0/655016406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f60d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.738+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 >> 172.21.15.60:0/655016406 conn(0x55e9c5f19720 msgr2=0x55e9c5f5b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.739+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 shutdown_connections 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.739+0000 7fc848533740 1 -- 172.21.15.60:0/655016406 wait complete. 2024-07-24T21:58:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.739+0000 7fc848533740 20 client.27347 trim_cache size 0 max 16384 2024-07-24T21:58:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.766+0000 7fc848533740 1 Processor -- start 2024-07-24T21:58:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 -- start start 2024-07-24T21:58:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f4b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f4be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401a6210 con 0x55e9c5f57f80 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc840073250 con 0x55e9c5f63db0 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc848533740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8400710c0 con 0x55e9c5f594e0 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35782/0 (socket says 172.21.15.60:35782) 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f4be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 -- 172.21.15.60:0/3598739163 learned_addr learned my addr 172.21.15.60:0/3598739163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83effd640 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f4b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 -- 172.21.15.60:0/3598739163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 msgr2=0x55e9c5f4b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f4b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 -- 172.21.15.60:0/3598739163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 msgr2=0x55e9c5f4be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f4be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83f7fe640 1 -- 172.21.15.60:0/3598739163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c5f48990 con 0x55e9c5f57f80 2024-07-24T21:58:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.767+0000 7fc83ffff640 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f4be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.768+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55e9c5f5abc0 tx=0x7fc8300099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655b8d8a25e6381a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.768+0000 7fc8257fa640 1 -- 172.21.15.60:0/3598739163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830045070 con 0x55e9c5f57f80 2024-07-24T21:58:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.768+0000 7fc8257fa640 1 -- 172.21.15.60:0/3598739163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83001f040 con 0x55e9c5f57f80 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.768+0000 7fc8257fa640 1 -- 172.21.15.60:0/3598739163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83001e040 con 0x55e9c5f57f80 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- 172.21.15.60:0/3598739163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 msgr2=0x55e9c5f57720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55e9c5f5abc0 tx=0x7fc8300099c0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- 172.21.15.60:0/3598739163 shutdown_connections 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f4be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f4b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 --2- 172.21.15.60:0/3598739163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f57720 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- 172.21.15.60:0/3598739163 >> 172.21.15.60:0/3598739163 conn(0x55e9c5f19720 msgr2=0x55e9c5f62b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- 172.21.15.60:0/3598739163 shutdown_connections 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- 172.21.15.60:0/3598739163 wait complete. 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.769+0000 7fc848533740 1 -- start start 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 Processor -- start 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f6bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc840071710 con 0x55e9c5f57f80 2024-07-24T21:58:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f5e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35790/0 (socket says 172.21.15.60:35790) 2024-07-24T21:58:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc83f7fe640 1 -- 172.21.15.60:0/2529848566 learned_addr learned my addr 172.21.15.60:0/2529848566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc83ffff640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc83effd640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f6bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.770+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc840072b80 con 0x55e9c5f63db0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc840071de0 con 0x55e9c5f594e0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc848533740 10 client.0 _ll_register_callbacks cb 0x7fc830005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 -- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 msgr2=0x55e9c5f6bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f6bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 -- 172.21.15.60:0/2529848566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 msgr2=0x55e9c5f5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c5f7b830 con 0x55e9c5f63db0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc83ffff640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f6c580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc838030dd0 tx=0x7fc838030a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f28fad75a362097 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.771+0000 7fc81effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83805d070 con 0x55e9c5f63db0 2024-07-24T21:58:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fc838049420 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc83001f040 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc83806a050 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc848533740 10 client.27353 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e9c5fdee40 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.772+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83001f040 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc838049a40 con 0x55e9c5f63db0 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 10 client.27353 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 10 client.27353 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 20 client.27353 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5fe16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 4 client.27353 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.5, "await_for": 45.0}] 2024-07-24T21:58:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.773+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.5, "await_for": 45.0}) -- 0x55e9c5ec75c0 con 0x55e9c5f73ed0 2024-07-24T21:58:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.774+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5fe16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.775+0000 7fc83f7fe640 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5fe16d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8caf42945a60b1 server_cookie=37d25e5649164510 in_seq=0 out_seq=0 2024-07-24T21:58:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:33.775+0000 7fc81effd640 10 client.27353 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:33.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:33 smithi060 ceph-mon[33000]: pgmap v746: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-07-24T21:58:34.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:33 smithi123 ceph-mon[37143]: pgmap v746: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-07-24T21:58:34.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:33 smithi086 ceph-mon[37128]: pgmap v746: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-07-24T21:58:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 -- 172.21.15.60:0/1490498416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 msgr2=0x7fa6d4107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1490498416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d4107410 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002a410 comp rx=0 tx=0).stop 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 -- 172.21.15.60:0/1490498416 shutdown_connections 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1490498416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d4107950 0x7fa6d410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1490498416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d4107410 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1490498416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d41015d0 0x7fa6d41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 -- 172.21.15.60:0/1490498416 >> 172.21.15.60:0/1490498416 conn(0x7fa6d40fd030 msgr2=0x7fa6d40ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 -- 172.21.15.60:0/1490498416 shutdown_connections 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.564+0000 7fa6dab61640 1 -- 172.21.15.60:0/1490498416 wait complete. 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 Processor -- start 2024-07-24T21:58:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 -- start start 2024-07-24T21:58:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 0x7fa6d40fd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 0x7fa6d40fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4111590 con 0x7fa6d4106fd0 2024-07-24T21:58:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4111410 con 0x7fa6d41015d0 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6dab61640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4111710 con 0x7fa6d4107950 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35804/0 (socket says 172.21.15.60:35804) 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6d3fff640 1 -- 172.21.15.60:0/1990216827 learned_addr learned my addr 172.21.15.60:0/1990216827 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 0x7fa6d40fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.565+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 0x7fa6d40fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 -- 172.21.15.60:0/1990216827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 msgr2=0x7fa6d40fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 0x7fa6d40fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 -- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 msgr2=0x7fa6d40fda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 -- 172.21.15.60:0/1990216827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d41a70d0 con 0x7fa6d41015d0 2024-07-24T21:58:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 0x7fa6d40fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 0x7fa6d40fd530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e9f0 tx=0x7fa6c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d1ffb640 1 -- 172.21.15.60:0/1990216827 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800ccc0 con 0x7fa6d41015d0 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d41a73c0 con 0x7fa6d41015d0 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d1ffb640 1 -- 172.21.15.60:0/1990216827 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c8005020 con 0x7fa6d41015d0 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6d1ffb640 1 -- 172.21.15.60:0/1990216827 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800ce60 con 0x7fa6d41015d0 2024-07-24T21:58:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.566+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41a78e0 con 0x7fa6d41015d0 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6d1ffb640 1 -- 172.21.15.60:0/1990216827 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa6c80051d0 con 0x7fa6d41015d0 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 Processor -- start 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 -- start start 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c67c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805c6a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b805bd1280 con 0x55b805c5b8c0 2024-07-24T21:58:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b805bc0780 con 0x55b805c68170 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.570+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b805bc47c0 con 0x55b805c5c1e0 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35818/0 (socket says 172.21.15.60:35818) 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 -- 172.21.15.60:0/527331730 learned_addr learned my addr 172.21.15.60:0/527331730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d1ffb640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 0x7fa6c40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c67c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805c6a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 0x7fa6c40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 -- 172.21.15.60:0/527331730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 msgr2=0x55b805c67c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c67c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 -- 172.21.15.60:0/527331730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 msgr2=0x55b805c6a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805c6a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 -- 172.21.15.60:0/527331730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b805924790 con 0x55b805c5b8c0 2024-07-24T21:58:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d1ffb640 1 -- 172.21.15.60:0/1990216827 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fa6c80e4020 con 0x7fa6d41015d0 2024-07-24T21:58:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.571+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0094d0 tx=0x7fa6cc00f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d68d11725fe2d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6beffd640 1 -- 172.21.15.60:0/527331730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01ee60 con 0x55b805c5b8c0 2024-07-24T21:58:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6beffd640 1 -- 172.21.15.60:0/527331730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6cc0108e0 con 0x55b805c5b8c0 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 0x7fa6c40b6200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc024000 tx=0x7fa6cc022000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6beffd640 1 -- 172.21.15.60:0/527331730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0277a0 con 0x55b805c5b8c0 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 -- 172.21.15.60:0/527331730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 msgr2=0x55b805c5bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0094d0 tx=0x7fa6cc00f7d0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 -- 172.21.15.60:0/527331730 shutdown_connections 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805c6a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c67c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/527331730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c5bca0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.572+0000 7fa6dd98b740 1 -- 172.21.15.60:0/527331730 >> 172.21.15.60:0/527331730 conn(0x55b805c596f0 msgr2=0x55b805c5a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- 172.21.15.60:0/527331730 shutdown_connections 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- 172.21.15.60:0/527331730 wait complete. 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- start start 2024-07-24T21:58:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 Processor -- start 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c68170 0x55b805c65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b805afaab0 con 0x55b805c5c1e0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b805b489e0 con 0x55b805c5b8c0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b805c6ae50 con 0x55b805c68170 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.573+0000 7fa6dd98b740 10 client.0 _ll_register_callbacks cb 0x55b8058d0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d90fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c68170 0x55b805c65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35828/0 (socket says 172.21.15.60:35828) 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 -- 172.21.15.60:0/2087971852 learned_addr learned my addr 172.21.15.60:0/2087971852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d88fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 -- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c68170 msgr2=0x55b805c65300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c68170 0x55b805c65300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 -- 172.21.15.60:0/2087971852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 msgr2=0x55b805c678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 -- 172.21.15.60:0/2087971852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80599d620 con 0x55b805c5c1e0 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000c00 tx=0x7fa6c0031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e209aff44c966b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0035040 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b805b6fce0 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.574+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fa6c0004450 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa6c0005020 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 10 client.17958 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b805c76600 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b805b6fce0 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa6c00035f0 con 0x55b805c5c1e0 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 10 client.17958 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 10 client.17958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 20 client.17958 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805c77b60 0x55b805c79fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 4 client.17958 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b805bd31d0 con 0x55b805c77b60 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.575+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805c77b60 0x55b805c79fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.576+0000 7fa6b3fff640 10 client.17958 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.576+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805c77b60 0x55b805c79fb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0c844f6f55570a server_cookie=d8fb26eebd88fd25 in_seq=0 out_seq=0 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6b3fff640 1 -- 172.21.15.60:0/2087971852 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b805bd31d0 con 0x55b805c77b60 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6b3fff640 10 client.17958 handle_command_reply: tid=0 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 client.17958 shutdown 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 msgr2=0x55b805c64dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000c00 tx=0x7fa6c0031810 comp rx=0 tx=0).stop 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 shutdown_connections 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805c77b60 0x55b805c79fb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c68170 0x55b805c65300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5c1e0 0x55b805c64dc0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6b3fff640 0 client.17958 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/2087971852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 >> 172.21.15.60:0/2087971852 conn(0x55b805c596f0 msgr2=0x55b805c5f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 shutdown_connections 2024-07-24T21:58:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 1 -- 172.21.15.60:0/2087971852 wait complete. 2024-07-24T21:58:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.577+0000 7fa6dd98b740 20 client.17958 trim_cache size 0 max 16384 2024-07-24T21:58:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.593+0000 7fa6dd98b740 1 Processor -- start 2024-07-24T21:58:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 -- start start 2024-07-24T21:58:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c5aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c7f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41b2df0 con 0x55b805c68170 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41b0600 con 0x55b805c5b8c0 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41ae470 con 0x55b805c5c1e0 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35832/0 (socket says 172.21.15.60:35832) 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 -- 172.21.15.60:0/3073023987 learned_addr learned my addr 172.21.15.60:0/3073023987 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c7f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c5aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 -- 172.21.15.60:0/3073023987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 msgr2=0x55b805c7f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c7f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 -- 172.21.15.60:0/3073023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 msgr2=0x55b805c5aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c5aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 -- 172.21.15.60:0/3073023987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b805c5c5c0 con 0x55b805c68170 2024-07-24T21:58:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.594+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc01e540 tx=0x7fa6cc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab932ebbe994259 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6be7fc640 1 -- 172.21.15.60:0/3073023987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc036070 con 0x55b805c68170 2024-07-24T21:58:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6be7fc640 1 -- 172.21.15.60:0/3073023987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6cc012050 con 0x55b805c68170 2024-07-24T21:58:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6be7fc640 1 -- 172.21.15.60:0/3073023987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc004c20 con 0x55b805c68170 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3073023987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 msgr2=0x55b805c7fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc01e540 tx=0x7fa6cc009520 comp rx=0 tx=0).stop 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3073023987 shutdown_connections 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c68170 0x55b805c7fdc0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c7f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3073023987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c5b8c0 0x55b805c5aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.595+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3073023987 >> 172.21.15.60:0/3073023987 conn(0x55b805c596f0 msgr2=0x55b805c5a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3073023987 shutdown_connections 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3073023987 wait complete. 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- start start 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 Processor -- start 2024-07-24T21:58:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c6ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805cda960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41aeac0 con 0x55b805c5b8c0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41aff30 con 0x55b805c68170 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.596+0000 7fa6dd98b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41af190 con 0x55b805c5c1e0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d88fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c6ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6dd98b740 10 client.0 _ll_register_callbacks cb 0x7fa6c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d88fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c6ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35838/0 (socket says 172.21.15.60:35838) 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d88fd640 1 -- 172.21.15.60:0/3169945218 learned_addr learned my addr 172.21.15.60:0/3169945218 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d90fe640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805cda960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 -- 172.21.15.60:0/3169945218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 msgr2=0x55b805cda960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805cda960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 -- 172.21.15.60:0/3169945218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 msgr2=0x55b805c6ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c6ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 -- 172.21.15.60:0/3169945218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b805cdb0a0 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6d3fff640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c6bfd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005940 tx=0x7fa6c0003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3df5f742c6daf68 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0035040 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7fa6c00051c0 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.597+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc012050 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa6c0004e60 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 10 client.27322 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b805ce2480 con 0x55b805c5c1e0 2024-07-24T21:58:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6cc012050 con 0x55b805c5c1e0 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa6c003d450 con 0x55b805c5c1e0 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 10 client.27322 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 20 client.27322 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805ce2c90 0x55b805ce50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 4 client.27322 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b805bcec70 con 0x55b805ce2c90 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.598+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805ce2c90 0x55b805ce50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.599+0000 7fa6d88fd640 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805ce2c90 0x55b805ce50b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904f945f6ff021ad server_cookie=676fae1c0c625a70 in_seq=0 out_seq=0 2024-07-24T21:58:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:34.599+0000 7fa6b3fff640 10 client.27322 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.705+0000 7f19e1ffb640 1 -- 172.21.15.60:0/40080227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4031810 con 0x7f19e4101360 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 -- 172.21.15.60:0/40080227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e4101360 msgr2=0x7f19e4101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 --2- 172.21.15.60:0/40080227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e4101360 0x7f19e4101740 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009900 tx=0x7f19d402a440 comp rx=0 tx=0).stop 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 -- 172.21.15.60:0/40080227 shutdown_connections 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 --2- 172.21.15.60:0/40080227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e410ee90 0x7f19e4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 --2- 172.21.15.60:0/40080227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101c80 0x7f19e410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 --2- 172.21.15.60:0/40080227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e4101360 0x7f19e4101740 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 -- 172.21.15.60:0/40080227 >> 172.21.15.60:0/40080227 conn(0x7f19e40fd120 msgr2=0x7f19e40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 -- 172.21.15.60:0/40080227 shutdown_connections 2024-07-24T21:58:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.706+0000 7f19e9406640 1 -- 172.21.15.60:0/40080227 wait complete. 2024-07-24T21:58:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.707+0000 7f19e9406640 1 Processor -- start 2024-07-24T21:58:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.707+0000 7f19e9406640 1 -- start start 2024-07-24T21:58:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e4101c80 0x7f19e4112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e410ee90 0x7f19e41174f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410b6e0 con 0x7f19e410ee90 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410b560 con 0x7f19e4101c80 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e9406640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410b860 con 0x7f19e4101360 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60414/0 (socket says 172.21.15.60:60414) 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1043613189 learned_addr learned my addr 172.21.15.60:0/1043613189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e27fc640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e4101c80 0x7f19e4112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.708+0000 7f19e37fe640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e410ee90 0x7f19e41174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1043613189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e4101c80 msgr2=0x7f19e4112150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e4101c80 0x7f19e4112150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e410ee90 msgr2=0x7f19e41174f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e410ee90 0x7f19e41174f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1043613189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e4117c70 con 0x7f19e4101360 2024-07-24T21:58:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.709+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19d402aa30 tx=0x7f19d4003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.712+0000 7f19cbfff640 1 -- 172.21.15.60:0/1043613189 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d403f070 con 0x7f19e4101360 2024-07-24T21:58:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.712+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e4117f00 con 0x7f19e4101360 2024-07-24T21:58:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.712+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e4118660 con 0x7f19e4101360 2024-07-24T21:58:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.712+0000 7f19cbfff640 1 -- 172.21.15.60:0/1043613189 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d403bd20 con 0x7f19e4101360 2024-07-24T21:58:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.712+0000 7f19cbfff640 1 -- 172.21.15.60:0/1043613189 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4035030 con 0x7f19e4101360 2024-07-24T21:58:35.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.716+0000 7f19eb7b7740 1 Processor -- start 2024-07-24T21:58:35.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.717+0000 7f19cbfff640 1 -- 172.21.15.60:0/1043613189 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f19d4042070 con 0x7f19e4101360 2024-07-24T21:58:35.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.721+0000 7f19eb7b7740 1 -- start start 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19cbfff640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 0x7f19cc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559138286d0 0x555913828ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x555913837370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55591379f9b0 con 0x555913828ff0 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55591378eeb0 con 0x555913834f80 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555913792ef0 con 0x5559138286d0 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19cbfff640 1 -- 172.21.15.60:0/1043613189 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f19d41058e0 con 0x7f19e4101360 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35862/0 (socket says 172.21.15.60:35862) 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 -- 172.21.15.60:0/641150550 learned_addr learned my addr 172.21.15.60:0/641150550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e37fe640 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x555913837370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559138286d0 0x555913828ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e37fe640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 0x7f19cc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 -- 172.21.15.60:0/641150550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559138286d0 msgr2=0x555913828ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559138286d0 0x555913828ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 -- 172.21.15.60:0/641150550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 msgr2=0x555913837370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x555913837370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.722+0000 7f19e27fc640 1 -- 172.21.15.60:0/641150550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913718320 con 0x555913828ff0 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.723+0000 7f19e37fe640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 0x7f19cc0b63d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0070b0 tx=0x7f19dc0093d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.723+0000 7f19e27fc640 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b7f0 tx=0x7f19d800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d61d9ce90d38b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.723+0000 7f19c97fa640 1 -- 172.21.15.60:0/641150550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d800d950 con 0x555913828ff0 2024-07-24T21:58:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.723+0000 7f19c97fa640 1 -- 172.21.15.60:0/641150550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d8018070 con 0x555913828ff0 2024-07-24T21:58:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.723+0000 7f19c97fa640 1 -- 172.21.15.60:0/641150550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80123e0 con 0x555913828ff0 2024-07-24T21:58:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 -- 172.21.15.60:0/641150550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 msgr2=0x555913834a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b7f0 tx=0x7f19d800bcb0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 -- 172.21.15.60:0/641150550 shutdown_connections 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x555913837370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555913828ff0 0x555913834a40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/641150550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559138286d0 0x555913828ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 -- 172.21.15.60:0/641150550 >> 172.21.15.60:0/641150550 conn(0x555913826500 msgr2=0x5559138276a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.724+0000 7f19eb7b7740 1 -- 172.21.15.60:0/641150550 shutdown_connections 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 -- 172.21.15.60:0/641150550 wait complete. 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 -- start start 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 Processor -- start 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913828ff0 0x555913831e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913834f80 0x5559138325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5559135f2700 con 0x5559138286d0 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559137315f0 con 0x555913828ff0 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555913837c00 con 0x555913834f80 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.725+0000 7f19eb7b7740 10 client.0 _ll_register_callbacks cb 0x555913751960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35864/0 (socket says 172.21.15.60:35864) 2024-07-24T21:58:35.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 -- 172.21.15.60:0/2330886436 learned_addr learned my addr 172.21.15.60:0/2330886436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 -- 172.21.15.60:0/2330886436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913834f80 msgr2=0x5559138325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913834f80 0x5559138325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 -- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913828ff0 msgr2=0x555913831e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913828ff0 0x555913831e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 -- 172.21.15.60:0/2330886436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555913724680 con 0x5559138286d0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f19d40c96a0 tx=0x7f19d40cbf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4881929743adc94 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.726+0000 7f19c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d40e6060 con 0x5559138286d0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f19d40d5050 con 0x5559138286d0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d8018070 con 0x5559138286d0 2024-07-24T21:58:35.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f19d4049040 con 0x5559138286d0 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19eb7b7740 10 client.17976 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559134ef650 con 0x5559138286d0 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.727+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d8018070 con 0x5559138286d0 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f19d403b870 con 0x5559138286d0 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 10 client.17976 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 10 client.17976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 20 client.17976 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x555913844ac0 0x555913846f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 4 client.17976 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559137a1900 con 0x555913844ac0 2024-07-24T21:58:35.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.728+0000 7f19e27fc640 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x555913844ac0 0x555913846f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.730+0000 7f19c6ff5640 10 client.17976 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:58:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:35.730+0000 7f19e27fc640 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x555913844ac0 0x555913846f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36d979d0a433a354 server_cookie=156523a5f07a9795 in_seq=0 out_seq=0 2024-07-24T21:58:36.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:35 smithi060 ceph-mon[33000]: pgmap v747: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-07-24T21:58:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:35 smithi123 ceph-mon[37143]: pgmap v747: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-07-24T21:58:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:35 smithi086 ceph-mon[37128]: pgmap v747: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 217 op/s 2024-07-24T21:58:38.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:37 smithi123 ceph-mon[37143]: pgmap v748: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-07-24T21:58:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:37 smithi086 ceph-mon[37128]: pgmap v748: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-07-24T21:58:38.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:37 smithi060 ceph-mon[33000]: pgmap v748: 97 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19c6ff5640 1 -- 172.21.15.60:0/2330886436 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559137a1900 con 0x555913844ac0 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19c6ff5640 10 client.17976 handle_command_reply: tid=0 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 client.17976 shutdown 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 msgr2=0x5559138349f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f19d40c96a0 tx=0x7f19d40cbf40 comp rx=0 tx=0).stop 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 shutdown_connections 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x555913844ac0 0x555913846f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.986+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913834f80 0x5559138325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913828ff0 0x555913831e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/2330886436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x5559138349f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19c6ff5640 0 client.17976 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:58:38.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 >> 172.21.15.60:0/2330886436 conn(0x555913826500 msgr2=0x55591382c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:38.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 shutdown_connections 2024-07-24T21:58:38.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 1 -- 172.21.15.60:0/2330886436 wait complete. 2024-07-24T21:58:38.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:38.987+0000 7f19eb7b7740 20 client.17976 trim_cache size 0 max 16384 2024-07-24T21:58:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 Processor -- start 2024-07-24T21:58:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 -- start start 2024-07-24T21:58:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x55591384c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x55591384cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19d00112d0 con 0x5559138286d0 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19d000eae0 con 0x555913834f80 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19d000c950 con 0x555913828ff0 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.002+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49490/0 (socket says 172.21.15.60:49490) 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1922724126 learned_addr learned my addr 172.21.15.60:0/1922724126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e37fe640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x55591384cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e27fc640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x55591384c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1922724126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 msgr2=0x55591384c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x55591384c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1922724126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 msgr2=0x55591384cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x55591384cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 -- 172.21.15.60:0/1922724126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913832940 con 0x5559138286d0 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e37fe640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x55591384cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e27fc640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x55591384c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cc3c0 tx=0x7f19d40d9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e4911c3e56a39f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19c8ff9640 1 -- 172.21.15.60:0/1922724126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d40d54e0 con 0x5559138286d0 2024-07-24T21:58:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.003+0000 7f19c8ff9640 1 -- 172.21.15.60:0/1922724126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d40eb030 con 0x5559138286d0 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19c8ff9640 1 -- 172.21.15.60:0/1922724126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d402ac40 con 0x5559138286d0 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 -- 172.21.15.60:0/1922724126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 msgr2=0x555913833b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cc3c0 tx=0x7f19d40d9000 comp rx=0 tx=0).stop 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 -- 172.21.15.60:0/1922724126 shutdown_connections 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x55591384cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x55591384c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/1922724126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913833b80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.004+0000 7f19eb7b7740 1 -- 172.21.15.60:0/1922724126 >> 172.21.15.60:0/1922724126 conn(0x555913826500 msgr2=0x555913833470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 -- 172.21.15.60:0/1922724126 shutdown_connections 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 -- 172.21.15.60:0/1922724126 wait complete. 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 -- start start 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 Processor -- start 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913838f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x5559138394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.005+0000 7f19eb7b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x5559138399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19d000cfa0 con 0x5559138286d0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19d000e410 con 0x555913834f80 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19eb7b7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19d000d670 con 0x555913828ff0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913838f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19eb7b7740 10 client.0 _ll_register_callbacks cb 0x7f19d40cc220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913838f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49498/0 (socket says 172.21.15.60:49498) 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e2ffd640 1 -- 172.21.15.60:0/825307672 learned_addr learned my addr 172.21.15.60:0/825307672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x5559138399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e27fc640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x5559138394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 -- 172.21.15.60:0/825307672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 msgr2=0x5559138394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x5559138394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 -- 172.21.15.60:0/825307672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 msgr2=0x555913838f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913838f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 -- 172.21.15.60:0/825307672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559138aea00 con 0x555913834f80 2024-07-24T21:58:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e27fc640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x5559138394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19e37fe640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x5559138399f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d040 tx=0x7f19dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379bb7e6c21f1a1d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:39.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.006+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc02b060 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f19dc01ab60 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d40eb030 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f19dc013040 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 10 client.27383 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5559138afd60 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d40eb030 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f19dc018380 con 0x555913834f80 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 10 client.27383 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 10 client.27383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 20 client.27383 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5559138b0570 0x5559138b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 4 client.27383 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:58:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55591379d3a0 con 0x5559138b0570 2024-07-24T21:58:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.007+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5559138b0570 0x5559138b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.008+0000 7f19c6ff5640 10 client.27383 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:58:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.008+0000 7f19e2ffd640 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5559138b0570 0x5559138b2990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcb22fe23a058db8 server_cookie=12df245c241beb9a in_seq=0 out_seq=0 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19c6ff5640 1 -- 172.21.15.60:0/825307672 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55591379d3a0 con 0x5559138b0570 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19c6ff5640 10 client.27383 handle_command_reply: tid=0 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 client.27383 shutdown 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 msgr2=0x5559138399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x5559138399f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d040 tx=0x7f19dc004a40 comp rx=0 tx=0).stop 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 shutdown_connections 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5559138b0570 0x5559138b2990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555913834f80 0x5559138399f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555913828ff0 0x5559138394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 --2- 172.21.15.60:0/825307672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5559138286d0 0x555913838f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19c6ff5640 0 client.27383 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 >> 172.21.15.60:0/825307672 conn(0x555913826500 msgr2=0x555913833c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 shutdown_connections 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 1 -- 172.21.15.60:0/825307672 wait complete. 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.079+0000 7f19eb7b7740 20 client.27383 trim_cache size 0 max 16384 2024-07-24T21:58:39.082 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 msgr2=0x7f19cc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 0x7f19cc0b63d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0070b0 tx=0x7f19dc0093d0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 msgr2=0x7f19e4111c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19d402aa30 tx=0x7f19d4003d50 comp rx=0 tx=0).stop 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 shutdown_connections 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19cc0b3f10 0x7f19cc0b63d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e410ee90 0x7f19e41174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19e4101c80 0x7f19e4112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 --2- 172.21.15.60:0/1043613189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e4101360 0x7f19e4111c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:39.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 >> 172.21.15.60:0/1043613189 conn(0x7f19e40fd120 msgr2=0x7f19e410deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:39.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 shutdown_connections 2024-07-24T21:58:39.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:39.080+0000 7f19e9406640 1 -- 172.21.15.60:0/1043613189 wait complete. 2024-07-24T21:58:39.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:39 smithi060 ceph-mon[33000]: pgmap v749: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-07-24T21:58:39.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:39 smithi123 ceph-mon[37143]: pgmap v749: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-07-24T21:58:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:39 smithi086 ceph-mon[37128]: pgmap v749: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-07-24T21:58:39.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:58:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.664+0000 7fc81effd640 1 -- 172.21.15.60:0/2529848566 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e9c5ec75c0 con 0x55e9c5f73ed0 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.664+0000 7fc81effd640 10 client.27353 handle_command_reply: tid=0 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.664+0000 7fc848533740 1 client.27353 shutdown 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 msgr2=0x55e9c5f6c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f6c580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc838030dd0 tx=0x7fc838030a90 comp rx=0 tx=0).stop 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 shutdown_connections 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e9c5f73ed0 0x55e9c5fe16d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9c5f63db0 0x55e9c5f6c580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9c5f594e0 0x55e9c5f6bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc81effd640 0 client.27353 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 --2- 172.21.15.60:0/2529848566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9c5f57f80 0x55e9c5f5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 >> 172.21.15.60:0/2529848566 conn(0x55e9c5f19720 msgr2=0x55e9c5f4b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 shutdown_connections 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 1 -- 172.21.15.60:0/2529848566 wait complete. 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.665+0000 7fc848533740 20 client.27353 trim_cache size 0 max 16384 2024-07-24T21:58:40.668 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 34, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "1378cc53": { 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 34, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:58:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.5, 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:40.670 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 msgr2=0x7fc8280b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 0x7fc8280b6280 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc838009620 tx=0x7fc838005fd0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 msgr2=0x7fc840199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc834009b20 tx=0x7fc834009f20 comp rx=0 tx=0).stop 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 shutdown_connections 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc8280b3dc0 0x7fc8280b6280 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc84010cac0 0x7fc84019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc840103f80 0x7fc840199010 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 --2- 172.21.15.60:0/3280621451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc840103660 0x7fc840198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 >> 172.21.15.60:0/3280621451 conn(0x7fc8400fd0c0 msgr2=0x7fc840104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 shutdown_connections 2024-07-24T21:58:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:40.666+0000 7fc846182640 1 -- 172.21.15.60:0/3280621451 wait complete. 2024-07-24T21:58:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.675+0000 7fa6b3fff640 1 -- 172.21.15.60:0/3169945218 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b805bcec70 con 0x55b805ce2c90 2024-07-24T21:58:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.675+0000 7fa6b3fff640 10 client.27322 handle_command_reply: tid=0 2024-07-24T21:58:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.675+0000 7fa6dd98b740 1 client.27322 shutdown 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 msgr2=0x55b805c6bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c6bfd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005940 tx=0x7fa6c0003e60 comp rx=0 tx=0).stop 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 shutdown_connections 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55b805ce2c90 0x55b805ce50b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b805c68170 0x55b805cda960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b805c5c1e0 0x55b805c6bfd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 --2- 172.21.15.60:0/3169945218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b805c5b8c0 0x55b805c6ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 >> 172.21.15.60:0/3169945218 conn(0x55b805c596f0 msgr2=0x55b805c5fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6b3fff640 0 client.27322 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 shutdown_connections 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 1 -- 172.21.15.60:0/3169945218 wait complete. 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.676+0000 7fa6dd98b740 20 client.27322 trim_cache size 0 max 16384 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:58:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:58:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 msgr2=0x7fa6c40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 0x7fa6c40b6200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc024000 tx=0x7fa6cc022000 comp rx=0 tx=0).stop 2024-07-24T21:58:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 msgr2=0x7fa6d40fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 0x7fa6d40fd530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e9f0 tx=0x7fa6c800edf0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 shutdown_connections 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa6c40b3d40 0x7fa6c40b6200 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4107950 0x7fa6d40fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6d4106fd0 0x7fa6d40fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 --2- 172.21.15.60:0/1990216827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d41015d0 0x7fa6d40fd530 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 >> 172.21.15.60:0/1990216827 conn(0x7fa6d40fd030 msgr2=0x7fa6d406b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 shutdown_connections 2024-07-24T21:58:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:40.677+0000 7fa6dab61640 1 -- 172.21.15.60:0/1990216827 wait complete. 2024-07-24T21:58:41.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:41 smithi123 ceph-mon[37143]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-07-24T21:58:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:41 smithi086 ceph-mon[37128]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-07-24T21:58:41.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:41 smithi060 ceph-mon[33000]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-07-24T21:58:42.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1378cc53', quiesce took 18.6 seconds. Will release after: 8.899999999999999 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 -- 172.21.15.60:0/2556373084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400107060 msgr2=0x7f04001074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 --2- 172.21.15.60:0/2556373084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400107060 0x7f04001074a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e802a410 comp rx=0 tx=0).stop 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 -- 172.21.15.60:0/2556373084 shutdown_connections 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 --2- 172.21.15.60:0/2556373084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 0x7f040010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 --2- 172.21.15.60:0/2556373084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400107060 0x7f04001074a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 --2- 172.21.15.60:0/2556373084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400101650 0x7f0400101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 -- 172.21.15.60:0/2556373084 >> 172.21.15.60:0/2556373084 conn(0x7f04000fd0c0 msgr2=0x7f04000ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 -- 172.21.15.60:0/2556373084 shutdown_connections 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.613+0000 7f04060d5640 1 -- 172.21.15.60:0/2556373084 wait complete. 2024-07-24T21:58:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 Processor -- start 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 -- start start 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400107060 0x7f040019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 0x7f04001a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f04001115e0 con 0x7f0400101650 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111460 con 0x7f04001079e0 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f04060d5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111760 con 0x7f0400107060 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49524/0 (socket says 172.21.15.60:49524) 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.614+0000 7f03ff7fe640 1 -- 172.21.15.60:0/3215413280 learned_addr learned my addr 172.21.15.60:0/3215413280 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400107060 0x7f040019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03fffff640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 0x7f04001a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 -- 172.21.15.60:0/3215413280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400107060 msgr2=0x7f040019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400107060 0x7f040019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 -- 172.21.15.60:0/3215413280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 msgr2=0x7f04001a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 0x7f04001a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 -- 172.21.15.60:0/3215413280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04001a0f40 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03f000d950 tx=0x7f03f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03fd7fa640 1 -- 172.21.15.60:0/3215413280 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0017070 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04001a1230 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03fd7fa640 1 -- 172.21.15.60:0/3215413280 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f0009c30 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f03fd7fa640 1 -- 172.21.15.60:0/3215413280 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000f620 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.615+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a7ac0 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 Processor -- start 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 -- start start 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03fd7fa640 1 -- 172.21.15.60:0/3215413280 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f03f000f7c0 con 0x7f0400101650 2024-07-24T21:58:42.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c71cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c728a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560a8c6939b0 con 0x560a8c728f80 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a8c682eb0 con 0x560a8c71cff0 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f0408486740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a8c686ef0 con 0x560a8c71c6d0 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49540/0 (socket says 172.21.15.60:49540) 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2755474417 learned_addr learned my addr 172.21.15.60:0/2755474417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c728a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.618+0000 7f03fffff640 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c71cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2755474417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 msgr2=0x560a8c71cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c71cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2755474417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 msgr2=0x560a8c728a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c728a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2755474417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a8c60c320 con 0x560a8c728f80 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e8032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3382dccc5ad130c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03fd7fa640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 0x7f03ec0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f5ffb640 1 -- 172.21.15.60:0/2755474417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8040070 con 0x560a8c728f80 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03fffff640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 0x7f03ec0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03fd7fa640 1 -- 172.21.15.60:0/3215413280 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f03f00e3be0 con 0x7f0400101650 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f5ffb640 1 -- 172.21.15.60:0/2755474417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e80316a0 con 0x560a8c728f80 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.619+0000 7f03f5ffb640 1 -- 172.21.15.60:0/2755474417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035040 con 0x560a8c728f80 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 -- 172.21.15.60:0/2755474417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 msgr2=0x560a8c72b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f03fffff640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 0x7f03ec0b6200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f03f800aec0 tx=0x7f03f800a4d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e8032e60 comp rx=0 tx=0).stop 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 -- 172.21.15.60:0/2755474417 shutdown_connections 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c72b370 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c728a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 --2- 172.21.15.60:0/2755474417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c71cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 -- 172.21.15.60:0/2755474417 >> 172.21.15.60:0/2755474417 conn(0x560a8c71a500 msgr2=0x560a8c71b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 -- 172.21.15.60:0/2755474417 shutdown_connections 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.620+0000 7f0408486740 1 -- 172.21.15.60:0/2755474417 wait complete. 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 -- start start 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 Processor -- start 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c725f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c726990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560a8c4e6700 con 0x560a8c71cff0 2024-07-24T21:58:42.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a8c6255f0 con 0x560a8c728f80 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a8c72bba0 con 0x560a8c71c6d0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f0408486740 10 client.0 _ll_register_callbacks cb 0x560a8c645960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49548/0 (socket says 172.21.15.60:49548) 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2938228124 learned_addr learned my addr 172.21.15.60:0/2938228124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.621+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c725f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03fffff640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c726990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 msgr2=0x560a8c725f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c725f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2938228124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 msgr2=0x560a8c726990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c726990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2938228124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a8c618680 con 0x560a8c71cff0 2024-07-24T21:58:42.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c725f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f03e802abe0 tx=0x7f03e8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05be1ffdf38caa0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03deffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035040 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f03e8004230 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e80316a0 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.622+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f03e8040040 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 10 client.17994 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560a8c3e3650 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e80316a0 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f03e804d4c0 con 0x560a8c71cff0 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 10 client.17994 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 20 client.17994 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:42.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7388b0 0x560a8c73aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 4 client.17994 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a8c695900 con 0x560a8c7388b0 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.623+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7388b0 0x560a8c73aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7388b0 0x560a8c73aca0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce890b53d47c8ce server_cookie=536f230efd122b84 in_seq=0 out_seq=0 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f03deffd640 10 client.17994 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f03deffd640 1 -- 172.21.15.60:0/2938228124 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a8c695900 con 0x560a8c7388b0 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f03deffd640 10 client.17994 handle_command_reply: tid=0 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f0408486740 1 client.17994 shutdown 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.624+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 msgr2=0x560a8c726450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f03e802abe0 tx=0x7f03e8032b20 comp rx=0 tx=0).stop 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 shutdown_connections 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7388b0 0x560a8c73aca0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c726990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c726450 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 --2- 172.21.15.60:0/2938228124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c725f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 >> 172.21.15.60:0/2938228124 conn(0x560a8c71a500 msgr2=0x560a8c7200a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f03deffd640 0 client.17994 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 shutdown_connections 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 1 -- 172.21.15.60:0/2938228124 wait complete. 2024-07-24T21:58:42.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.625+0000 7f0408486740 20 client.17994 trim_cache size 0 max 16384 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 Processor -- start 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 -- start start 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c723120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c725e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b2e30 con 0x560a8c71cff0 2024-07-24T21:58:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b0640 con 0x560a8c728f80 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f0408486740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04001ae4b0 con 0x560a8c71c6d0 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f03f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49558/0 (socket says 172.21.15.60:49558) 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.639+0000 7f03f6ffd640 1 -- 172.21.15.60:0/1635254729 learned_addr learned my addr 172.21.15.60:0/1635254729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c723120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03fffff640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c725e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 -- 172.21.15.60:0/1635254729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 msgr2=0x560a8c723120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c723120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 -- 172.21.15.60:0/1635254729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 msgr2=0x560a8c725e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c725e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 -- 172.21.15.60:0/1635254729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a8c72b6b0 con 0x560a8c71cff0 2024-07-24T21:58:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03fffff640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c725e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f03e802a930 tx=0x7f03e8032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7ef31c711d4111 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f57fa640 1 -- 172.21.15.60:0/1635254729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8059020 con 0x560a8c71cff0 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f57fa640 1 -- 172.21.15.60:0/1635254729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e804dd00 con 0x560a8c71cff0 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.640+0000 7f03f57fa640 1 -- 172.21.15.60:0/1635254729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035070 con 0x560a8c71cff0 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- 172.21.15.60:0/1635254729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 msgr2=0x560a8c723660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f03e802a930 tx=0x7f03e8032d40 comp rx=0 tx=0).stop 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- 172.21.15.60:0/1635254729 shutdown_connections 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c728f80 0x560a8c725e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c71cff0 0x560a8c723660 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 --2- 172.21.15.60:0/1635254729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c723120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- 172.21.15.60:0/1635254729 >> 172.21.15.60:0/1635254729 conn(0x560a8c71a500 msgr2=0x560a8c7211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- 172.21.15.60:0/1635254729 shutdown_connections 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- 172.21.15.60:0/1635254729 wait complete. 2024-07-24T21:58:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 -- start start 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.641+0000 7f0408486740 1 Processor -- start 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c73b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c73b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f04001aeb00 con 0x560a8c728f80 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04001aff70 con 0x560a8c71cff0 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04001af1d0 con 0x560a8c71c6d0 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f0408486740 10 client.0 _ll_register_callbacks cb 0x7f03e8032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49572/0 (socket says 172.21.15.60:49572) 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03fffff640 1 -- 172.21.15.60:0/2335945723 learned_addr learned my addr 172.21.15.60:0/2335945723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c73b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c73b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 msgr2=0x560a8c73b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c73b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2335945723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 msgr2=0x560a8c79b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03f6ffd640 1 -- 172.21.15.60:0/2335945723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a8c79b8c0 con 0x560a8c71cff0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.642+0000 7f03fffff640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03f6ffd640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c73b700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61557d6761e3a0db server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03deffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035020 con 0x560a8c71cff0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8609+0+0 (secure 0 0 0) 0x7f03e8004440 con 0x560a8c71cff0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e804dd00 con 0x560a8c71cff0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f03e8059070 con 0x560a8c71cff0 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f0408486740 10 client.27395 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560a8c7a2ca0 con 0x560a8c71cff0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.643+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e804dd00 con 0x560a8c71cff0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f03e803c440 con 0x560a8c71cff0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 10 client.27395 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 20 client.27395 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7a32a0 0x560a8c7a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 4 client.27395 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a8c725770 con 0x560a8c7a32a0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7a32a0 0x560a8c7a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03deffd640 10 client.27395 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03ff7fe640 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7a32a0 0x560a8c7a56c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0de2fa232a284a server_cookie=94f639408e7af27c in_seq=0 out_seq=0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03deffd640 1 -- 172.21.15.60:0/2335945723 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a8c725770 con 0x560a8c7a32a0 2024-07-24T21:58:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f03deffd640 10 client.27395 handle_command_reply: tid=0 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.644+0000 7f0408486740 1 client.27395 shutdown 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 msgr2=0x560a8c73b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c73b700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e8005fd0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 shutdown_connections 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560a8c7a32a0 0x560a8c7a56c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a8c728f80 0x560a8c79b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a8c71cff0 0x560a8c73b700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 --2- 172.21.15.60:0/2335945723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a8c71c6d0 0x560a8c73b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 >> 172.21.15.60:0/2335945723 conn(0x560a8c71a500 msgr2=0x560a8c727540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f03deffd640 0 client.27395 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 shutdown_connections 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 1 -- 172.21.15.60:0/2335945723 wait complete. 2024-07-24T21:58:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f0408486740 20 client.27395 trim_cache size 0 max 16384 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 msgr2=0x7f03ec0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.645+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 0x7f03ec0b6200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f03f800aec0 tx=0x7f03f800a4d0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 msgr2=0x7f040019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03f000d950 tx=0x7f03f000de10 comp rx=0 tx=0).stop 2024-07-24T21:58:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 shutdown_connections 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f03ec0b3d40 0x7f03ec0b6200 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04001079e0 0x7f04001a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0400107060 0x7f040019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 --2- 172.21.15.60:0/3215413280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0400101650 0x7f040019ae90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 >> 172.21.15.60:0/3215413280 conn(0x7f04000fd0c0 msgr2=0x7f0400104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 shutdown_connections 2024-07-24T21:58:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:42.646+0000 7f04060d5640 1 -- 172.21.15.60:0/3215413280 wait complete. 2024-07-24T21:58:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:43 smithi123 ceph-mon[37143]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-24T21:58:43.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:43 smithi123 ceph-mon[37143]: osdmap e101: 12 total, 12 up, 12 in 2024-07-24T21:58:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:43 smithi086 ceph-mon[37128]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-24T21:58:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:43 smithi086 ceph-mon[37128]: osdmap e101: 12 total, 12 up, 12 in 2024-07-24T21:58:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:43 smithi060 ceph-mon[33000]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-24T21:58:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:43 smithi060 ceph-mon[33000]: osdmap e101: 12 total, 12 up, 12 in 2024-07-24T21:58:44.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:58:44.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:58:44.673 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:45.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:45 smithi123 ceph-mon[37143]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 519 KiB/s wr, 165 op/s 2024-07-24T21:58:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:45 smithi086 ceph-mon[37128]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 519 KiB/s wr, 165 op/s 2024-07-24T21:58:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:45 smithi060 ceph-mon[33000]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 519 KiB/s wr, 165 op/s 2024-07-24T21:58:46.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 -- 172.21.15.60:0/2631741118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14107080 msgr2=0x7f6e141074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 --2- 172.21.15.60:0/2631741118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14107080 0x7f6e141074c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009900 tx=0x7f6e0402a410 comp rx=0 tx=0).stop 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 -- 172.21.15.60:0/2631741118 shutdown_connections 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 --2- 172.21.15.60:0/2631741118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107a00 0x7f6e1410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 --2- 172.21.15.60:0/2631741118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14107080 0x7f6e141074c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 --2- 172.21.15.60:0/2631741118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14101670 0x7f6e14101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 -- 172.21.15.60:0/2631741118 >> 172.21.15.60:0/2631741118 conn(0x7f6e140fd0c0 msgr2=0x7f6e140ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 -- 172.21.15.60:0/2631741118 shutdown_connections 2024-07-24T21:58:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.409+0000 7f6e1c759640 1 -- 172.21.15.60:0/2631741118 wait complete. 2024-07-24T21:58:46.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 Processor -- start 2024-07-24T21:58:46.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 -- start start 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107080 0x7f6e1410ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14107a00 0x7f6e1410f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141127e0 con 0x7f6e14101670 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14112660 con 0x7f6e14107080 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.410+0000 7f6e1c759640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14112960 con 0x7f6e14107a00 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51726/0 (socket says 172.21.15.60:51726) 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/763353949 learned_addr learned my addr 172.21.15.60:0/763353949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14107a00 0x7f6e1410f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107080 0x7f6e1410ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/763353949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14107a00 msgr2=0x7f6e1410f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14107a00 0x7f6e1410f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/763353949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107080 msgr2=0x7f6e1410ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107080 0x7f6e1410ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/763353949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141ab690 con 0x7f6e14101670 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b4a0 tx=0x7f6e1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e0b7fe640 1 -- 172.21.15.60:0/763353949 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10014070 con 0x7f6e14101670 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e0b7fe640 1 -- 172.21.15.60:0/763353949 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e10010ac0 con 0x7f6e14101670 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e0b7fe640 1 -- 172.21.15.60:0/763353949 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10017c30 con 0x7f6e14101670 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e141ab860 con 0x7f6e14101670 2024-07-24T21:58:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.411+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141abd20 con 0x7f6e14101670 2024-07-24T21:58:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 Processor -- start 2024-07-24T21:58:46.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 -- start start 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e0b7fe640 1 -- 172.21.15.60:0/763353949 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6e10017070 con 0x7f6e14101670 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe6519f0 con 0x55d0fe6da840 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe640ef0 con 0x55d0fe6e67d0 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe644f30 con 0x55d0fe6d9f20 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51734/0 (socket says 172.21.15.60:51734) 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/1098601178 learned_addr learned my addr 172.21.15.60:0/1098601178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.415+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/1098601178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 msgr2=0x55d0fe6da300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6da300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/1098601178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6e67d0 msgr2=0x55d0fe6e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/1098601178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fe3a1930 con 0x55d0fe6da840 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6e100e0950 tx=0x7f6e100eb890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbe3d5108608853 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e08ff9640 1 -- 172.21.15.60:0/1098601178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e100f6070 con 0x55d0fe6da840 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e08ff9640 1 -- 172.21.15.60:0/1098601178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e100e3ae0 con 0x55d0fe6da840 2024-07-24T21:58:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e08ff9640 1 -- 172.21.15.60:0/1098601178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e100f1a30 con 0x55d0fe6da840 2024-07-24T21:58:46.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e0b7fe640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 0x7f6e000b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 0x7f6e000b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.416+0000 7f6e0b7fe640 1 -- 172.21.15.60:0/763353949 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f6e100e3110 con 0x7f6e14101670 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1098601178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 msgr2=0x55d0fe6e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 0x7f6e000b6280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005fd0 tx=0x7f6e0c009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6e100e0950 tx=0x7f6e100eb890 comp rx=0 tx=0).stop 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1098601178 shutdown_connections 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e6290 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1098601178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6da300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.417+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1098601178 >> 172.21.15.60:0/1098601178 conn(0x55d0fe6d7d50 msgr2=0x55d0fe6d8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1098601178 shutdown_connections 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1098601178 wait complete. 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 -- start start 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 Processor -- start 2024-07-24T21:58:46.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.418+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe4a4890 con 0x55d0fe6da840 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe5e3390 con 0x55d0fe6d9f20 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe6e9450 con 0x55d0fe6e67d0 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1eb0a740 10 client.0 _ll_register_callbacks cb 0x55d0fe5ed500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51748/0 (socket says 172.21.15.60:51748) 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 -- 172.21.15.60:0/1144396438 learned_addr learned my addr 172.21.15.60:0/1144396438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 -- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6e67d0 msgr2=0x55d0fe6e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 -- 172.21.15.60:0/1144396438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 msgr2=0x55d0fe6e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 -- 172.21.15.60:0/1144396438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fe5ddd60 con 0x55d0fe6da840 2024-07-24T21:58:46.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:46.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009900 tx=0x7f6e040316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82f4a16d098ae4d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.419+0000 7f6dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04035040 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f6e04031b10 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e100e3ae0 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6e04031d10 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6e1eb0a740 10 client.18012 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.421+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d0fe6f5380 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e100e3ae0 con 0x55d0fe6da840 2024-07-24T21:58:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6e04004750 con 0x55d0fe6da840 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 10 client.18012 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 20 client.18012 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe6f6840 0x55d0fe6f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 4 client.18012 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fe653940 con 0x55d0fe6f6840 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe6f6840 0x55d0fe6f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe6f6840 0x55d0fe6f8cc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d892dd49c1b44408 server_cookie=bd595ac075478c8f in_seq=0 out_seq=0 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.422+0000 7f6dfdffb640 10 client.18012 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1144396438 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0fe653940 con 0x55d0fe6f6840 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6dfdffb640 10 client.18012 handle_command_reply: tid=0 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 client.18012 shutdown 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 msgr2=0x55d0fe6e3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009900 tx=0x7f6e040316d0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 shutdown_connections 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe6f6840 0x55d0fe6f8cc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6da840 0x55d0fe6e3c90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6dfdffb640 0 client.18012 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1144396438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 >> 172.21.15.60:0/1144396438 conn(0x55d0fe6d7d50 msgr2=0x55d0fe6dd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.423+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 shutdown_connections 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.424+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1144396438 wait complete. 2024-07-24T21:58:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.424+0000 7f6e1eb0a740 20 client.18012 trim_cache size 0 max 16384 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 Processor -- start 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 -- start start 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6da840 0x55d0fe6fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b71f0 con 0x55d0fe6e67d0 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b4a00 con 0x55d0fe6d9f20 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b2870 con 0x55d0fe6da840 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1acf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1acf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51758/0 (socket says 172.21.15.60:51758) 2024-07-24T21:58:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1546631548 learned_addr learned my addr 172.21.15.60:0/1546631548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.442+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1546631548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6da840 msgr2=0x55d0fe6fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6da840 0x55d0fe6fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1546631548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 msgr2=0x55d0fe6de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1546631548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fe6d7b80 con 0x55d0fe6e67d0 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c011870 tx=0x7f6e0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed607dd3fe4ba0e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6dfffff640 1 -- 172.21.15.60:0/1546631548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c003970 con 0x55d0fe6e67d0 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6dfffff640 1 -- 172.21.15.60:0/1546631548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0c004050 con 0x55d0fe6e67d0 2024-07-24T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.443+0000 7f6dfffff640 1 -- 172.21.15.60:0/1546631548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c020680 con 0x55d0fe6e67d0 2024-07-24T21:58:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1546631548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 msgr2=0x55d0fe6feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c011870 tx=0x7f6e0c016000 comp rx=0 tx=0).stop 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1546631548 shutdown_connections 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6feb10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6da840 0x55d0fe6fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1546631548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1546631548 >> 172.21.15.60:0/1546631548 conn(0x55d0fe6d7d50 msgr2=0x55d0fe6e55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1546631548 shutdown_connections 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1546631548 wait complete. 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 -- start start 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 Processor -- start 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.444+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 0x55d0fe6eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b2ec0 con 0x55d0fe6e67d0 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b4330 con 0x55d0fe6da840 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b3590 con 0x55d0fe6d9f20 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1eb0a740 10 client.0 _ll_register_callbacks cb 0x7f6e04032d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e19cf4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 0x55d0fe6eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1a4f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60666/0 (socket says 172.21.15.60:60666) 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1a4f5640 1 -- 172.21.15.60:0/1706808034 learned_addr learned my addr 172.21.15.60:0/1706808034 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51766/0 (socket says 172.21.15.60:51766) 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 msgr2=0x55d0fe6eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1706808034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 msgr2=0x55d0fe6eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 0x55d0fe6eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 -- 172.21.15.60:0/1706808034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fe6ebc70 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e19cf4640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 0x55d0fe6eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6e1acf6640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c011870 tx=0x7f6e0c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efde98fe3ea5fb9a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.445+0000 7f6dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c035420 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0c004050 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f6e0c026040 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c0355c0 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 10 client.18018 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d0fe7624a0 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0c004050 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6e0c00f040 con 0x55d0fe6e67d0 2024-07-24T21:58:46.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 10 client.18018 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 20 client.18018 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe762c30 0x55d0fe765020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 4 client.18018 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0fe64f3e0 con 0x55d0fe762c30 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.446+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe762c30 0x55d0fe765020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.447+0000 7f6dfdffb640 10 client.18018 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.447+0000 7f6e1a4f5640 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe762c30 0x55d0fe765020 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0308fb52796e16b server_cookie=2b4430146651131b in_seq=0 out_seq=0 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.447+0000 7f6dfdffb640 1 -- 172.21.15.60:0/1706808034 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0fe64f3e0 con 0x55d0fe762c30 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.447+0000 7f6dfdffb640 10 client.18018 handle_command_reply: tid=0 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.447+0000 7f6e1eb0a740 1 client.18018 shutdown 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 msgr2=0x55d0fe6eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c011870 tx=0x7f6e0c003ba0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 shutdown_connections 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d0fe762c30 0x55d0fe765020 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d0fe6e67d0 0x55d0fe6eb600 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0fe6da840 0x55d0fe6eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 --2- 172.21.15.60:0/1706808034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe6d9f20 0x55d0fe6eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 >> 172.21.15.60:0/1706808034 conn(0x55d0fe6d7d50 msgr2=0x55d0fe6d65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6dfdffb640 0 client.18018 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 shutdown_connections 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 1 -- 172.21.15.60:0/1706808034 wait complete. 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1eb0a740 20 client.18018 trim_cache size 0 max 16384 2024-07-24T21:58:46.454 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "ed4ea5c0-c190-405a-b90b-826c937e9bc3", 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 msgr2=0x7f6e000b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 0x7f6e000b6280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005fd0 tx=0x7f6e0c009600 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 msgr2=0x7f6e141107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.448+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b4a0 tx=0x7f6e1000b960 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 shutdown_connections 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6e000b3dc0 0x7f6e000b6280 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14107a00 0x7f6e1410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e14107080 0x7f6e1410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 --2- 172.21.15.60:0/763353949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e14101670 0x7f6e141107c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 >> 172.21.15.60:0/763353949 conn(0x7f6e140fd0c0 msgr2=0x7f6e140ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 shutdown_connections 2024-07-24T21:58:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:46.449+0000 7f6e1c759640 1 -- 172.21.15.60:0/763353949 wait complete. 2024-07-24T21:58:47.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 362 KiB/s wr, 140 op/s 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:58:47.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:47 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:58:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 362 KiB/s wr, 140 op/s 2024-07-24T21:58:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:58:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:58:47.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:58:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:58:47.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:47 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:58:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 362 KiB/s wr, 140 op/s 2024-07-24T21:58:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:58:47.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:47 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:58:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:48 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:48.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:48 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:48 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:49.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:49 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:49.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:49 smithi060 ceph-mon[33000]: pgmap v755: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 95 KiB/s wr, 253 op/s 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.641+0000 7f5d18527640 1 -- 172.21.15.60:0/2109646612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103660 msgr2=0x7f5d10103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.641+0000 7f5d18527640 1 --2- 172.21.15.60:0/2109646612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103660 0x7f5d10103a40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009900 tx=0x7f5d0c02a410 comp rx=0 tx=0).stop 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 -- 172.21.15.60:0/2109646612 shutdown_connections 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 --2- 172.21.15.60:0/2109646612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d1010cac0 0x7f5d1010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 --2- 172.21.15.60:0/2109646612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d1010c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 --2- 172.21.15.60:0/2109646612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103660 0x7f5d10103a40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 -- 172.21.15.60:0/2109646612 >> 172.21.15.60:0/2109646612 conn(0x7f5d100fd0a0 msgr2=0x7f5d100ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 -- 172.21.15.60:0/2109646612 shutdown_connections 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 -- 172.21.15.60:0/2109646612 wait complete. 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.642+0000 7f5d18527640 1 Processor -- start 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 -- start start 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d10103660 0x7f5d10101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1010cac0 0x7f5d10101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10112700 con 0x7f5d10103660 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10112580 con 0x7f5d10103f80 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d18527640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10112880 con 0x7f5d1010cac0 2024-07-24T21:58:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39516/0 (socket says 172.21.15.60:39516) 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 -- 172.21.15.60:0/1478935478 learned_addr learned my addr 172.21.15.60:0/1478935478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1010cac0 0x7f5d10101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d162c3640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d10103660 0x7f5d10101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 -- 172.21.15.60:0/1478935478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1010cac0 msgr2=0x7f5d10101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1010cac0 0x7f5d10101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 -- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d10103660 msgr2=0x7f5d10101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.643+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d10103660 0x7f5d10101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d15ac2640 1 -- 172.21.15.60:0/1478935478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d101a70f0 con 0x7f5d10103f80 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9c0 tx=0x7f5d0000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d077fe640 1 -- 172.21.15.60:0/1478935478 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000cd80 con 0x7f5d10103f80 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101a73e0 con 0x7f5d10103f80 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d077fe640 1 -- 172.21.15.60:0/1478935478 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d000096c0 con 0x7f5d10103f80 2024-07-24T21:58:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d077fe640 1 -- 172.21.15.60:0/1478935478 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00018840 con 0x7f5d10103f80 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.644+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a78b0 con 0x7f5d10103f80 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.647+0000 7f5d077fe640 1 -- 172.21.15.60:0/1478935478 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5d0001f020 con 0x7f5d10103f80 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.647+0000 7f5d1a8d8740 1 Processor -- start 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.647+0000 7f5d1a8d8740 1 -- start start 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55af94069990 con 0x55af94107770 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af94058e90 con 0x55af940f26b0 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af9405ced0 con 0x55af940f2fd0 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d16ac4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51792/0 (socket says 172.21.15.60:51792) 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d16ac4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39526/0 (socket says 172.21.15.60:39526) 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.648+0000 7f5d15ac2640 1 -- 172.21.15.60:0/2007279409 learned_addr learned my addr 172.21.15.60:0/2007279409 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d162c3640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 -- 172.21.15.60:0/2007279409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 msgr2=0x55af94107230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d077fe640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 0x7f5cfc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94107230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 -- 172.21.15.60:0/2007279409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 msgr2=0x55af94109b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 -- 172.21.15.60:0/2007279409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af93fe2300 con 0x55af940f26b0 2024-07-24T21:58:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d162c3640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d077fe640 1 -- 172.21.15.60:0/1478935478 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f5d000e3f60 con 0x7f5d10103f80 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d08005fd0 tx=0x7f5d08011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebc78bc344471b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 0x7f5cfc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.649+0000 7f5d04ff9640 1 -- 172.21.15.60:0/2007279409 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003c70 con 0x55af940f26b0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d04ff9640 1 -- 172.21.15.60:0/2007279409 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d080046a0 con 0x55af940f26b0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d04ff9640 1 -- 172.21.15.60:0/2007279409 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0801a3c0 con 0x55af940f26b0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/2007279409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 msgr2=0x55af940f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d08005fd0 tx=0x7f5d08011810 comp rx=0 tx=0).stop 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 0x7f5cfc0b6280 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d08003fd0 tx=0x7f5d0801aa70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/2007279409 shutdown_connections 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/2007279409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af940f2a90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.650+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/2007279409 >> 172.21.15.60:0/2007279409 conn(0x55af940f04e0 msgr2=0x55af940f1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/2007279409 shutdown_connections 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/2007279409 wait complete. 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- start start 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 Processor -- start 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af941046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55af93ebc700 con 0x55af94107770 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af93ffb5d0 con 0x55af940f26b0 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.651+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55af9410a420 con 0x55af940f2fd0 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d1a8d8740 10 client.0 _ll_register_callbacks cb 0x55af9401b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:60714/0 (socket says 172.21.15.60:60714) 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3407674541 learned_addr learned my addr 172.21.15.60:0/3407674541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af941046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3407674541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 msgr2=0x55af941046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af941046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3407674541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 msgr2=0x55af94105120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94105120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3407674541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af93fee660 con 0x55af940f2fd0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d000a6bc0 tx=0x7f5d00010da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baeb4c11eff40020 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:58:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d000aa910 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.652+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f5d000e7c60 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d080046a0 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5d000aa2b0 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 10 client.27370 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55af93db9650 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d080046a0 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5d000aada0 con 0x55af940f2fd0 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 10 client.27370 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 10 client.27370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 20 client.27370 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af94117150 0x55af94119560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 4 client.27370 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.653+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9406b8e0 con 0x55af94117150 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.654+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af94117150 0x55af94119560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.654+0000 7f5cf9ffb640 10 client.27370 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.654+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af94117150 0x55af94119560 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd81f51ee816989 server_cookie=87a912533b421bb0 in_seq=0 out_seq=0 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3407674541 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af9406b8e0 con 0x55af94117150 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5cf9ffb640 10 client.27370 handle_command_reply: tid=0 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 client.27370 shutdown 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 msgr2=0x55af94104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d000a6bc0 tx=0x7f5d00010da0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 shutdown_connections 2024-07-24T21:58:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af94117150 0x55af94119560 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f2fd0 0x55af94104be0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3407674541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f26b0 0x55af941046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 >> 172.21.15.60:0/3407674541 conn(0x55af940f04e0 msgr2=0x55af940f6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5cf9ffb640 0 client.27370 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.655+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 shutdown_connections 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.656+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3407674541 wait complete. 2024-07-24T21:58:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.656+0000 7f5d1a8d8740 20 client.27370 trim_cache size 0 max 16384 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.679+0000 7f5d1a8d8740 1 Processor -- start 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 -- start start 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 0x55af94102200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af94104670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101aea00 con 0x55af94107770 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101ac210 con 0x55af940f2fd0 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d1a8d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101aa080 con 0x55af940f26b0 2024-07-24T21:58:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d16ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d16ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51806/0 (socket says 172.21.15.60:51806) 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d16ac4640 1 -- 172.21.15.60:0/3176087765 learned_addr learned my addr 172.21.15.60:0/3176087765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 0x55af94102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af94104670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 -- 172.21.15.60:0/3176087765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 msgr2=0x55af94104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af94104670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 -- 172.21.15.60:0/3176087765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 msgr2=0x55af94104bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94104bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.680+0000 7f5d162c3640 1 -- 172.21.15.60:0/3176087765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af94109ed0 con 0x55af940f26b0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.681+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 0x55af94102200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009230 tx=0x7f5d0c005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d2414852315ba1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.681+0000 7f5cfbfff640 1 -- 172.21.15.60:0/3176087765 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c049070 con 0x55af940f26b0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.681+0000 7f5cfbfff640 1 -- 172.21.15.60:0/3176087765 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c0054f0 con 0x55af940f26b0 2024-07-24T21:58:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.681+0000 7f5cfbfff640 1 -- 172.21.15.60:0/3176087765 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c035040 con 0x55af940f26b0 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3176087765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 msgr2=0x55af94102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 0x55af94102200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009230 tx=0x7f5d0c005890 comp rx=0 tx=0).stop 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3176087765 shutdown_connections 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af94107770 0x55af94104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af94104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3176087765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af940f26b0 0x55af94102200 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3176087765 >> 172.21.15.60:0/3176087765 conn(0x55af940f04e0 msgr2=0x55af94102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3176087765 shutdown_connections 2024-07-24T21:58:49.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3176087765 wait complete. 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 -- start start 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.682+0000 7f5d1a8d8740 1 Processor -- start 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 0x55af9411ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d1a8d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af9411f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d162c3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 0x55af9411ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d162c3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 0x55af9411ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51816/0 (socket says 172.21.15.60:51816) 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d162c3640 1 -- 172.21.15.60:0/3657788977 learned_addr learned my addr 172.21.15.60:0/3657788977 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.683+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af94107770 0x55af9411f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101aa6d0 con 0x55af940f26b0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101abb40 con 0x55af940f2fd0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101aada0 con 0x55af94107770 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d1a8d8740 10 client.0 _ll_register_callbacks cb 0x7f5d000a6fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af9411f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d16ac4640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af94107770 0x55af9411f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af94107770 msgr2=0x55af9411f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af94107770 0x55af9411f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3657788977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 msgr2=0x55af9411ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 0x55af9411ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af94180f90 con 0x55af940f2fd0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d15ac2640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af9411f180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d000aa840 tx=0x7f5d000a7090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa682a90afe853e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:58:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d000aa490 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f5d000b3960 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.684+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c0054f0 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5d000aa910 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5d1a8d8740 10 client.27431 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55af94182280 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d000c5470 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5d000a64b0 con 0x55af940f2fd0 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5d1a8d8740 10 client.27431 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5d1a8d8740 10 client.27431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.685+0000 7f5d1a8d8740 20 client.27431 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af941827c0 0x55af94184be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d1a8d8740 4 client.27431 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:58:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5d0c0054f0 con 0x55af941827c0 2024-07-24T21:58:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af941827c0 0x55af94184be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d162c3640 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af941827c0 0x55af94184be0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43e3af3d22b6124 server_cookie=48f9a1fb560d9501 in_seq=0 out_seq=0 2024-07-24T21:58:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5cf9ffb640 10 client.27431 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/3657788977 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5d0c0054f0 con 0x55af941827c0 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5cf9ffb640 10 client.27431 handle_command_reply: tid=0 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.686+0000 7f5d1a8d8740 1 client.27431 shutdown 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 msgr2=0x55af9411f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af9411f180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d000aa840 tx=0x7f5d000a7090 comp rx=0 tx=0).stop 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 shutdown_connections 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55af941827c0 0x55af94184be0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55af94107770 0x55af9411f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af940f2fd0 0x55af9411f180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 --2- 172.21.15.60:0/3657788977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55af940f26b0 0x55af9411ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 >> 172.21.15.60:0/3657788977 conn(0x55af940f04e0 msgr2=0x55af940f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5cf9ffb640 0 client.27431 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 shutdown_connections 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 1 -- 172.21.15.60:0/3657788977 wait complete. 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.687+0000 7f5d1a8d8740 20 client.27431 trim_cache size 0 max 16384 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stdout: "ed4ea5c0-c190-405a-b90b-826c937e9bc3": { 2024-07-24T21:58:49.695 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "ed4ea5c0-c190-405a-b90b-826c937e9bc3", 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 msgr2=0x7f5cfc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 0x7f5cfc0b6280 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d08003fd0 tx=0x7f5d0801aa70 comp rx=0 tx=0).stop 2024-07-24T21:58:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 msgr2=0x7f5d10101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9c0 tx=0x7f5d0000edc0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 shutdown_connections 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5cfc0b3dc0 0x7f5cfc0b6280 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d1010cac0 0x7f5d10101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10103f80 0x7f5d10101850 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 --2- 172.21.15.60:0/1478935478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d10103660 0x7f5d10101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 >> 172.21.15.60:0/1478935478 conn(0x7f5d100fd0a0 msgr2=0x7f5d100fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 shutdown_connections 2024-07-24T21:58:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:49.688+0000 7f5d18527640 1 -- 172.21.15.60:0/1478935478 wait complete. 2024-07-24T21:58:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:49 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:49 smithi123 ceph-mon[37143]: pgmap v755: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 95 KiB/s wr, 253 op/s 2024-07-24T21:58:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:49 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:49 smithi086 ceph-mon[37128]: pgmap v755: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 95 KiB/s wr, 253 op/s 2024-07-24T21:58:50.610 INFO:tasks.cephfs.filesystem:scrub status for tag:ed4ea5c0-c190-405a-b90b-826c937e9bc3 - {'path': '/', 'tag': 'ed4ea5c0-c190-405a-b90b-826c937e9bc3', 'options': 'recursive,force'} 2024-07-24T21:58:50.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:50 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:50.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:50 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:50.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:50 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:58:51.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:58:51.551 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:51.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:51 smithi060 ceph-mon[33000]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 60 KiB/s wr, 1.57k op/s 2024-07-24T21:58:51.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:51 smithi123 ceph-mon[37143]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 60 KiB/s wr, 1.57k op/s 2024-07-24T21:58:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:51 smithi086 ceph-mon[37128]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 60 KiB/s wr, 1.57k op/s 2024-07-24T21:58:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.285+0000 7f9e0df06640 1 -- 172.21.15.60:0/3611552727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 msgr2=0x7f9e0810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3611552727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0810f320 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009ca0 tx=0x7f9e0002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 -- 172.21.15.60:0/3611552727 shutdown_connections 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3611552727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0810f320 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3611552727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081081a0 0x7f9e081085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3611552727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e08102790 0x7f9e08102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 -- 172.21.15.60:0/3611552727 >> 172.21.15.60:0/3611552727 conn(0x7f9e080fe200 msgr2=0x7f9e08100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 -- 172.21.15.60:0/3611552727 shutdown_connections 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 -- 172.21.15.60:0/3611552727 wait complete. 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 Processor -- start 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.286+0000 7f9e0df06640 1 -- start start 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e08102790 0x7f9e08198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e081081a0 0x7f9e08199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081127a0 con 0x7f9e08108b20 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08112620 con 0x7f9e08102790 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e0df06640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08112920 con 0x7f9e081081a0 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51846/0 (socket says 172.21.15.60:51846) 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 -- 172.21.15.60:0/3382514489 learned_addr learned my addr 172.21.15.60:0/3382514489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e06ffd640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e081081a0 0x7f9e08199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e077fe640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e08102790 0x7f9e08198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 -- 172.21.15.60:0/3382514489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e081081a0 msgr2=0x7f9e08199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e081081a0 0x7f9e08199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 -- 172.21.15.60:0/3382514489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e08102790 msgr2=0x7f9e08198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e08102790 0x7f9e08198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 -- 172.21.15.60:0/3382514489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0819eca0 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.287+0000 7f9e07fff640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005cc0 tx=0x7f9e0002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.288+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00003cf0 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.288+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e000044f0 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.288+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0819ef30 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.288+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002fc30 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.288+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a56d0 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.289+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08102b70 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.291+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9e000046a0 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.292+0000 7f9e04ff9640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 0x7f9df00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.292+0000 7f9e077fe640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 0x7f9df00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.292+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f9e00105750 con 0x7f9e08108b20 2024-07-24T21:58:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.292+0000 7f9e077fe640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 0x7f9df00b6200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009990 tx=0x7f9df8009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.295+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9e000ccae0 con 0x7f9e08108b20 2024-07-24T21:58:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.533+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e0802e000 con 0x7f9e08108b20 2024-07-24T21:58:53.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:53 smithi060 ceph-mon[33000]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 22 KiB/s wr, 3.29k op/s 2024-07-24T21:58:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.533+0000 7f9e04ff9640 1 -- 172.21.15.60:0/3382514489 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f9e000cc3e0 con 0x7f9e08108b20 2024-07-24T21:58:53.536 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:58:53.536 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:58:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:58:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.535+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 msgr2=0x7f9df00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.535+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 0x7f9df00b6200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009990 tx=0x7f9df8009110 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.535+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 msgr2=0x7f9e0819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.535+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005cc0 tx=0x7f9e0002f470 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 shutdown_connections 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9df00b3d40 0x7f9df00b6200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e08108b20 0x7f9e0819e4d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e081081a0 0x7f9e08199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 --2- 172.21.15.60:0/3382514489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e08102790 0x7f9e08198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 >> 172.21.15.60:0/3382514489 conn(0x7f9e080fe200 msgr2=0x7f9e081005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 shutdown_connections 2024-07-24T21:58:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:53.536+0000 7f9e0df06640 1 -- 172.21.15.60:0/3382514489 wait complete. 2024-07-24T21:58:53.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:53 smithi123 ceph-mon[37143]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 22 KiB/s wr, 3.29k op/s 2024-07-24T21:58:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- 172.21.15.60:0/488281217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714106fb0 msgr2=0x7f47141073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 --2- 172.21.15.60:0/488281217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714106fb0 0x7f47141073f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4704009900 tx=0x7f470402a410 comp rx=0 tx=0).stop 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- 172.21.15.60:0/488281217 shutdown_connections 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 --2- 172.21.15.60:0/488281217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4714107930 0x7f471410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 --2- 172.21.15.60:0/488281217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714106fb0 0x7f47141073f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 --2- 172.21.15.60:0/488281217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47141015b0 0x7f4714101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- 172.21.15.60:0/488281217 >> 172.21.15.60:0/488281217 conn(0x7f47140fd010 msgr2=0x7f47140ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- 172.21.15.60:0/488281217 shutdown_connections 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- 172.21.15.60:0/488281217 wait complete. 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 Processor -- start 2024-07-24T21:58:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.770+0000 7f471bccb640 1 -- start start 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141015b0 0x7f47141999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4714106fb0 0x7f4714199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47141115d0 con 0x7f4714107930 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4714111450 con 0x7f4714106fb0 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4714111750 con 0x7f47141015b0 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51862/0 (socket says 172.21.15.60:51862) 2024-07-24T21:58:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 -- 172.21.15.60:0/3778360055 learned_addr learned my addr 172.21.15.60:0/3778360055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f4719a67640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141015b0 0x7f47141999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f4719266640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4714106fb0 0x7f4714199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 -- 172.21.15.60:0/3778360055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141015b0 msgr2=0x7f47141999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141015b0 0x7f47141999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 -- 172.21.15.60:0/3778360055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4714106fb0 msgr2=0x7f4714199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4714106fb0 0x7f4714199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 -- 172.21.15.60:0/3778360055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47141a4f80 con 0x7f4714107930 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471a268640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f470c00cad0 tx=0x7f470c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f470affd640 1 -- 172.21.15.60:0/3778360055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c00ae20 con 0x7f4714107930 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f470affd640 1 -- 172.21.15.60:0/3778360055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470c005020 con 0x7f4714107930 2024-07-24T21:58:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47141a5150 con 0x7f4714107930 2024-07-24T21:58:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.771+0000 7f470affd640 1 -- 172.21.15.60:0/3778360055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c00f6b0 con 0x7f4714107930 2024-07-24T21:58:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.772+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47141a5610 con 0x7f4714107930 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.774+0000 7f471ea29740 1 Processor -- start 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.774+0000 7f470affd640 1 -- 172.21.15.60:0/3778360055 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f470c0051d0 con 0x7f4714107930 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 -- start start 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff6f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5562afed2280 con 0x5562aff5d1e0 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562afec1780 con 0x5562aff6d580 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f471ea29740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562afec57c0 con 0x5562aff5c8c0 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51864/0 (socket says 172.21.15.60:51864) 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 -- 172.21.15.60:0/3632725800 learned_addr learned my addr 172.21.15.60:0/3632725800 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719266640 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719266640 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff6f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 -- 172.21.15.60:0/3632725800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 msgr2=0x5562aff5cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff5cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 -- 172.21.15.60:0/3632725800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 msgr2=0x5562aff6f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff6f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f4719a67640 1 -- 172.21.15.60:0/3632725800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562afc25790 con 0x5562aff5d1e0 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.775+0000 7f470affd640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 0x7f47000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f4719266640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 0x7f47000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f4719a67640 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f471000bcb0 tx=0x7f471000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50486820c238557d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f470affd640 1 -- 172.21.15.60:0/3778360055 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f470c0e4080 con 0x7f4714107930 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f46fffff640 1 -- 172.21.15.60:0/3632725800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710015070 con 0x5562aff5d1e0 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f46fffff640 1 -- 172.21.15.60:0/3632725800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471001a910 con 0x5562aff5d1e0 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.776+0000 7f46fffff640 1 -- 172.21.15.60:0/3632725800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710011960 con 0x5562aff5d1e0 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- 172.21.15.60:0/3632725800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 msgr2=0x5562aff6d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f4719266640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 0x7f47000b6200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f471000bcb0 tx=0x7f471000f730 comp rx=0 tx=0).stop 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- 172.21.15.60:0/3632725800 shutdown_connections 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff6f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6d040 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 --2- 172.21.15.60:0/3632725800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff5cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- 172.21.15.60:0/3632725800 >> 172.21.15.60:0/3632725800 conn(0x5562aff5a6f0 msgr2=0x5562aff5b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- 172.21.15.60:0/3632725800 shutdown_connections 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- 172.21.15.60:0/3632725800 wait complete. 2024-07-24T21:58:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 -- start start 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.777+0000 7f471ea29740 1 Processor -- start 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff6ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff6d580 0x5562aff6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5562afdfbab0 con 0x5562aff5d1e0 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562afe499e0 con 0x5562aff5c8c0 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562aff70260 con 0x5562aff6d580 2024-07-24T21:58:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 10 client.0 _ll_register_callbacks cb 0x5562afbd1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51874/0 (socket says 172.21.15.60:51874) 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 -- 172.21.15.60:0/523022419 learned_addr learned my addr 172.21.15.60:0/523022419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471a268640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff6d580 0x5562aff6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719a67640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff6ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 -- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff6d580 msgr2=0x5562aff6a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff6d580 0x5562aff6a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 -- 172.21.15.60:0/523022419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 msgr2=0x5562aff6ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff6ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 -- 172.21.15.60:0/523022419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562afc9e620 con 0x5562aff5d1e0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f4719266640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f47040096c0 tx=0x7f4704009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76cd3b935d0f00e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f46fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470404f070 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f47040316a0 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.778+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562afe70ce0 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f470405b050 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 10 client.18054 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5562aff7b9e0 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5562afe70ce0 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f470403a040 con 0x5562aff5d1e0 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 10 client.18054 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 20 client.18054 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562aff7cef0 0x5562aff7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 4 client.18054 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562afed41d0 con 0x5562aff7cef0 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.779+0000 7f4719a67640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562aff7cef0 0x5562aff7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.780+0000 7f46fd7fa640 10 client.18054 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.780+0000 7f4719a67640 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562aff7cef0 0x5562aff7f380 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a54cb5ac0d1ec5 server_cookie=376f3c62720e9822 in_seq=0 out_seq=0 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.780+0000 7f46fd7fa640 1 -- 172.21.15.60:0/523022419 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562afed41d0 con 0x5562aff7cef0 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.780+0000 7f46fd7fa640 10 client.18054 handle_command_reply: tid=0 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 client.18054 shutdown 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 msgr2=0x5562aff6a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f47040096c0 tx=0x7f4704009260 comp rx=0 tx=0).stop 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 shutdown_connections 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562aff7cef0 0x5562aff7f380 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff6d580 0x5562aff6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6a1a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f46fd7fa640 0 client.18054 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 --2- 172.21.15.60:0/523022419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff6ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 >> 172.21.15.60:0/523022419 conn(0x5562aff5a6f0 msgr2=0x5562aff60290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 shutdown_connections 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 1 -- 172.21.15.60:0/523022419 wait complete. 2024-07-24T21:58:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.781+0000 7f471ea29740 20 client.18054 trim_cache size 0 max 16384 2024-07-24T21:58:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:53 smithi086 ceph-mon[37128]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 22 KiB/s wr, 3.29k op/s 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.796+0000 7f471ea29740 1 Processor -- start 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.796+0000 7f471ea29740 1 -- start start 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.796+0000 7f471ea29740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff6f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f471ea29740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f471ea29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f471ea29740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47141ae930 con 0x5562aff5d1e0 2024-07-24T21:58:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f471ea29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47141ac140 con 0x5562aff6d580 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f471ea29740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47141a9fb0 con 0x5562aff5c8c0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51876/0 (socket says 172.21.15.60:51876) 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 -- 172.21.15.60:0/2597865889 learned_addr learned my addr 172.21.15.60:0/2597865889 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 -- 172.21.15.60:0/2597865889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 msgr2=0x5562aff6f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff6f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 -- 172.21.15.60:0/2597865889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 msgr2=0x5562aff58f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 -- 172.21.15.60:0/2597865889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562aff5a520 con 0x5562aff5d1e0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f4719266640 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4704009900 tx=0x7f47040095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ea8ca647cb946 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f46ff7fe640 1 -- 172.21.15.60:0/2597865889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470404f070 con 0x5562aff5d1e0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f46ff7fe640 1 -- 172.21.15.60:0/2597865889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4704045070 con 0x5562aff5d1e0 2024-07-24T21:58:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.797+0000 7f46ff7fe640 1 -- 172.21.15.60:0/2597865889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704035040 con 0x5562aff5d1e0 2024-07-24T21:58:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- 172.21.15.60:0/2597865889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 msgr2=0x5562aff6f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4704009900 tx=0x7f47040095a0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- 172.21.15.60:0/2597865889 shutdown_connections 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff6d580 0x5562aff58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff5d1e0 0x5562aff6f7a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 --2- 172.21.15.60:0/2597865889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5c8c0 0x5562aff6f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- 172.21.15.60:0/2597865889 >> 172.21.15.60:0/2597865889 conn(0x5562aff5a6f0 msgr2=0x5562aff6a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- 172.21.15.60:0/2597865889 shutdown_connections 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- 172.21.15.60:0/2597865889 wait complete. 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 -- start start 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.798+0000 7f471ea29740 1 Processor -- start 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5d1e0 0x5562aff754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f47141aa600 con 0x5562aff6d580 2024-07-24T21:58:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47141aba70 con 0x5562aff5c8c0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47141aacd0 con 0x5562aff5d1e0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471ea29740 10 client.0 _ll_register_callbacks cb 0x7f4704009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51886/0 (socket says 172.21.15.60:51886) 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 -- 172.21.15.60:0/69769272 learned_addr learned my addr 172.21.15.60:0/69769272 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f4719266640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5d1e0 0x5562aff754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f4719a67640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 -- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5d1e0 msgr2=0x5562aff754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5d1e0 0x5562aff754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 -- 172.21.15.60:0/69769272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 msgr2=0x5562aff85560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff85560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 -- 172.21.15.60:0/69769272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562aff83de0 con 0x5562aff6d580 2024-07-24T21:58:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f471a268640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a6e0 tx=0x7f470c0a6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b6ed539a54a53c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.799+0000 7f46fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c01d040 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7f470c0922f0 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4704045070 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f470c0aac60 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 10 client.18060 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5562affe7060 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4704045070 con 0x5562aff6d580 2024-07-24T21:58:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f470c0a9930 con 0x5562aff6d580 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 10 client.18060 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 20 client.18060 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562affe7820 0x5562affe9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 4 client.18060 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.800+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562afecfc70 con 0x5562affe7820 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.801+0000 7f4719a67640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562affe7820 0x5562affe9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.801+0000 7f4719a67640 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562affe7820 0x5562affe9c40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd3db381c6cb0ba server_cookie=9ee083815033b672 in_seq=0 out_seq=0 2024-07-24T21:58:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.801+0000 7f46fd7fa640 10 client.18060 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.851+0000 7f46fd7fa640 1 -- 172.21.15.60:0/69769272 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562afecfc70 con 0x5562affe7820 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.851+0000 7f46fd7fa640 10 client.18060 handle_command_reply: tid=0 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.851+0000 7f471ea29740 1 client.18060 shutdown 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.851+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 msgr2=0x5562aff75c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.851+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a6e0 tx=0x7f470c0a6460 comp rx=0 tx=0).stop 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 shutdown_connections 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562affe7820 0x5562affe9c40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562aff6d580 0x5562aff75c80 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562aff5d1e0 0x5562aff754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 --2- 172.21.15.60:0/69769272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562aff5c8c0 0x5562aff85560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f46fd7fa640 0 client.18060 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 >> 172.21.15.60:0/69769272 conn(0x5562aff5a6f0 msgr2=0x5562aff6f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 shutdown_connections 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 1 -- 172.21.15.60:0/69769272 wait complete. 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471ea29740 20 client.18060 trim_cache size 0 max 16384 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:58:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:58:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 msgr2=0x7f47000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 0x7f47000b6200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704031400 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 msgr2=0x7f47141a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f470c00cad0 tx=0x7f470c00cf90 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 shutdown_connections 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f47000b3d40 0x7f47000b6200 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4714107930 0x7f47141a4a40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4714106fb0 0x7f4714199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 --2- 172.21.15.60:0/3778360055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141015b0 0x7f47141999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.852+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 >> 172.21.15.60:0/3778360055 conn(0x7f47140fd010 msgr2=0x7f47140ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.853+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 shutdown_connections 2024-07-24T21:58:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:58:53.853+0000 7f471bccb640 1 -- 172.21.15.60:0/3778360055 wait complete. 2024-07-24T21:58:54.348 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 1378cc53 --release --await' 2024-07-24T21:58:54.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 1378cc53 --release --await 2024-07-24T21:58:54.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:54 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3382514489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:58:54.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:54 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3382514489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:54 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3382514489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:58:55.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:55 smithi060 ceph-mon[33000]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 KiB/s wr, 3.33k op/s 2024-07-24T21:58:55.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:55 smithi123 ceph-mon[37143]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 KiB/s wr, 3.33k op/s 2024-07-24T21:58:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:55 smithi086 ceph-mon[37128]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 KiB/s wr, 3.33k op/s 2024-07-24T21:58:56.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 -- 172.21.15.60:0/2073014904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 msgr2=0x7fcd4c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 --2- 172.21.15.60:0/2073014904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c102b50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009900 tx=0x7fcd3c02a410 comp rx=0 tx=0).stop 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 -- 172.21.15.60:0/2073014904 shutdown_connections 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 --2- 172.21.15.60:0/2073014904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 0x7fcd4c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 --2- 172.21.15.60:0/2073014904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 0x7fcd4c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 --2- 172.21.15.60:0/2073014904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c102b50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 -- 172.21.15.60:0/2073014904 >> 172.21.15.60:0/2073014904 conn(0x7fcd4c0fe240 msgr2=0x7fcd4c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 -- 172.21.15.60:0/2073014904 shutdown_connections 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.266+0000 7fcd50dae640 1 -- 172.21.15.60:0/2073014904 wait complete. 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 Processor -- start 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 -- start start 2024-07-24T21:58:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 0x7fcd4c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 0x7fcd4c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c112710 con 0x7fcd4c102770 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c112590 con 0x7fcd4c108b00 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.267+0000 7fcd50dae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c112890 con 0x7fcd4c108180 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50544/0 (socket says 172.21.15.60:50544) 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 -- 172.21.15.60:0/3560299353 learned_addr learned my addr 172.21.15.60:0/3560299353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 0x7fcd4c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 0x7fcd4c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 -- 172.21.15.60:0/3560299353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 msgr2=0x7fcd4c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 0x7fcd4c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 -- 172.21.15.60:0/3560299353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 msgr2=0x7fcd4c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 0x7fcd4c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 -- 172.21.15.60:0/3560299353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c19ed50 con 0x7fcd4c102770 2024-07-24T21:58:56.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c005940 tx=0x7fcd3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd3b7fe640 1 -- 172.21.15.60:0/3560299353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c03f030 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd3b7fe640 1 -- 172.21.15.60:0/3560299353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd3c003cb0 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c19efe0 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd3b7fe640 1 -- 172.21.15.60:0/3560299353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c035070 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.268+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c1a5700 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.271+0000 7fcd3b7fe640 1 -- 172.21.15.60:0/3560299353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fcd3c042030 con 0x7fcd4c102770 2024-07-24T21:58:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.271+0000 7fcd5315f740 1 Processor -- start 2024-07-24T21:58:56.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.271+0000 7fcd5315f740 1 -- start start 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca4b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c9b1dd0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c9a12d0 con 0x55dc4ca3b350 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd5315f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c9a5310 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50556/0 (socket says 172.21.15.60:50556) 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/2140630751 learned_addr learned my addr 172.21.15.60:0/2140630751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd3b7fe640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 0x7fcd340b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/2140630751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 msgr2=0x55dc4ca4dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca4dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/2140630751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 msgr2=0x55dc4ca4b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca4b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/2140630751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4c701b00 con 0x55dc4ca3aa30 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 0x7fcd340b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcd440092b0 tx=0x7fcd44010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa38f36cf90af74f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.272+0000 7fcd3b7fe640 1 -- 172.21.15.60:0/3560299353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7fcd3c104640 con 0x7fcd4c102770 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd38ff9640 1 -- 172.21.15.60:0/2140630751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401c900 con 0x55dc4ca3aa30 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd38ff9640 1 -- 172.21.15.60:0/2140630751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd440113e0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd38ff9640 1 -- 172.21.15.60:0/2140630751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44013640 con 0x55dc4ca3aa30 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 0x7fcd340b6200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd44020000 tx=0x7fcd4401d000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd5315f740 1 -- 172.21.15.60:0/2140630751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 msgr2=0x55dc4ca3ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.273+0000 7fcd5315f740 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcd440092b0 tx=0x7fcd44010800 comp rx=0 tx=0).stop 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- 172.21.15.60:0/2140630751 shutdown_connections 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca4dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- 172.21.15.60:0/2140630751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3ae10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- 172.21.15.60:0/2140630751 >> 172.21.15.60:0/2140630751 conn(0x55dc4ca38860 msgr2=0x55dc4ca39a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- 172.21.15.60:0/2140630751 shutdown_connections 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- 172.21.15.60:0/2140630751 wait complete. 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- start start 2024-07-24T21:58:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 Processor -- start 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca48640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c804bb0 con 0x55dc4ca3b350 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c943900 con 0x55dc4ca3aa30 2024-07-24T21:58:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ca4e3a0 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd5315f740 10 client.0 _ll_register_callbacks cb 0x55dc4c94dc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd49d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd49d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50560/0 (socket says 172.21.15.60:50560) 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.274+0000 7fcd49d9b640 1 -- 172.21.15.60:0/1570833685 learned_addr learned my addr 172.21.15.60:0/1570833685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca48640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 -- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 msgr2=0x55dc4ca490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 -- 172.21.15.60:0/1570833685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3aa30 msgr2=0x55dc4ca48640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca48640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 -- 172.21.15.60:0/1570833685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4c778960 con 0x55dc4ca3b350 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000bf30 tx=0x7fcd400128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b31d8369e0d64eb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4001f060 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7fcd400108a0 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.275+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd440113e0 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd40010aa0 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 10 client.18072 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc4ca59f40 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd40010c40 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fcd40019030 con 0x55dc4ca3b350 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 10 client.18072 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 10 client.18072 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 20 client.18072 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4ca5b400 0x55dc4ca5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 4 client.18072 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:58:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd440113e0 con 0x55dc4ca5b400 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.276+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4ca5b400 0x55dc4ca5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.277+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4ca5b400 0x55dc4ca5d850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d630ddd6ebc2363 server_cookie=20cc35e7fa9db05b in_seq=0 out_seq=0 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.277+0000 7fcd31ffb640 10 client.18072 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd31ffb640 1 -- 172.21.15.60:0/1570833685 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcd440113e0 con 0x55dc4ca5b400 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd31ffb640 10 client.18072 handle_command_reply: tid=0 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 client.18072 shutdown 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 msgr2=0x55dc4ca48b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000bf30 tx=0x7fcd400128e0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 shutdown_connections 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4ca5b400 0x55dc4ca5d850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3b350 0x55dc4ca48b80 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 --2- 172.21.15.60:0/1570833685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca48640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 >> 172.21.15.60:0/1570833685 conn(0x55dc4ca38860 msgr2=0x55dc4ca3e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd31ffb640 0 client.18072 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 shutdown_connections 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 1 -- 172.21.15.60:0/1570833685 wait complete. 2024-07-24T21:58:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.278+0000 7fcd5315f740 20 client.18072 trim_cache size 0 max 16384 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 Processor -- start 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 -- start start 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca46ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1ae9b0 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1ac1c0 con 0x55dc4ca3b350 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd5315f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1aa030 con 0x55dc4ca3aa30 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50562/0 (socket says 172.21.15.60:50562) 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/864191162 learned_addr learned my addr 172.21.15.60:0/864191162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca46ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/864191162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3aa30 msgr2=0x55dc4ca3a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/864191162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 msgr2=0x55dc4ca46ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca46ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.299+0000 7fcd4ad9d640 1 -- 172.21.15.60:0/864191162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4ca46b40 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.300+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcd44011d00 tx=0x7fcd44009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842e484de499c2d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.300+0000 7fcd33fff640 1 -- 172.21.15.60:0/864191162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44034070 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.300+0000 7fcd33fff640 1 -- 172.21.15.60:0/864191162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd440098b0 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.300+0000 7fcd33fff640 1 -- 172.21.15.60:0/864191162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401b870 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- 172.21.15.60:0/864191162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 msgr2=0x55dc4ca46180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcd44011d00 tx=0x7fcd44009830 comp rx=0 tx=0).stop 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- 172.21.15.60:0/864191162 shutdown_connections 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca46180 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca3b350 0x55dc4ca46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 --2- 172.21.15.60:0/864191162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca3a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- 172.21.15.60:0/864191162 >> 172.21.15.60:0/864191162 conn(0x55dc4ca38860 msgr2=0x55dc4ca49c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- 172.21.15.60:0/864191162 shutdown_connections 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- 172.21.15.60:0/864191162 wait complete. 2024-07-24T21:58:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 -- start start 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.301+0000 7fcd5315f740 1 Processor -- start 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3b350 0x55dc4ca62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca63430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1aa680 con 0x55dc4ca3aa30 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1abaf0 con 0x55dc4ca4b6f0 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c1aad50 con 0x55dc4ca3b350 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 10 client.0 _ll_register_callbacks cb 0x7fcd4000bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50572/0 (socket says 172.21.15.60:50572) 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 -- 172.21.15.60:0/4010378959 learned_addr learned my addr 172.21.15.60:0/4010378959 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3b350 0x55dc4ca62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4ad9d640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca63430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 -- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3b350 msgr2=0x55dc4ca62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3b350 0x55dc4ca62d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 -- 172.21.15.60:0/4010378959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca4b6f0 msgr2=0x55dc4ca63430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca63430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 -- 172.21.15.60:0/4010378959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4cac5180 con 0x55dc4ca3aa30 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd4a59c640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0cb150 tx=0x7fcd3c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814587477254bfaa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c049040 con 0x55dc4ca3aa30 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8565+0+0 (secure 0 0 0) 0x7fcd3c03bbf0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.302+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc4ca5dbe0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd3c102180 con 0x55dc4ca3aa30 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 10 client.18078 fetch_fsmap learned FSMap version 44 2024-07-24T21:58:56.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc4cac6430 con 0x55dc4ca3aa30 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dc4ca5dbe0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fcd3c0ec2c0 con 0x55dc4ca3aa30 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 10 client.18078 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 10 client.18078 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 20 client.18078 populate_metadata read hostname 'smithi060' 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4cac6c60 0x55dc4cac9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 4 client.18078 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1378cc53", "release": true, "await": true}] 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1378cc53", "release": true, "await": true}) -- 0x55dc4c9b3d20 con 0x55dc4cac6c60 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.303+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4cac6c60 0x55dc4cac9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.304+0000 7fcd49d9b640 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4cac6c60 0x55dc4cac9080 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1bd264c8b6e9d1 server_cookie=134da0c14db3d1e3 in_seq=0 out_seq=0 2024-07-24T21:58:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:56.304+0000 7fcd31ffb640 10 client.18078 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd31ffb640 1 -- 172.21.15.60:0/4010378959 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dc4c9b3d20 con 0x55dc4cac6c60 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd31ffb640 10 client.18078 handle_command_reply: tid=0 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd5315f740 1 client.18078 shutdown 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 msgr2=0x55dc4ca53e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0cb150 tx=0x7fcd3c032f60 comp rx=0 tx=0).stop 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 shutdown_connections 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.401+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dc4cac6c60 0x55dc4cac9080 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd31ffb640 0 client.18078 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc4ca4b6f0 0x55dc4ca63430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc4ca3b350 0x55dc4ca62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 --2- 172.21.15.60:0/4010378959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dc4ca3aa30 0x55dc4ca53e70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 >> 172.21.15.60:0/4010378959 conn(0x55dc4ca38860 msgr2=0x55dc4ca4a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 shutdown_connections 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 1 -- 172.21.15.60:0/4010378959 wait complete. 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.402+0000 7fcd5315f740 20 client.18078 trim_cache size 0 max 16384 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 36, 2024-07-24T21:58:57.405 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "1378cc53": { 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 36, 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.5, 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:58:57.406 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:58:57.407 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 msgr2=0x7fcd340b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 0x7fcd340b6200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcd44020000 tx=0x7fcd4401d000 comp rx=0 tx=0).stop 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 msgr2=0x7fcd4c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c005940 tx=0x7fcd3c0042f0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 shutdown_connections 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcd340b3d40 0x7fcd340b6200 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd4c108b00 0x7fcd4c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd4c108180 0x7fcd4c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 --2- 172.21.15.60:0/3560299353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c102770 0x7fcd4c198ca0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:58:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 >> 172.21.15.60:0/3560299353 conn(0x7fcd4c0fe240 msgr2=0x7fcd4c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:58:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 shutdown_connections 2024-07-24T21:58:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:58:57.403+0000 7fcd50dae640 1 -- 172.21.15.60:0/3560299353 wait complete. 2024-07-24T21:58:57.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:57 smithi123 ceph-mon[37143]: pgmap v759: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 KiB/s wr, 3.27k op/s 2024-07-24T21:58:57.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:57 smithi060 ceph-mon[33000]: pgmap v759: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 KiB/s wr, 3.27k op/s 2024-07-24T21:58:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:57 smithi086 ceph-mon[37128]: pgmap v759: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 KiB/s wr, 3.27k op/s 2024-07-24T21:58:58.209 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1378cc53', total seconds elapsed: 34.4 2024-07-24T21:58:59.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:58:59 smithi123 ceph-mon[37143]: pgmap v760: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 67 KiB/s wr, 3.27k op/s 2024-07-24T21:58:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:58:59 smithi086 ceph-mon[37128]: pgmap v760: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 67 KiB/s wr, 3.27k op/s 2024-07-24T21:58:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:58:59 smithi060 ceph-mon[33000]: pgmap v760: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 67 KiB/s wr, 3.27k op/s 2024-07-24T21:59:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:01 smithi123 ceph-mon[37143]: pgmap v761: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 74 KiB/s wr, 3.15k op/s 2024-07-24T21:59:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:59:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:59:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:01 smithi086 ceph-mon[37128]: pgmap v761: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 74 KiB/s wr, 3.15k op/s 2024-07-24T21:59:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:59:01.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:59:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:01 smithi060 ceph-mon[33000]: pgmap v761: 97 pgs: 97 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 74 KiB/s wr, 3.15k op/s 2024-07-24T21:59:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:59:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:59:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T21:59:02.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:02 smithi123 ceph-mon[37143]: osdmap e102: 12 total, 12 up, 12 in 2024-07-24T21:59:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T21:59:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:02 smithi086 ceph-mon[37128]: osdmap e102: 12 total, 12 up, 12 in 2024-07-24T21:59:02.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T21:59:02.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:02 smithi060 ceph-mon[33000]: osdmap e102: 12 total, 12 up, 12 in 2024-07-24T21:59:03.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:03 smithi123 ceph-mon[37143]: pgmap v763: 97 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 83 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 716 op/s 2024-07-24T21:59:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:03 smithi086 ceph-mon[37128]: pgmap v763: 97 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 83 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 716 op/s 2024-07-24T21:59:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:03 smithi060 ceph-mon[33000]: pgmap v763: 97 pgs: 6 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 83 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 716 op/s 2024-07-24T21:59:05.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:05 smithi123 ceph-mon[37143]: pgmap v764: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 868 KiB/s wr, 301 op/s 2024-07-24T21:59:05.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:05 smithi086 ceph-mon[37128]: pgmap v764: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 868 KiB/s wr, 301 op/s 2024-07-24T21:59:05.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:05 smithi060 ceph-mon[33000]: pgmap v764: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 868 KiB/s wr, 301 op/s 2024-07-24T21:59:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.000+0000 7ff653b83640 1 -- 172.21.15.60:0/655664557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101390 msgr2=0x7ff64c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.000+0000 7ff653b83640 1 --2- 172.21.15.60:0/655664557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101390 0x7ff64c101770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff64801a910 tx=0x7ff64803b470 comp rx=0 tx=0).stop 2024-07-24T21:59:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.006+0000 7ff653b83640 1 -- 172.21.15.60:0/655664557 shutdown_connections 2024-07-24T21:59:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.006+0000 7ff653b83640 1 --2- 172.21.15.60:0/655664557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.006+0000 7ff653b83640 1 --2- 172.21.15.60:0/655664557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101cb0 0x7ff64c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.006+0000 7ff653b83640 1 --2- 172.21.15.60:0/655664557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101390 0x7ff64c101770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.006+0000 7ff653b83640 1 -- 172.21.15.60:0/655664557 >> 172.21.15.60:0/655664557 conn(0x7ff64c0fd010 msgr2=0x7ff64c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.009+0000 7ff653b83640 1 -- 172.21.15.60:0/655664557 shutdown_connections 2024-07-24T21:59:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.009+0000 7ff653b83640 1 -- 172.21.15.60:0/655664557 wait complete. 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 Processor -- start 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 -- start start 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 0x7ff64c115500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101cb0 0x7ff64c115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c111980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10a030 con 0x7ff64c1067c0 2024-07-24T21:59:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c109eb0 con 0x7ff64c101cb0 2024-07-24T21:59:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.011+0000 7ff653b83640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10a1b0 con 0x7ff64c101390 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff65191f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 0x7ff64c115500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff65191f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 0x7ff64c115500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47268/0 (socket says 172.21.15.60:47268) 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff65191f640 1 -- 172.21.15.60:0/2861818508 learned_addr learned my addr 172.21.15.60:0/2861818508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff652120640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c111980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff652120640 1 -- 172.21.15.60:0/2861818508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 msgr2=0x7ff64c115500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff652120640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 0x7ff64c115500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff652120640 1 -- 172.21.15.60:0/2861818508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101cb0 msgr2=0x7ff64c115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.012+0000 7ff652120640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101cb0 0x7ff64c115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.013+0000 7ff652120640 1 -- 172.21.15.60:0/2861818508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c112240 con 0x7ff64c1067c0 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.013+0000 7ff652120640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c111980 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff64400be00 tx=0x7ff64400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.013+0000 7ff642ffd640 1 -- 172.21.15.60:0/2861818508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644019070 con 0x7ff64c1067c0 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.013+0000 7ff642ffd640 1 -- 172.21.15.60:0/2861818508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6440148f0 con 0x7ff64c1067c0 2024-07-24T21:59:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.013+0000 7ff642ffd640 1 -- 172.21.15.60:0/2861818508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644019070 con 0x7ff64c1067c0 2024-07-24T21:59:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.015+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c112530 con 0x7ff64c1067c0 2024-07-24T21:59:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.015+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1a34a0 con 0x7ff64c1067c0 2024-07-24T21:59:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.019+0000 7ff642ffd640 1 -- 172.21.15.60:0/2861818508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff644014440 con 0x7ff64c1067c0 2024-07-24T21:59:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.021+0000 7ff642ffd640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 0x7ff6380b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.021+0000 7ff642ffd640 1 -- 172.21.15.60:0/2861818508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7ff64400cc70 con 0x7ff64c1067c0 2024-07-24T21:59:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.024+0000 7ff6569b5740 1 Processor -- start 2024-07-24T21:59:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.025+0000 7ff65191f640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 0x7ff6380b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 -- start start 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50b01e0 0x5587a50bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50be560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5587a5025280 con 0x5587a50bc170 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587a5014780 con 0x5587a50af8c0 2024-07-24T21:59:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.026+0000 7ff6569b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5587a50187c0 con 0x5587a50b01e0 2024-07-24T21:59:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39644/0 (socket says 172.21.15.60:39644) 2024-07-24T21:59:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 -- 172.21.15.60:0/758292248 learned_addr learned my addr 172.21.15.60:0/758292248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 0x7ff6380b6650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff64801a6d0 tx=0x7ff648005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 -- 172.21.15.60:0/758292248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50b01e0 msgr2=0x5587a50bbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50b01e0 0x5587a50bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 -- 172.21.15.60:0/758292248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 msgr2=0x5587a50be560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.027+0000 7ff65191f640 1 -- 172.21.15.60:0/758292248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a4d78790 con 0x5587a50af8c0 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.028+0000 7ff65191f640 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff648005880 tx=0x7ff64804f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c68e5b783e104fd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.028+0000 7ff637fff640 1 -- 172.21.15.60:0/758292248 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64805f070 con 0x5587a50af8c0 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.028+0000 7ff637fff640 1 -- 172.21.15.60:0/758292248 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff64805b900 con 0x5587a50af8c0 2024-07-24T21:59:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.028+0000 7ff637fff640 1 -- 172.21.15.60:0/758292248 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648042850 con 0x5587a50af8c0 2024-07-24T21:59:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.029+0000 7ff6569b5740 1 -- 172.21.15.60:0/758292248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 msgr2=0x5587a50afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.029+0000 7ff6569b5740 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff648005880 tx=0x7ff64804f000 comp rx=0 tx=0).stop 2024-07-24T21:59:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.038+0000 7ff6569b5740 1 -- 172.21.15.60:0/758292248 shutdown_connections 2024-07-24T21:59:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.038+0000 7ff6569b5740 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50be560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.038+0000 7ff6569b5740 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50b01e0 0x5587a50bbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.038+0000 7ff6569b5740 1 --2- 172.21.15.60:0/758292248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50af8c0 0x5587a50afca0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.038+0000 7ff6569b5740 1 -- 172.21.15.60:0/758292248 >> 172.21.15.60:0/758292248 conn(0x5587a50ad6f0 msgr2=0x5587a50ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.041+0000 7ff6569b5740 1 -- 172.21.15.60:0/758292248 shutdown_connections 2024-07-24T21:59:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.043+0000 7ff6569b5740 1 -- 172.21.15.60:0/758292248 wait complete. 2024-07-24T21:59:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.043+0000 7ff6569b5740 1 -- start start 2024-07-24T21:59:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.043+0000 7ff6569b5740 1 Processor -- start 2024-07-24T21:59:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5587a4f4eab0 con 0x5587a50bc170 2024-07-24T21:59:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587a4f9c9e0 con 0x5587a50b01e0 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff65191f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff65191f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50bb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47290/0 (socket says 172.21.15.60:47290) 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff65191f640 1 -- 172.21.15.60:0/1256828262 learned_addr learned my addr 172.21.15.60:0/1256828262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff65111e640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.044+0000 7ff6569b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5587a50bedf0 con 0x5587a50af8c0 2024-07-24T21:59:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff6569b5740 10 client.0 _ll_register_callbacks cb 0x5587a4d24a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 -- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 msgr2=0x5587a50bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 -- 172.21.15.60:0/1256828262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 msgr2=0x5587a50b9320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50b9320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 -- 172.21.15.60:0/1256828262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a4df1620 con 0x5587a50b01e0 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.045+0000 7ff65111e640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50b8de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff64c115240 tx=0x7ff63c00ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342151061e183681 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.046+0000 7ff6357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.051+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c010840 con 0x5587a50b01e0 2024-07-24T21:59:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.051+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587a4fc3ce0 con 0x5587a50b01e0 2024-07-24T21:59:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.051+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7ff63c0109e0 con 0x5587a50b01e0 2024-07-24T21:59:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.054+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff63c010be0 con 0x5587a50b01e0 2024-07-24T21:59:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.054+0000 7ff6569b5740 10 client.27473 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.054+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5587a50ca610 con 0x5587a50b01e0 2024-07-24T21:59:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.058+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5587a4fc3ce0 con 0x5587a50b01e0 2024-07-24T21:59:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.058+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff63c0173d0 con 0x5587a50b01e0 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 10 client.27473 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 10 client.27473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 20 client.27473 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a50cbf40 0x5587a50ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 4 client.27473 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.059+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a50271d0 con 0x5587a50cbf40 2024-07-24T21:59:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.061+0000 7ff65191f640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a50cbf40 0x5587a50ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.063+0000 7ff65191f640 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a50cbf40 0x5587a50ce3a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ac9dc2aca2ca7e server_cookie=3e13880ef045c44f in_seq=0 out_seq=0 2024-07-24T21:59:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.063+0000 7ff6357fa640 10 client.27473 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6357fa640 1 -- 172.21.15.60:0/1256828262 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a50271d0 con 0x5587a50cbf40 2024-07-24T21:59:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6357fa640 10 client.27473 handle_command_reply: tid=0 2024-07-24T21:59:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 client.27473 shutdown 2024-07-24T21:59:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 msgr2=0x5587a50b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50b8de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff64c115240 tx=0x7ff63c00ea60 comp rx=0 tx=0).stop 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 shutdown_connections 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a50cbf40 0x5587a50ce3a0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50b9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50b8de0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 --2- 172.21.15.60:0/1256828262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 >> 172.21.15.60:0/1256828262 conn(0x5587a50ad6f0 msgr2=0x5587a50b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.064+0000 7ff6357fa640 0 client.27473 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.069+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 shutdown_connections 2024-07-24T21:59:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.070+0000 7ff6569b5740 1 -- 172.21.15.60:0/1256828262 wait complete. 2024-07-24T21:59:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.072+0000 7ff6569b5740 20 client.27473 trim_cache size 0 max 16384 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 Processor -- start 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 -- start start 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50af8c0 0x5587a50b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aeba0 con 0x5587a50af8c0 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1ac3b0 con 0x5587a50b01e0 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.104+0000 7ff6569b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aa220 con 0x5587a50bc170 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff65111e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47306/0 (socket says 172.21.15.60:47306) 2024-07-24T21:59:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 -- 172.21.15.60:0/3356723928 learned_addr learned my addr 172.21.15.60:0/3356723928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff65191f640 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50af8c0 0x5587a50b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 -- 172.21.15.60:0/3356723928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 msgr2=0x5587a50d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50d3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 -- 172.21.15.60:0/3356723928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50af8c0 msgr2=0x5587a50b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50af8c0 0x5587a50b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 -- 172.21.15.60:0/3356723928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a50b0010 con 0x5587a50bc170 2024-07-24T21:59:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.106+0000 7ff652120640 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff6440a9dc0 tx=0x7ff6440a9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17e25d7dbd84249 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.107+0000 7ff6377fe640 1 -- 172.21.15.60:0/3356723928 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644019070 con 0x5587a50bc170 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.107+0000 7ff6377fe640 1 -- 172.21.15.60:0/3356723928 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6440a9940 con 0x5587a50bc170 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.107+0000 7ff6377fe640 1 -- 172.21.15.60:0/3356723928 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440b29a0 con 0x5587a50bc170 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 -- 172.21.15.60:0/3356723928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 msgr2=0x5587a50d41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff6440a9dc0 tx=0x7ff6440a9780 comp rx=0 tx=0).stop 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 -- 172.21.15.60:0/3356723928 shutdown_connections 2024-07-24T21:59:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50bc170 0x5587a50d41b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50d3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 --2- 172.21.15.60:0/3356723928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50af8c0 0x5587a50b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.108+0000 7ff6569b5740 1 -- 172.21.15.60:0/3356723928 >> 172.21.15.60:0/3356723928 conn(0x5587a50ad6f0 msgr2=0x5587a50bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.111+0000 7ff6569b5740 1 -- 172.21.15.60:0/3356723928 shutdown_connections 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.111+0000 7ff6569b5740 1 -- 172.21.15.60:0/3356723928 wait complete. 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.111+0000 7ff6569b5740 1 -- start start 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 Processor -- start 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aa870 con 0x5587a50bc170 2024-07-24T21:59:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1abce0 con 0x5587a50b01e0 2024-07-24T21:59:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aaf40 con 0x5587a50af8c0 2024-07-24T21:59:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.112+0000 7ff6569b5740 10 client.0 _ll_register_callbacks cb 0x7ff64c06bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54320/0 (socket says 172.21.15.60:54320) 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 -- 172.21.15.60:0/2295560489 learned_addr learned my addr 172.21.15.60:0/2295560489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff65191f640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 -- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 msgr2=0x5587a50c10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50c10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 -- 172.21.15.60:0/2295560489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 msgr2=0x5587a50c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.113+0000 7ff652120640 1 -- 172.21.15.60:0/2295560489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a5134e40 con 0x5587a50bc170 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff652120640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6440e0970 tx=0x7ff6440a62f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ddbafdc6ca802e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff6357fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644019070 con 0x5587a50bc170 2024-07-24T21:59:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6440a9940 con 0x5587a50bc170 2024-07-24T21:59:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7ff6440b2bb0 con 0x5587a50bc170 2024-07-24T21:59:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.114+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff6440ae380 con 0x5587a50bc170 2024-07-24T21:59:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 10 client.18087 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5587a5158140 con 0x5587a50bc170 2024-07-24T21:59:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6440a9940 con 0x5587a50bc170 2024-07-24T21:59:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff6440a8c10 con 0x5587a50bc170 2024-07-24T21:59:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 10 client.18087 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 10 client.18087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 20 client.18087 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a51587f0 0x5587a515ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 4 client.18087 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.115+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587a5022c70 con 0x5587a51587f0 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.116+0000 7ff65191f640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a51587f0 0x5587a515ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.116+0000 7ff65191f640 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a51587f0 0x5587a515ac10 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9860287d4146a9 server_cookie=b050cf56d1783632 in_seq=0 out_seq=0 2024-07-24T21:59:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:07.116+0000 7ff6357fa640 10 client.18087 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:07.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:07 smithi123 ceph-mon[37143]: pgmap v765: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-07-24T21:59:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:07 smithi086 ceph-mon[37128]: pgmap v765: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-07-24T21:59:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:07 smithi060 ceph-mon[33000]: pgmap v765: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.076+0000 7ff6357fa640 1 -- 172.21.15.60:0/2295560489 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587a5022c70 con 0x5587a51587f0 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6357fa640 10 client.18087 handle_command_reply: tid=0 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 client.18087 shutdown 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 msgr2=0x5587a50c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6440e0970 tx=0x7ff6440a62f0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 shutdown_connections 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5587a51587f0 0x5587a515ac10 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587a50bc170 0x5587a50c1b30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587a50b01e0 0x5587a50c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 --2- 172.21.15.60:0/2295560489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587a50af8c0 0x5587a50c10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 >> 172.21.15.60:0/2295560489 conn(0x5587a50ad6f0 msgr2=0x5587a50b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6357fa640 0 client.18087 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 shutdown_connections 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.077+0000 7ff6569b5740 1 -- 172.21.15.60:0/2295560489 wait complete. 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff6569b5740 20 client.18087 trim_cache size 0 max 16384 2024-07-24T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:59:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:59:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:59:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:59:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 msgr2=0x7ff6380b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 0x7ff6380b6650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff64801a6d0 tx=0x7ff648005b50 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 msgr2=0x7ff64c111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c111980 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff64400be00 tx=0x7ff64400bef0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 shutdown_connections 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff6380b4190 0x7ff6380b6650 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64c1067c0 0x7ff64c111980 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff64c101cb0 0x7ff64c115a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 --2- 172.21.15.60:0/2861818508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff64c101390 0x7ff64c115500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.078+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 >> 172.21.15.60:0/2861818508 conn(0x7ff64c0fd010 msgr2=0x7ff64c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.079+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 shutdown_connections 2024-07-24T21:59:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:08.079+0000 7ff653b83640 1 -- 172.21.15.60:0/2861818508 wait complete. 2024-07-24T21:59:08.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:08 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:59:08.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:08 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:59:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:08 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T21:59:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:09 smithi086 ceph-mon[37128]: pgmap v766: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-07-24T21:59:09.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:59:09.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:59:09.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:59:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:09 smithi060 ceph-mon[33000]: pgmap v766: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-07-24T21:59:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:59:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:59:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:59:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:09 smithi123 ceph-mon[37143]: pgmap v766: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-07-24T21:59:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T21:59:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T21:59:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T21:59:11.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:11 smithi086 ceph-mon[37128]: pgmap v767: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-07-24T21:59:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:11 smithi060 ceph-mon[33000]: pgmap v767: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-07-24T21:59:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:11 smithi123 ceph-mon[37143]: pgmap v767: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-07-24T21:59:13.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:13 smithi086 ceph-mon[37128]: pgmap v768: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2024-07-24T21:59:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:13 smithi060 ceph-mon[33000]: pgmap v768: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2024-07-24T21:59:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:13 smithi123 ceph-mon[37143]: pgmap v768: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 299 op/s 2024-07-24T21:59:15.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:15 smithi086 ceph-mon[37128]: pgmap v769: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-07-24T21:59:15.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:15 smithi060 ceph-mon[33000]: pgmap v769: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-07-24T21:59:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:15 smithi123 ceph-mon[37143]: pgmap v769: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-07-24T21:59:17.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:17 smithi086 ceph-mon[37128]: pgmap v770: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-07-24T21:59:17.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:17 smithi060 ceph-mon[33000]: pgmap v770: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-07-24T21:59:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:17 smithi123 ceph-mon[37143]: pgmap v770: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-07-24T21:59:19.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:19 smithi086 ceph-mon[37128]: pgmap v771: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-07-24T21:59:19.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:19 smithi060 ceph-mon[33000]: pgmap v771: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-07-24T21:59:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:19 smithi123 ceph-mon[37143]: pgmap v771: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-07-24T21:59:20.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:59:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 -- 172.21.15.60:0/3127820369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 msgr2=0x7f164c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 --2- 172.21.15.60:0/3127820369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c110ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f163c009900 tx=0x7f163c02a420 comp rx=0 tx=0).stop 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 -- 172.21.15.60:0/3127820369 shutdown_connections 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 --2- 172.21.15.60:0/3127820369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 0x7f164c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 --2- 172.21.15.60:0/3127820369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 0x7f164c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 --2- 172.21.15.60:0/3127820369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c110ba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 -- 172.21.15.60:0/3127820369 >> 172.21.15.60:0/3127820369 conn(0x7f164c06d760 msgr2=0x7f164c06db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.235+0000 7f1651d08640 1 -- 172.21.15.60:0/3127820369 shutdown_connections 2024-07-24T21:59:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.236+0000 7f1651d08640 1 -- 172.21.15.60:0/3127820369 wait complete. 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.236+0000 7f1651d08640 1 Processor -- start 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 -- start start 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 0x7f164c1a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 0x7f164c1ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1214e0 con 0x7f164c11ba90 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f164c121360 con 0x7f164c109b40 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f1651d08640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f164c121660 con 0x7f164c109f20 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f164bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 0x7f164c1ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34028/0 (socket says 172.21.15.60:34028) 2024-07-24T21:59:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f164b7fe640 1 -- 172.21.15.60:0/3428989293 learned_addr learned my addr 172.21.15.60:0/3428989293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.237+0000 7f164affd640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 0x7f164c1a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.238+0000 7f164b7fe640 1 -- 172.21.15.60:0/3428989293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 msgr2=0x7f164c1a7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.238+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 0x7f164c1a7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.238+0000 7f164b7fe640 1 -- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 msgr2=0x7f164c1ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.238+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 0x7f164c1ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.238+0000 7f164b7fe640 1 -- 172.21.15.60:0/3428989293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c1adb20 con 0x7f164c109b40 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.241+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.241+0000 7f1648ff9640 1 -- 172.21.15.60:0/3428989293 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c040070 con 0x7f164c109b40 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.241+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c1addb0 con 0x7f164c109b40 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.242+0000 7f1648ff9640 1 -- 172.21.15.60:0/3428989293 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f163c0316d0 con 0x7f164c109b40 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.242+0000 7f1648ff9640 1 -- 172.21.15.60:0/3428989293 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c035020 con 0x7f164c109b40 2024-07-24T21:59:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.242+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c1b4550 con 0x7f164c109b40 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 Processor -- start 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 -- start start 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562475828c0 0x556247582ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 0x55624758ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5562474f8280 con 0x5562475831e0 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562474e7780 con 0x55624758f170 2024-07-24T21:59:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.245+0000 7f1654a5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562474eb7c0 con 0x5562475828c0 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562475828c0 0x556247582ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34038/0 (socket says 172.21.15.60:34038) 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 -- 172.21.15.60:0/3993974662 learned_addr learned my addr 172.21.15.60:0/3993974662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f1648ff9640 1 -- 172.21.15.60:0/3428989293 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f163c043070 con 0x7f164c109b40 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 -- 172.21.15.60:0/3993974662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562475828c0 msgr2=0x556247582ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562475828c0 0x556247582ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 0x55624758ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 -- 172.21.15.60:0/3993974662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 msgr2=0x55624758ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 0x55624758ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 -- 172.21.15.60:0/3993974662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624724b790 con 0x55624758f170 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 0x55624758ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.246+0000 7f164affd640 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1640009c20 tx=0x7f16400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369a2e8f2e659f4a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.247+0000 7f1631ffb640 1 -- 172.21.15.60:0/3993974662 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640012450 con 0x55624758f170 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.247+0000 7f1648ff9640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 0x7f16340b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.247+0000 7f1648ff9640 1 -- 172.21.15.60:0/3428989293 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f163c1048f0 con 0x7f164c109b40 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.247+0000 7f164bfff640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 0x7f16340b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.247+0000 7f1631ffb640 1 -- 172.21.15.60:0/3993974662 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1640012aa0 con 0x55624758f170 2024-07-24T21:59:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 -- 172.21.15.60:0/3993974662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 msgr2=0x556247591560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1640009c20 tx=0x7f16400117b0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f164bfff640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 0x7f16340b6280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f164c06bc30 tx=0x7f164400d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 -- 172.21.15.60:0/3993974662 shutdown_connections 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55624758f170 0x556247591560 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475831e0 0x55624758ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3993974662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562475828c0 0x556247582ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 -- 172.21.15.60:0/3993974662 >> 172.21.15.60:0/3993974662 conn(0x5562475806f0 msgr2=0x556247581890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 -- 172.21.15.60:0/3993974662 shutdown_connections 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.248+0000 7f1654a5a740 1 -- 172.21.15.60:0/3993974662 wait complete. 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.249+0000 7f1654a5a740 1 -- start start 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.249+0000 7f1654a5a740 1 Processor -- start 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 0x55624758e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x556247421ab0 con 0x5562475828c0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55624746f9e0 con 0x5562475831e0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556247591ef0 con 0x55624758f170 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f1654a5a740 10 client.0 _ll_register_callbacks cb 0x5562471f7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 0x55624758e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34054/0 (socket says 172.21.15.60:34054) 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164affd640 1 -- 172.21.15.60:0/2163912740 learned_addr learned my addr 172.21.15.60:0/2163912740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164bfff640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 -- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 msgr2=0x55624758c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 -- 172.21.15.60:0/2163912740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 msgr2=0x55624758bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 -- 172.21.15.60:0/2163912740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562472c4620 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164affd640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f164b7fe640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 0x55624758e750 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f163c101f80 tx=0x7f163c0caee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7606215d413bb2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.250+0000 7f162affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c04a030 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f163c0d4060 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556247496ce0 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f163c102410 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 10 client.18093 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55624759d580 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556247496ce0 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f163c104ce0 con 0x5562475828c0 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 10 client.18093 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 10 client.18093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 20 client.18093 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55624759ec00 0x5562475a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 4 client.18093 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.251+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562474fa1d0 con 0x55624759ec00 2024-07-24T21:59:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.252+0000 7f164affd640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55624759ec00 0x5562475a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.256+0000 7f164affd640 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55624759ec00 0x5562475a10c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a573149cb6d2c1a3 server_cookie=df17854106007ec3 in_seq=0 out_seq=0 2024-07-24T21:59:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.256+0000 7f162affd640 10 client.18093 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.260+0000 7f162affd640 1 -- 172.21.15.60:0/2163912740 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562474fa1d0 con 0x55624759ec00 2024-07-24T21:59:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.260+0000 7f162affd640 10 client.18093 handle_command_reply: tid=0 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.262+0000 7f1654a5a740 1 client.18093 shutdown 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.262+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 msgr2=0x55624758e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.262+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 0x55624758e750 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f163c101f80 tx=0x7f163c0caee0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 shutdown_connections 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55624759ec00 0x5562475a10c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624758bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 --2- 172.21.15.60:0/2163912740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562475828c0 0x55624758e750 secure :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f163c101f80 tx=0x7f163c0caee0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 >> 172.21.15.60:0/2163912740 conn(0x5562475806f0 msgr2=0x556247586290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.263+0000 7f162affd640 0 client.18093 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.265+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 shutdown_connections 2024-07-24T21:59:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.265+0000 7f1654a5a740 1 -- 172.21.15.60:0/2163912740 wait complete. 2024-07-24T21:59:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.265+0000 7f1654a5a740 20 client.18093 trim_cache size 0 max 16384 2024-07-24T21:59:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.293+0000 7f1654a5a740 1 Processor -- start 2024-07-24T21:59:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.293+0000 7f1654a5a740 1 -- start start 2024-07-24T21:59:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624759b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624758f170 0x55624758bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f16380112e0 con 0x55624758f170 2024-07-24T21:59:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163800eaf0 con 0x5562475831e0 2024-07-24T21:59:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f1654a5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f163800c960 con 0x55624759b630 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48572/0 (socket says 172.21.15.60:48572) 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 -- 172.21.15.60:0/1713646053 learned_addr learned my addr 172.21.15.60:0/1713646053 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164affd640 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624758f170 0x55624758bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164b7fe640 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624759b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 -- 172.21.15.60:0/1713646053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 msgr2=0x55624759b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624759b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 -- 172.21.15.60:0/1713646053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624758f170 msgr2=0x55624758bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624758f170 0x55624758bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.294+0000 7f164bfff640 1 -- 172.21.15.60:0/1713646053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562475a1430 con 0x55624759b630 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.295+0000 7f164bfff640 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1644018910 tx=0x7f1644014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ffe9ee78b00d5b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.295+0000 7f16317fa640 1 -- 172.21.15.60:0/1713646053 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644016dc0 con 0x55624759b630 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.295+0000 7f16317fa640 1 -- 172.21.15.60:0/1713646053 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1644029070 con 0x55624759b630 2024-07-24T21:59:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.295+0000 7f16317fa640 1 -- 172.21.15.60:0/1713646053 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644017620 con 0x55624759b630 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 -- 172.21.15.60:0/1713646053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 msgr2=0x55624758c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1644018910 tx=0x7f1644014000 comp rx=0 tx=0).stop 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 -- 172.21.15.60:0/1713646053 shutdown_connections 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624759b630 0x55624758c140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624758f170 0x55624758bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 --2- 172.21.15.60:0/1713646053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x55624759b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 -- 172.21.15.60:0/1713646053 >> 172.21.15.60:0/1713646053 conn(0x5562475806f0 msgr2=0x55624758e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 -- 172.21.15.60:0/1713646053 shutdown_connections 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.296+0000 7f1654a5a740 1 -- 172.21.15.60:0/1713646053 wait complete. 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.297+0000 7f1654a5a740 1 -- start start 2024-07-24T21:59:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.297+0000 7f1654a5a740 1 Processor -- start 2024-07-24T21:59:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.298+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x556247590a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.298+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.298+0000 7f1654a5a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 0x55624758fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f164affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f164affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48586/0 (socket says 172.21.15.60:48586) 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f164affd640 1 -- 172.21.15.60:0/3064110156 learned_addr learned my addr 172.21.15.60:0/3064110156 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f163800cfb0 con 0x55624759b630 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163800e420 con 0x5562475831e0 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f163800d680 con 0x55624758f170 2024-07-24T21:59:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.299+0000 7f1654a5a740 10 client.0 _ll_register_callbacks cb 0x7f163c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.301+0000 7f164bfff640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 0x55624758fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.301+0000 7f164bfff640 1 -- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 msgr2=0x55624758f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.301+0000 7f164bfff640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.301+0000 7f164bfff640 1 -- 172.21.15.60:0/3064110156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 msgr2=0x556247590a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.301+0000 7f164bfff640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x556247590a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.302+0000 7f164bfff640 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562475900a0 con 0x55624759b630 2024-07-24T21:59:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.302+0000 7f164bfff640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 0x55624758fa90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1644024da0 tx=0x7f1644009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975f0d2765376abb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.302+0000 7f162affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.302+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644032070 con 0x55624759b630 2024-07-24T21:59:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.303+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f1644037460 con 0x55624759b630 2024-07-24T21:59:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.304+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1644029070 con 0x55624759b630 2024-07-24T21:59:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.305+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1644037660 con 0x55624759b630 2024-07-24T21:59:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.309+0000 7f1654a5a740 10 client.18099 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.309+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5562475970a0 con 0x55624759b630 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1644029070 con 0x55624759b630 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f164402e400 con 0x55624759b630 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 10 client.18099 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 10 client.18099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 20 client.18099 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562475977b0 0x55624760b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 4 client.18099 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:59:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.310+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562474f5c70 con 0x5562475977b0 2024-07-24T21:59:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.311+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562475977b0 0x55624760b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.315+0000 7f164b7fe640 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562475977b0 0x55624760b9b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5526bf0670f4e952 server_cookie=6729b9a82c1526d in_seq=0 out_seq=0 2024-07-24T21:59:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.315+0000 7f162affd640 10 client.18099 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:21 smithi086 ceph-mon[37128]: pgmap v772: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 960 KiB/s wr, 232 op/s 2024-07-24T21:59:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.817+0000 7f162affd640 1 -- 172.21.15.60:0/3064110156 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562474f5c70 con 0x5562475977b0 2024-07-24T21:59:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.817+0000 7f162affd640 10 client.18099 handle_command_reply: tid=0 2024-07-24T21:59:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.817+0000 7f1654a5a740 1 client.18099 shutdown 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.817+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 msgr2=0x55624758fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.817+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 0x55624758fa90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1644024da0 tx=0x7f1644009e30 comp rx=0 tx=0).stop 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 shutdown_connections 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5562475977b0 0x55624760b9b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55624759b630 0x55624758fa90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f162affd640 0 client.18099 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55624758f170 0x55624758f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 --2- 172.21.15.60:0/3064110156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562475831e0 0x556247590a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 >> 172.21.15.60:0/3064110156 conn(0x5562475806f0 msgr2=0x55624758bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 shutdown_connections 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 1 -- 172.21.15.60:0/3064110156 wait complete. 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.818+0000 7f1654a5a740 20 client.18099 trim_cache size 0 max 16384 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:59:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:59:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 msgr2=0x7f16340b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 0x7f16340b6280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f164c06bc30 tx=0x7f164400d040 comp rx=0 tx=0).stop 2024-07-24T21:59:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 msgr2=0x7f164c1a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c0045e0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 shutdown_connections 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f16340b3dc0 0x7f16340b6280 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f164c11ba90 0x7f164c1ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f164c109f20 0x7f164c1a7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 --2- 172.21.15.60:0/3428989293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164c109b40 0x7f164c1a7a80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 >> 172.21.15.60:0/3428989293 conn(0x7f164c06d760 msgr2=0x7f164c11bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.819+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 shutdown_connections 2024-07-24T21:59:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:21.820+0000 7f1651d08640 1 -- 172.21.15.60:0/3428989293 wait complete. 2024-07-24T21:59:21.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:21 smithi060 ceph-mon[33000]: pgmap v772: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 960 KiB/s wr, 232 op/s 2024-07-24T21:59:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:21 smithi123 ceph-mon[37143]: pgmap v772: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 960 KiB/s wr, 232 op/s 2024-07-24T21:59:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.132+0000 7f53fd2d8640 1 -- 172.21.15.60:0/2423372988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109f20 msgr2=0x7f53f810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.132+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/2423372988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109f20 0x7f53f810a360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009900 tx=0x7f53ec02a4d0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.135+0000 7f53fd2d8640 1 -- 172.21.15.60:0/2423372988 shutdown_connections 2024-07-24T21:59:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.135+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/2423372988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.135+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/2423372988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109f20 0x7f53f810a360 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009900 tx=0x7f53ec02a4d0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.135+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/2423372988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f8109b40 0x7f53f8110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.135+0000 7f53fd2d8640 1 -- 172.21.15.60:0/2423372988 >> 172.21.15.60:0/2423372988 conn(0x7f53f806d980 msgr2=0x7f53f806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.137+0000 7f53fd2d8640 1 -- 172.21.15.60:0/2423372988 shutdown_connections 2024-07-24T21:59:23.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.137+0000 7f53fd2d8640 1 -- 172.21.15.60:0/2423372988 wait complete. 2024-07-24T21:59:23.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.138+0000 7f53fd2d8640 1 Processor -- start 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 -- start start 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109b40 0x7f53f811acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f8114f50 0x7f53f8115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8111740 con 0x7f53f811ba90 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81115c0 con 0x7f53f8109b40 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.139+0000 7f53fd2d8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81118c0 con 0x7f53f8114f50 2024-07-24T21:59:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38646/0 (socket says 172.21.15.60:38646) 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 -- 172.21.15.60:0/1239726394 learned_addr learned my addr 172.21.15.60:0/1239726394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 -- 172.21.15.60:0/1239726394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f8114f50 msgr2=0x7f53f8115400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f8114f50 0x7f53f8115400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 -- 172.21.15.60:0/1239726394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109b40 msgr2=0x7f53f811acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109b40 0x7f53f811acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.140+0000 7f53f67fc640 1 -- 172.21.15.60:0/1239726394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8115c20 con 0x7f53f811ba90 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.141+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f53ec032000 tx=0x7f53ec005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.142+0000 7f53dffff640 1 -- 172.21.15.60:0/1239726394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec040030 con 0x7f53f811ba90 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.142+0000 7f53dffff640 1 -- 172.21.15.60:0/1239726394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ec03cb00 con 0x7f53f811ba90 2024-07-24T21:59:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.142+0000 7f53dffff640 1 -- 172.21.15.60:0/1239726394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec031a20 con 0x7f53f811ba90 2024-07-24T21:59:23.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.148+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81bcdd0 con 0x7f53f811ba90 2024-07-24T21:59:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.148+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81bd170 con 0x7f53f811ba90 2024-07-24T21:59:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.154+0000 7f53ff689740 1 Processor -- start 2024-07-24T21:59:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 -- start start 2024-07-24T21:59:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 0x55f59c7ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c7fffd0 0x55f59c80ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 0x55f59c80e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f59c776990 con 0x55f59c80bf60 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f59c765e90 con 0x55f59c7fffd0 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.155+0000 7f53ff689740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f59c769ed0 con 0x55f59c7ff6b0 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.157+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 0x55f59c7ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.157+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 0x55f59c7ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48618/0 (socket says 172.21.15.60:48618) 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.157+0000 7f53f77fe640 1 -- 172.21.15.60:0/1948478530 learned_addr learned my addr 172.21.15.60:0/1948478530 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.157+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c7fffd0 0x55f59c80ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.157+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 0x55f59c80e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.158+0000 7f53dffff640 1 -- 172.21.15.60:0/1239726394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f53ec043030 con 0x7f53f811ba90 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.159+0000 7f53f67fc640 1 -- 172.21.15.60:0/1948478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 msgr2=0x55f59c7ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.159+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 0x55f59c7ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.159+0000 7f53f67fc640 1 -- 172.21.15.60:0/1948478530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c7fffd0 msgr2=0x55f59c80ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.159+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c7fffd0 0x55f59c80ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.159+0000 7f53f67fc640 1 -- 172.21.15.60:0/1948478530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59c6ef300 con 0x55f59c80bf60 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.160+0000 7f53f67fc640 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 0x55f59c80e350 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f53ec101f70 tx=0x7f53ec10abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328c542f6ba55b61 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.160+0000 7f53dffff640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 0x7f53e00b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.160+0000 7f53dd7fa640 1 -- 172.21.15.60:0/1948478530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0caba0 con 0x55f59c80bf60 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.160+0000 7f53dffff640 1 -- 172.21.15.60:0/1239726394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f53ec1048e0 con 0x7f53f811ba90 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.160+0000 7f53dd7fa640 1 -- 172.21.15.60:0/1948478530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ec114020 con 0x55f59c80bf60 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.161+0000 7f53dd7fa640 1 -- 172.21.15.60:0/1948478530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0cad80 con 0x55f59c80bf60 2024-07-24T21:59:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.163+0000 7f53ff689740 1 -- 172.21.15.60:0/1948478530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 msgr2=0x55f59c80e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.163+0000 7f53f77fe640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 0x7f53e00b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.165+0000 7f53ff689740 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 0x55f59c80e350 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f53ec101f70 tx=0x7f53ec10abf0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53ff689740 1 -- 172.21.15.60:0/1948478530 shutdown_connections 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53ff689740 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c80bf60 0x55f59c80e350 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53ff689740 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c7fffd0 0x55f59c80ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53ff689740 1 --2- 172.21.15.60:0/1948478530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7ff6b0 0x55f59c7ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53ff689740 1 -- 172.21.15.60:0/1948478530 >> 172.21.15.60:0/1948478530 conn(0x55f59c7fd4e0 msgr2=0x55f59c7fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.166+0000 7f53f77fe640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 0x7f53e00b6400 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009270 tx=0x7f53f00094c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.167+0000 7f53ff689740 1 -- 172.21.15.60:0/1948478530 shutdown_connections 2024-07-24T21:59:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.171+0000 7f53ff689740 1 -- 172.21.15.60:0/1948478530 wait complete. 2024-07-24T21:59:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.171+0000 7f53ff689740 1 -- start start 2024-07-24T21:59:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.171+0000 7f53ff689740 1 Processor -- start 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.172+0000 7f53ff689740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c808ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.172+0000 7f53ff689740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c809430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.172+0000 7f53ff689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c809970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.172+0000 7f53ff689740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f59c5c9700 con 0x55f59c7ff6b0 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53ff689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f59c7085d0 con 0x55f59c80bf60 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c809970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c809970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34090/0 (socket says 172.21.15.60:34090) 2024-07-24T21:59:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53f77fe640 1 -- 172.21.15.60:0/4068840478 learned_addr learned my addr 172.21.15.60:0/4068840478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:23.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f59c80e9a0 con 0x55f59c7fffd0 2024-07-24T21:59:23.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c808ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.173+0000 7f53ff689740 10 client.0 _ll_register_callbacks cb 0x55f59c728940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:23.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 -- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 msgr2=0x55f59c809430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:59:23.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c809430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 -- 172.21.15.60:0/4068840478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 msgr2=0x55f59c809970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c809970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 -- 172.21.15.60:0/4068840478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59c6fb660 con 0x55f59c7ff6b0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c808ef0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f53e800bc80 tx=0x7f53e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430bb8a660e19d5a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.174+0000 7f53daff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8015070 con 0x55f59c7ff6b0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f53e8004220 con 0x55f59c7ff6b0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53ec114020 con 0x55f59c7ff6b0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f53e8004420 con 0x55f59c7ff6b0 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53ff689740 10 client.18111 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:23.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.175+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f59c4c6650 con 0x55f59c7ff6b0 2024-07-24T21:59:23.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.180+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ec114020 con 0x55f59c7ff6b0 2024-07-24T21:59:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f53e80116e0 con 0x55f59c7ff6b0 2024-07-24T21:59:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 10 client.18111 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 20 client.18111 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c81bbf0 0x55f59c81e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 4 client.18111 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.181+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f59c7788e0 con 0x55f59c81bbf0 2024-07-24T21:59:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.182+0000 7f53f67fc640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c81bbf0 0x55f59c81e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.183+0000 7f53f67fc640 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c81bbf0 0x55f59c81e0b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66613b20d95e6a27 server_cookie=585e3fa3c5d0b2a6 in_seq=0 out_seq=0 2024-07-24T21:59:23.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.183+0000 7f53daff5640 10 client.18111 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:23.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.188+0000 7f53daff5640 1 -- 172.21.15.60:0/4068840478 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f59c7788e0 con 0x55f59c81bbf0 2024-07-24T21:59:23.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.189+0000 7f53daff5640 10 client.18111 handle_command_reply: tid=0 2024-07-24T21:59:23.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.189+0000 7f53ff689740 1 client.18111 shutdown 2024-07-24T21:59:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.190+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 msgr2=0x55f59c808ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.190+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c808ef0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f53e800bc80 tx=0x7f53e800f730 comp rx=0 tx=0).stop 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 shutdown_connections 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c81bbf0 0x55f59c81e0b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c809970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c809430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 --2- 172.21.15.60:0/4068840478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c808ef0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 >> 172.21.15.60:0/4068840478 conn(0x55f59c7fd4e0 msgr2=0x55f59c803080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.191+0000 7f53daff5640 0 client.18111 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.192+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 shutdown_connections 2024-07-24T21:59:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.192+0000 7f53ff689740 1 -- 172.21.15.60:0/4068840478 wait complete. 2024-07-24T21:59:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.192+0000 7f53ff689740 20 client.18111 trim_cache size 0 max 16384 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.226+0000 7f53ff689740 1 Processor -- start 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.226+0000 7f53ff689740 1 -- start start 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.226+0000 7f53ff689740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53ff689740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c80ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53ff689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c823980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53ff689740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80801b0 con 0x55f59c7ff6b0 2024-07-24T21:59:23.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38692/0 (socket says 172.21.15.60:38692) 2024-07-24T21:59:23.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53f6ffd640 1 -- 172.21.15.60:0/2828153758 learned_addr learned my addr 172.21.15.60:0/2828153758 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:23.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.227+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53f807da00 con 0x55f59c80bf60 2024-07-24T21:59:23.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8116250 con 0x55f59c7fffd0 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 -- 172.21.15.60:0/2828153758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 msgr2=0x55f59c80ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c80ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 -- 172.21.15.60:0/2828153758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 msgr2=0x55f59c823980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c823980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 -- 172.21.15.60:0/2828153758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59c807e50 con 0x55f59c7ff6b0 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.228+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b950 tx=0x7f53e8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f057ac9456afc38 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.229+0000 7f53dcff9640 1 -- 172.21.15.60:0/2828153758 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8015070 con 0x55f59c7ff6b0 2024-07-24T21:59:23.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.229+0000 7f53dcff9640 1 -- 172.21.15.60:0/2828153758 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8021490 con 0x55f59c7ff6b0 2024-07-24T21:59:23.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.229+0000 7f53dcff9640 1 -- 172.21.15.60:0/2828153758 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8019670 con 0x55f59c7ff6b0 2024-07-24T21:59:23.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 msgr2=0x55f59c81dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b950 tx=0x7f53e8005760 comp rx=0 tx=0).stop 2024-07-24T21:59:23.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 shutdown_connections 2024-07-24T21:59:23.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c823980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c7fffd0 0x55f59c80ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 --2- 172.21.15.60:0/2828153758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c7ff6b0 0x55f59c81dc00 secure :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b950 tx=0x7f53e8005760 comp rx=0 tx=0).stop 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.230+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 >> 172.21.15.60:0/2828153758 conn(0x55f59c7fd4e0 msgr2=0x55f59c80a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.231+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 shutdown_connections 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.231+0000 7f53ff689740 1 -- 172.21.15.60:0/2828153758 wait complete. 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 -- start start 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 Processor -- start 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c80fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c8145e0 0x55f59c80ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53f807bec0 con 0x55f59c810640 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53f807d330 con 0x55f59c80bf60 2024-07-24T21:59:23.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f807c590 con 0x55f59c8145e0 2024-07-24T21:59:23.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.232+0000 7f53ff689740 10 client.0 _ll_register_callbacks cb 0x7f53e800b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:23.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38700/0 (socket says 172.21.15.60:38700) 2024-07-24T21:59:23.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 -- 172.21.15.60:0/1043336580 learned_addr learned my addr 172.21.15.60:0/1043336580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 -- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c8145e0 msgr2=0x55f59c80ff50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c8145e0 0x55f59c80ff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 -- 172.21.15.60:0/1043336580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 msgr2=0x55f59c80fa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c80fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.233+0000 7f53f77fe640 1 -- 172.21.15.60:0/1043336580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59c8861c0 con 0x55f59c810640 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.234+0000 7f53f77fe640 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f53f00042c0 tx=0x7f53f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f0cce1f874a4a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:23.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.234+0000 7f53daff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.234+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f003f820 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.235+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8021490 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.235+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f53f003f9c0 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.235+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f53f003fbc0 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.235+0000 7f53ff689740 10 client.18117 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.235+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f59c887560 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8021490 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f53f0047020 con 0x55f59c810640 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 10 client.18117 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 20 client.18117 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c887e10 0x55f59c88a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:23.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 4 client.18117 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:59:23.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.236+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f59c774380 con 0x55f59c887e10 2024-07-24T21:59:23.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.237+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c887e10 0x55f59c88a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.246+0000 7f53f6ffd640 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c887e10 0x55f59c88a290 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a23410198773a3b server_cookie=fd16f97eea53f174 in_seq=0 out_seq=0 2024-07-24T21:59:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.246+0000 7f53daff5640 10 client.18117 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.251+0000 7f53daff5640 1 -- 172.21.15.60:0/1043336580 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f59c774380 con 0x55f59c887e10 2024-07-24T21:59:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.251+0000 7f53daff5640 10 client.18117 handle_command_reply: tid=0 2024-07-24T21:59:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.251+0000 7f53ff689740 1 client.18117 shutdown 2024-07-24T21:59:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.251+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 msgr2=0x55f59c885990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.251+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f53f00042c0 tx=0x7f53f0004ab0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 shutdown_connections 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f59c887e10 0x55f59c88a290 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f59c810640 0x55f59c885990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f59c8145e0 0x55f59c80ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 --2- 172.21.15.60:0/1043336580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f59c80bf60 0x55f59c80fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 >> 172.21.15.60:0/1043336580 conn(0x55f59c7fd4e0 msgr2=0x55f59c7fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.252+0000 7f53daff5640 0 client.18117 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.253+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 shutdown_connections 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.253+0000 7f53ff689740 1 -- 172.21.15.60:0/1043336580 wait complete. 2024-07-24T21:59:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.253+0000 7f53ff689740 20 client.18117 trim_cache size 0 max 16384 2024-07-24T21:59:23.256 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:59:23.256 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T21:59:23.256 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T21:59:23.256 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.255+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 msgr2=0x7f53e00b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.255+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 0x7f53e00b6400 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009270 tx=0x7f53f00094c0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.255+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 msgr2=0x7f53f811b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.255+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f53ec032000 tx=0x7f53ec005480 comp rx=0 tx=0).stop 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 shutdown_connections 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53e00b3f40 0x7f53e00b6400 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f8114f50 0x7f53f8115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f811ba90 0x7f53f811b210 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 --2- 172.21.15.60:0/1239726394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8109b40 0x7f53f811acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.256+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 >> 172.21.15.60:0/1239726394 conn(0x7f53f806d980 msgr2=0x7f53f80711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.258+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 shutdown_connections 2024-07-24T21:59:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:23.259+0000 7f53fd2d8640 1 -- 172.21.15.60:0/1239726394 wait complete. 2024-07-24T21:59:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:23 smithi086 ceph-mon[37128]: pgmap v773: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-07-24T21:59:23.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:23 smithi060 ceph-mon[33000]: pgmap v773: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-07-24T21:59:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:23 smithi123 ceph-mon[37143]: pgmap v773: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-07-24T21:59:24.026 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T21:59:24.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:59:24.489 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:25.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:25 smithi060 ceph-mon[33000]: pgmap v774: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 949 KiB/s wr, 167 op/s 2024-07-24T21:59:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:25 smithi123 ceph-mon[37143]: pgmap v774: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 949 KiB/s wr, 167 op/s 2024-07-24T21:59:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:25 smithi086 ceph-mon[37128]: pgmap v774: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 949 KiB/s wr, 167 op/s 2024-07-24T21:59:26.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2839909130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241014c0 msgr2=0x7fe7241018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:26.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2839909130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241014c0 0x7fe7241018a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe720009900 tx=0x7fe72002a3f0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2839909130 shutdown_connections 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2839909130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724107850 0x7fe72410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2839909130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724106ed0 0x7fe724107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2839909130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7241014c0 0x7fe7241018a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2839909130 >> 172.21.15.60:0/2839909130 conn(0x7fe7240fcf90 msgr2=0x7fe7240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2839909130 shutdown_connections 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.469+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2839909130 wait complete. 2024-07-24T21:59:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 Processor -- start 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 -- start start 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724106ed0 0x7fe724111f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 0x7fe724117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe72ae4e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe724110360 con 0x7fe7241014c0 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.470+0000 7fe729e4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe729e4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51620/0 (socket says 172.21.15.60:51620) 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe729e4c640 1 -- 172.21.15.60:0/2337710649 learned_addr learned my addr 172.21.15.60:0/2337710649 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241101e0 con 0x7fe724107850 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241104e0 con 0x7fe724106ed0 2024-07-24T21:59:26.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 0x7fe724117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72964b640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724106ed0 0x7fe724111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 -- 172.21.15.60:0/2337710649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724106ed0 msgr2=0x7fe724111f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:26.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724106ed0 0x7fe724111f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 -- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 msgr2=0x7fe724111a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.471+0000 7fe72a64d640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe724117ab0 con 0x7fe724107850 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe729e4c640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe72a64d640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 0x7fe724117330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009fd0 tx=0x7fe71c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00ce20 con 0x7fe724107850 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe724117da0 con 0x7fe724107850 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72411a4b0 con 0x7fe724107850 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71c005020 con 0x7fe724107850 2024-07-24T21:59:26.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.472+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c018670 con 0x7fe724107850 2024-07-24T21:59:26.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.474+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7080052c0 con 0x7fe724107850 2024-07-24T21:59:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.476+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe71c004a40 con 0x7fe724107850 2024-07-24T21:59:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.477+0000 7fe71affd640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 0x7fe7100b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.478+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fe71c0a81a0 con 0x7fe724107850 2024-07-24T21:59:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.478+0000 7fe729e4c640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 0x7fe7100b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.480+0000 7fe729e4c640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 0x7fe7100b62a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe720031440 tx=0x7fe7200058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.485+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe71c0ac2b0 con 0x7fe724107850 2024-07-24T21:59:26.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.895+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7080055b0 con 0x7fe724107850 2024-07-24T21:59:26.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.897+0000 7fe71affd640 1 -- 172.21.15.60:0/2337710649 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fe71c0abbb0 con 0x7fe724107850 2024-07-24T21:59:26.900 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:59:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:59:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:59:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 msgr2=0x7fe7100b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 0x7fe7100b62a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe720031440 tx=0x7fe7200058d0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 msgr2=0x7fe724117330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 0x7fe724117330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009fd0 tx=0x7fe71c00ef10 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 shutdown_connections 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe7100b3de0 0x7fe7100b62a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe724107850 0x7fe724117330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.901+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724106ed0 0x7fe724111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.902+0000 7fe72ae4e640 1 --2- 172.21.15.60:0/2337710649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7241014c0 0x7fe724111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.902+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 >> 172.21.15.60:0/2337710649 conn(0x7fe7240fcf90 msgr2=0x7fe7240fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.902+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 shutdown_connections 2024-07-24T21:59:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:26.902+0000 7fe72ae4e640 1 -- 172.21.15.60:0/2337710649 wait complete. 2024-07-24T21:59:27.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T21:59:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:27 smithi060 ceph-mon[33000]: pgmap v775: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-07-24T21:59:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2337710649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:27 smithi123 ceph-mon[37143]: pgmap v775: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-07-24T21:59:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2337710649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:27 smithi086 ceph-mon[37128]: pgmap v775: 97 pgs: 15 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 49 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-07-24T21:59:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2337710649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:29.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:29 smithi060 ceph-mon[33000]: pgmap v776: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-07-24T21:59:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:29 smithi123 ceph-mon[37143]: pgmap v776: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-07-24T21:59:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:29 smithi086 ceph-mon[37128]: pgmap v776: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-07-24T21:59:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.427+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2668675139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381027b0 msgr2=0x7fd038102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.427+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2668675139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381027b0 0x7fd038102b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd028009860 tx=0x7fd02802a340 comp rx=0 tx=0).stop 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2668675139 shutdown_connections 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2668675139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 0x7fd03810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2668675139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381081c0 0x7fd038108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2668675139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381027b0 0x7fd038102b90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2668675139 >> 172.21.15.60:0/2668675139 conn(0x7fd0380fe200 msgr2=0x7fd038100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2668675139 shutdown_connections 2024-07-24T21:59:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.428+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2668675139 wait complete. 2024-07-24T21:59:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.429+0000 7fd03ccc7640 1 Processor -- start 2024-07-24T21:59:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.429+0000 7fd03ccc7640 1 -- start start 2024-07-24T21:59:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381081c0 0x7fd038074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 0x7fd0380754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51648/0 (socket says 172.21.15.60:51648) 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03659c640 1 -- 172.21.15.60:0/2085537597 learned_addr learned my addr 172.21.15.60:0/2085537597 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd038112760 con 0x7fd0381027b0 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0381125e0 con 0x7fd038108b40 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0381128e0 con 0x7fd0381081c0 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd035d9b640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381081c0 0x7fd038074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.430+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 0x7fd0380754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 -- 172.21.15.60:0/2085537597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381081c0 msgr2=0x7fd038074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381081c0 0x7fd038074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 -- 172.21.15.60:0/2085537597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 msgr2=0x7fd0380754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 0x7fd0380754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0381ab430 con 0x7fd0381027b0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd03659c640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd02802a930 tx=0x7fd028005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.431+0000 7fd0277fe640 1 -- 172.21.15.60:0/2085537597 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028040020 con 0x7fd0381027b0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.432+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0381ab600 con 0x7fd0381027b0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.432+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0381abab0 con 0x7fd0381027b0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.432+0000 7fd0277fe640 1 -- 172.21.15.60:0/2085537597 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0280328f0 con 0x7fd0381027b0 2024-07-24T21:59:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.432+0000 7fd0277fe640 1 -- 172.21.15.60:0/2085537597 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028037040 con 0x7fd0381027b0 2024-07-24T21:59:30.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.435+0000 7fd03f078740 1 Processor -- start 2024-07-24T21:59:30.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.435+0000 7fd03f078740 1 -- start start 2024-07-24T21:59:30.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 0x55d8148ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d8148139b0 con 0x55d81489c6d0 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d814802eb0 con 0x55d8148a8f80 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03f078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d814806ef0 con 0x55d81489cff0 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51650/0 (socket says 172.21.15.60:51650) 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 -- 172.21.15.60:0/1482770876 learned_addr learned my addr 172.21.15.60:0/1482770876 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd0277fe640 1 -- 172.21.15.60:0/2085537597 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd028043020 con 0x7fd0381027b0 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd03659c640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 -- 172.21.15.60:0/1482770876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 msgr2=0x55d8148a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd036d9d640 1 -- 172.21.15.60:0/1482770876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 msgr2=0x55d8148ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.436+0000 7fd035d9b640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 0x55d8148ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd036d9d640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 0x55d8148ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd036d9d640 1 -- 172.21.15.60:0/1482770876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81478c320 con 0x55d81489c6d0 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd03659c640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd035d9b640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 0x55d8148ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd036d9d640 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd030009f50 tx=0x7fd0300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac126cb32afc6d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd024ff9640 1 -- 172.21.15.60:0/1482770876 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030004d20 con 0x55d81489c6d0 2024-07-24T21:59:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd024ff9640 1 -- 172.21.15.60:0/1482770876 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd030004050 con 0x55d81489c6d0 2024-07-24T21:59:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.437+0000 7fd024ff9640 1 -- 172.21.15.60:0/1482770876 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030005280 con 0x55d81489c6d0 2024-07-24T21:59:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd0277fe640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 0x7fd0200b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03659c640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 0x7fd0200b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd0277fe640 1 -- 172.21.15.60:0/2085537597 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fd028103670 con 0x7fd0381027b0 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03f078740 1 -- 172.21.15.60:0/1482770876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 msgr2=0x55d81489cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03f078740 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd030009f50 tx=0x7fd0300118d0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03659c640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 0x7fd0200b6200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd028100f20 tx=0x7fd02800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03f078740 1 -- 172.21.15.60:0/1482770876 shutdown_connections 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.438+0000 7fd03f078740 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8148a8f80 0x55d8148ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 --2- 172.21.15.60:0/1482770876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81489c6d0 0x55d81489cab0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 -- 172.21.15.60:0/1482770876 >> 172.21.15.60:0/1482770876 conn(0x55d81489a500 msgr2=0x55d81489b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 -- 172.21.15.60:0/1482770876 shutdown_connections 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 -- 172.21.15.60:0/1482770876 wait complete. 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.439+0000 7fd03f078740 1 -- start start 2024-07-24T21:59:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 Processor -- start 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d814666700 con 0x55d8148a8f80 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8147a55f0 con 0x55d81489c6d0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8148abc00 con 0x55d81489cff0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd03f078740 10 client.0 _ll_register_callbacks cb 0x55d8147c5960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55694/0 (socket says 172.21.15.60:55694) 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd036d9d640 1 -- 172.21.15.60:0/2603081767 learned_addr learned my addr 172.21.15.60:0/2603081767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.440+0000 7fd035d9b640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 -- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 msgr2=0x55d8148a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 -- 172.21.15.60:0/2603081767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 msgr2=0x55d8148a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 -- 172.21.15.60:0/2603081767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d814798680 con 0x55d81489c6d0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd030009d60 tx=0x7fd0300047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fe9a31ec83ea9c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03002c020 con 0x55d81489c6d0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd030004050 con 0x55d81489c6d0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fd03000f690 con 0x55d81489c6d0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd030037050 con 0x55d81489c6d0 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd03f078740 10 client.27515 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.441+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d814563650 con 0x55d81489c6d0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd030004050 con 0x55d81489c6d0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd0300289c0 con 0x55d81489c6d0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 10 client.27515 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 10 client.27515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 20 client.27515 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8148bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 4 client.27515 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.442+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d814815900 con 0x55d8148b88a0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.443+0000 7fd035d9b640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8148bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.443+0000 7fd01dffb640 10 client.27515 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.443+0000 7fd035d9b640 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8148bac90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28597fbdf63dd39c server_cookie=3ca28e2435ba858c in_seq=0 out_seq=0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd01dffb640 1 -- 172.21.15.60:0/2603081767 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d814815900 con 0x55d8148b88a0 2024-07-24T21:59:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd01dffb640 10 client.27515 handle_command_reply: tid=0 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 client.27515 shutdown 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 msgr2=0x55d8148a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd030009d60 tx=0x7fd0300047b0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 shutdown_connections 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8148bac90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd01dffb640 0 client.27515 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 --2- 172.21.15.60:0/2603081767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148a89d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.444+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 >> 172.21.15.60:0/2603081767 conn(0x55d81489a500 msgr2=0x55d8148a00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.445+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 shutdown_connections 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.445+0000 7fd03f078740 1 -- 172.21.15.60:0/2603081767 wait complete. 2024-07-24T21:59:30.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.445+0000 7fd03f078740 20 client.27515 trim_cache size 0 max 16384 2024-07-24T21:59:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.463+0000 7fd03f078740 1 Processor -- start 2024-07-24T21:59:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.463+0000 7fd03f078740 1 -- start start 2024-07-24T21:59:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd0180112d0 con 0x55d8148a8f80 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800eae0 con 0x55d81489c6d0 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd03f078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800c950 con 0x55d81489cff0 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd035d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55704/0 (socket says 172.21.15.60:55704) 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 -- 172.21.15.60:0/2120071980 learned_addr learned my addr 172.21.15.60:0/2120071980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 -- 172.21.15.60:0/2120071980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 msgr2=0x55d8148c0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148c0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 -- 172.21.15.60:0/2120071980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 msgr2=0x55d8148aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.464+0000 7fd036d9d640 1 -- 172.21.15.60:0/2120071980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8148b4d80 con 0x55d81489c6d0 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.465+0000 7fd036d9d640 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd030009bb0 tx=0x7fd0300047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54be4ab1df3c0afa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.465+0000 7fd01ffff640 1 -- 172.21.15.60:0/2120071980 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03002c070 con 0x55d81489c6d0 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.465+0000 7fd01ffff640 1 -- 172.21.15.60:0/2120071980 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd030028900 con 0x55d81489c6d0 2024-07-24T21:59:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.465+0000 7fd01ffff640 1 -- 172.21.15.60:0/2120071980 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000f8f0 con 0x55d81489c6d0 2024-07-24T21:59:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 -- 172.21.15.60:0/2120071980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 msgr2=0x55d8148c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd030009bb0 tx=0x7fd0300047b0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 -- 172.21.15.60:0/2120071980 shutdown_connections 2024-07-24T21:59:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489cff0 0x55d8148c0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 --2- 172.21.15.60:0/2120071980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489c6d0 0x55d8148c0450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 -- 172.21.15.60:0/2120071980 >> 172.21.15.60:0/2120071980 conn(0x55d81489a500 msgr2=0x55d8148a46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 -- 172.21.15.60:0/2120071980 shutdown_connections 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.466+0000 7fd03f078740 1 -- 172.21.15.60:0/2120071980 wait complete. 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 -- start start 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 Processor -- start 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489cff0 0x55d8148b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800cfa0 con 0x55d8148a8f80 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800e410 con 0x55d81489cff0 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800d670 con 0x55d81489c6d0 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd03f078740 10 client.0 _ll_register_callbacks cb 0x7fd030009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42002/0 (socket says 172.21.15.60:42002) 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.467+0000 7fd036d9d640 1 -- 172.21.15.60:0/4275489862 learned_addr learned my addr 172.21.15.60:0/4275489862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 -- 172.21.15.60:0/4275489862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489cff0 msgr2=0x55d8148b1440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489cff0 0x55d8148b1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 -- 172.21.15.60:0/4275489862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 msgr2=0x55d8148abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 -- 172.21.15.60:0/4275489862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8148ac3f0 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd036d9d640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd030028e80 tx=0x7fd030004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f173d7ba13c11f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.468+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03002c040 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd030028900 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fd030028450 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd03004c050 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 10 client.27466 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d814923c20 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd030028900 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd03000f690 con 0x55d81489c6d0 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 10 client.27466 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 10 client.27466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T21:59:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 20 client.27466 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8149262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 4 client.27466 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.469+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8148a5680 con 0x55d8148b88a0 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.470+0000 7fd035d9b640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8149262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.470+0000 7fd01dffb640 10 client.27466 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T21:59:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.470+0000 7fd035d9b640 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8149262d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f989cb93a50566e2 server_cookie=b7eb1ca8f9a38db2 in_seq=0 out_seq=0 2024-07-24T21:59:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.471+0000 7fd01dffb640 1 -- 172.21.15.60:0/4275489862 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8148a5680 con 0x55d8148b88a0 2024-07-24T21:59:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.471+0000 7fd01dffb640 10 client.27466 handle_command_reply: tid=0 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.471+0000 7fd03f078740 1 client.27466 shutdown 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 msgr2=0x55d8148b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd030028e80 tx=0x7fd030004820 comp rx=0 tx=0).stop 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 shutdown_connections 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55d8148b88a0 0x55d8149262d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8148a8f80 0x55d8148abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d81489cff0 0x55d8148b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 --2- 172.21.15.60:0/4275489862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d81489c6d0 0x55d8148b0f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 >> 172.21.15.60:0/4275489862 conn(0x55d81489a500 msgr2=0x55d8148a7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd01dffb640 0 client.27466 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 shutdown_connections 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 1 -- 172.21.15.60:0/4275489862 wait complete. 2024-07-24T21:59:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.472+0000 7fd03f078740 20 client.27466 trim_cache size 0 max 16384 2024-07-24T21:59:30.479 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 msgr2=0x7fd0200b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 0x7fd0200b6200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd028100f20 tx=0x7fd02800b040 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 msgr2=0x7fd038076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd02802a930 tx=0x7fd028005d30 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 shutdown_connections 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd0200b3d40 0x7fd0200b6200 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd038108b40 0x7fd0380754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0381081c0 0x7fd038074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 --2- 172.21.15.60:0/2085537597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0381027b0 0x7fd038076940 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 >> 172.21.15.60:0/2085537597 conn(0x7fd0380fe200 msgr2=0x7fd0380fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.473+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 shutdown_connections 2024-07-24T21:59:30.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:30.474+0000 7fd03ccc7640 1 -- 172.21.15.60:0/2085537597 wait complete. 2024-07-24T21:59:31.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T21:59:31.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:31 smithi060 ceph-mon[33000]: pgmap v777: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-07-24T21:59:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:31 smithi123 ceph-mon[37143]: pgmap v777: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-07-24T21:59:32.014 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:31 smithi086 ceph-mon[37128]: pgmap v777: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-07-24T21:59:33.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:33 smithi060 ceph-mon[33000]: pgmap v778: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-07-24T21:59:33.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:33 smithi123 ceph-mon[37143]: pgmap v778: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-07-24T21:59:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:33 smithi086 ceph-mon[37128]: pgmap v778: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-07-24T21:59:34.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.324+0000 7f74aedd1640 1 -- 172.21.15.60:0/1846162653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107a00 msgr2=0x7f74a810e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.324+0000 7f74aedd1640 1 --2- 172.21.15.60:0/1846162653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107a00 0x7f74a810e200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009ca0 tx=0x7f74a002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 -- 172.21.15.60:0/1846162653 shutdown_connections 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 --2- 172.21.15.60:0/1846162653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107a00 0x7f74a810e200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 --2- 172.21.15.60:0/1846162653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8107080 0x7f74a81074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 --2- 172.21.15.60:0/1846162653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8101670 0x7f74a8101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 -- 172.21.15.60:0/1846162653 >> 172.21.15.60:0/1846162653 conn(0x7f74a80fd100 msgr2=0x7f74a80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 -- 172.21.15.60:0/1846162653 shutdown_connections 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 -- 172.21.15.60:0/1846162653 wait complete. 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.325+0000 7f74aedd1640 1 Processor -- start 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 -- start start 2024-07-24T21:59:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8101670 0x7f74a8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8107a00 0x7f74a819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8110600 con 0x7f74a8107a00 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8110480 con 0x7f74a8107080 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74aedd1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8110780 con 0x7f74a8101670 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55738/0 (socket says 172.21.15.60:55738) 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74ad36e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8107a00 0x7f74a819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.326+0000 7f74a7fff640 1 -- 172.21.15.60:0/3577767492 learned_addr learned my addr 172.21.15.60:0/3577767492 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74acb6d640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8101670 0x7f74a8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 -- 172.21.15.60:0/3577767492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8101670 msgr2=0x7f74a8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8101670 0x7f74a8196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 -- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8107a00 msgr2=0x7f74a819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8107a00 0x7f74a819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 -- 172.21.15.60:0/3577767492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a819ccc0 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a7fff640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f749400b4a0 tx=0x7f749400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a5ffb640 1 -- 172.21.15.60:0/3577767492 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494014070 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.327+0000 7f74a5ffb640 1 -- 172.21.15.60:0/3577767492 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7494010ac0 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.328+0000 7f74a5ffb640 1 -- 172.21.15.60:0/3577767492 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494017be0 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.328+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a819cfb0 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.328+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81a7a80 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.332+0000 7f74a5ffb640 1 -- 172.21.15.60:0/3577767492 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7494017070 con 0x7f74a8107080 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.334+0000 7f74b1182740 1 Processor -- start 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.335+0000 7f74a5ffb640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 0x7f74980b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.335+0000 7f74ad36e640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 0x7f74980b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.335+0000 7f74a5ffb640 1 -- 172.21.15.60:0/3577767492 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f74940e32e0 con 0x7f74a8107080 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.336+0000 7f74ad36e640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 0x7f74980b6300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009a60 tx=0x7f74a0009540 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.336+0000 7f74b1182740 1 -- start start 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d316eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d317aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d317d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f0d30e59b0 con 0x55f0d316e6d0 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0d30d4eb0 con 0x55f0d316eff0 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74b1182740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0d30d8ef0 con 0x55f0d317af80 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d316eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d316eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51704/0 (socket says 172.21.15.60:51704) 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74acb6d640 1 -- 172.21.15.60:0/4145472526 learned_addr learned my addr 172.21.15.60:0/4145472526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74a7fff640 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d317aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.337+0000 7f74ad36e640 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d317d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 -- 172.21.15.60:0/4145472526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 msgr2=0x55f0d317d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d317d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 -- 172.21.15.60:0/4145472526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 msgr2=0x55f0d316eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d316eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 -- 172.21.15.60:0/4145472526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d305e320 con 0x55f0d316eff0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f74a7fff640 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d317aa40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74940e0a80 tx=0x7f74940047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89b6405c11c743a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f7492ffd640 1 -- 172.21.15.60:0/4145472526 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940b2ff0 con 0x55f0d316eff0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f7492ffd640 1 -- 172.21.15.60:0/4145472526 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7494010c70 con 0x55f0d316eff0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.338+0000 7f7492ffd640 1 -- 172.21.15.60:0/4145472526 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940e0bf0 con 0x55f0d316eff0 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.339+0000 7f74b1182740 1 -- 172.21.15.60:0/4145472526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 msgr2=0x55f0d317aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.339+0000 7f74b1182740 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d317aa40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74940e0a80 tx=0x7f74940047d0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.339+0000 7f74b1182740 1 -- 172.21.15.60:0/4145472526 shutdown_connections 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d317d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d317aa40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 --2- 172.21.15.60:0/4145472526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d316eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 -- 172.21.15.60:0/4145472526 >> 172.21.15.60:0/4145472526 conn(0x55f0d316c500 msgr2=0x55f0d316d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 -- 172.21.15.60:0/4145472526 shutdown_connections 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.340+0000 7f74b1182740 1 -- 172.21.15.60:0/4145472526 wait complete. 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.341+0000 7f74b1182740 1 -- start start 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.341+0000 7f74b1182740 1 Processor -- start 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74b1182740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74b1182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3178420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42046/0 (socket says 172.21.15.60:42046) 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74acb6d640 1 -- 172.21.15.60:0/793819495 learned_addr learned my addr 172.21.15.60:0/793819495 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74a7fff640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3178420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3178960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.342+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f0d2f38700 con 0x55f0d317af80 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0d30775f0 con 0x55f0d316eff0 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0d317dc00 con 0x55f0d316e6d0 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74ad36e640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3178960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 -- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 msgr2=0x55f0d3178420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3178420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 -- 172.21.15.60:0/793819495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 msgr2=0x55f0d3178960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3178960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d306a680 con 0x55f0d316e6d0 2024-07-24T21:59:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74acb6d640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f749c00a670 tx=0x7f749c00ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea353ec0f284a92a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f7487fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.343+0000 7f74b1182740 10 client.0 _ll_register_callbacks cb 0x55f0d3097960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7494010c70 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f7487fff640 1 -- 172.21.15.60:0/793819495 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c008910 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f7487fff640 1 -- 172.21.15.60:0/793819495 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f749c008ab0 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f7487fff640 1 -- 172.21.15.60:0/793819495 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f749c008cf0 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f74b1182740 10 client.27472 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.344+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55f0d2e35650 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f7487fff640 1 -- 172.21.15.60:0/793819495 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f749c017470 con 0x55f0d316e6d0 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 10 client.27472 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 10 client.27472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 20 client.27472 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d318a8a0 0x55f0d318ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 4 client.27472 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0d30e7900 con 0x55f0d318a8a0 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74a7fff640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d318a8a0 0x55f0d318ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f7487fff640 10 client.27472 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.345+0000 7f74a7fff640 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d318a8a0 0x55f0d318ccf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0a731477e8f8dbe server_cookie=27f5e82e62e5953b in_seq=0 out_seq=0 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f7487fff640 1 -- 172.21.15.60:0/793819495 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f749408ae80 con 0x55f0d318a8a0 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f7487fff640 10 client.27472 handle_command_reply: tid=0 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 client.27472 shutdown 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 msgr2=0x55f0d3177ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f749c00a670 tx=0x7f749c00ab30 comp rx=0 tx=0).stop 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 shutdown_connections 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d318a8a0 0x55f0d318ccf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3178960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3178420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 --2- 172.21.15.60:0/793819495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3177ee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 >> 172.21.15.60:0/793819495 conn(0x55f0d316c500 msgr2=0x55f0d31720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.346+0000 7f7487fff640 0 client.27472 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.347+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 shutdown_connections 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.347+0000 7f74b1182740 1 -- 172.21.15.60:0/793819495 wait complete. 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.347+0000 7f74b1182740 20 client.27472 trim_cache size 0 max 16384 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.367+0000 7f74b1182740 1 Processor -- start 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.367+0000 7f74b1182740 1 -- start start 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3192600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3192b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81b2ef0 con 0x55f0d317af80 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81b0700 con 0x55f0d316eff0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74b1182740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ae570 con 0x55f0d316e6d0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74acb6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42050/0 (socket says 172.21.15.60:42050) 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74acb6d640 1 -- 172.21.15.60:0/1515477188 learned_addr learned my addr 172.21.15.60:0/1515477188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.368+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3192600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74ad36e640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3192b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 -- 172.21.15.60:0/1515477188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 msgr2=0x55f0d3172820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 -- 172.21.15.60:0/1515477188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 msgr2=0x55f0d3192b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3192b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 -- 172.21.15.60:0/1515477188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d31763e0 con 0x55f0d316eff0 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74acb6d640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74ad36e640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3192b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3192600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f74940e0a80 tx=0x7f74940093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5701f12ba4fa9a91 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74927fc640 1 -- 172.21.15.60:0/1515477188 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940a9d10 con 0x55f0d316eff0 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74927fc640 1 -- 172.21.15.60:0/1515477188 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74940c9070 con 0x55f0d316eff0 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.369+0000 7f74927fc640 1 -- 172.21.15.60:0/1515477188 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940a91a0 con 0x55f0d316eff0 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 -- 172.21.15.60:0/1515477188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 msgr2=0x55f0d3192600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3192600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f74940e0a80 tx=0x7f74940093b0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 -- 172.21.15.60:0/1515477188 shutdown_connections 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d317af80 0x55f0d3192b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3192600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 --2- 172.21.15.60:0/1515477188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d316e6d0 0x55f0d3172820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 -- 172.21.15.60:0/1515477188 >> 172.21.15.60:0/1515477188 conn(0x55f0d316c500 msgr2=0x55f0d316b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.370+0000 7f74b1182740 1 -- 172.21.15.60:0/1515477188 shutdown_connections 2024-07-24T21:59:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.371+0000 7f74b1182740 1 -- 172.21.15.60:0/1515477188 wait complete. 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.371+0000 7f74b1182740 1 -- start start 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.371+0000 7f74b1182740 1 Processor -- start 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d317fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d31f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aebc0 con 0x55f0d316e6d0 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81b0030 con 0x55f0d316eff0 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81af290 con 0x55f0d317af80 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74b1182740 10 client.0 _ll_register_callbacks cb 0x7f749c00b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74ad36e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d31f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55770/0 (socket says 172.21.15.60:55770) 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 -- 172.21.15.60:0/1398327489 learned_addr learned my addr 172.21.15.60:0/1398327489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74acb6d640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d317fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 -- 172.21.15.60:0/1398327489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 msgr2=0x55f0d31f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d31f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 -- 172.21.15.60:0/1398327489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 msgr2=0x55f0d317fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d317fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.372+0000 7f74a7fff640 1 -- 172.21.15.60:0/1398327489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d31f64b0 con 0x55f0d316eff0 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f74a7fff640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74940aa100 tx=0x7f74940a9ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a306ad7b8ab70c9d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f7487fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940a61d0 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f74940e0bf0 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74940c9070 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f74940a6370 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f74b1182740 10 client.27536 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.373+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f0d31fd7c0 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74940c9070 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f74940a59d0 con 0x55f0d316eff0 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 10 client.27536 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 10 client.27536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 20 client.27536 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d31fdaf0 0x55f0d31ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 4 client.27536 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:59:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0d30e33a0 con 0x55f0d31fdaf0 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.374+0000 7f74acb6d640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d31fdaf0 0x55f0d31ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74acb6d640 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d31fdaf0 0x55f0d31ffee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75989f291791bdb8 server_cookie=ff97ffe7a9e78960 in_seq=0 out_seq=0 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f7487fff640 10 client.27536 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f7487fff640 1 -- 172.21.15.60:0/1398327489 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0d30e33a0 con 0x55f0d31fdaf0 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f7487fff640 10 client.27536 handle_command_reply: tid=0 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 client.27536 shutdown 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 msgr2=0x55f0d3180130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74940aa100 tx=0x7f74940a9ad0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.530 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 shutdown_connections 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55f0d31fdaf0 0x55f0d31ffee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f7487fff640 0 client.27536 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d317af80 0x55f0d31f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0d316eff0 0x55f0d3180130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 --2- 172.21.15.60:0/1398327489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d316e6d0 0x55f0d317fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.375+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 >> 172.21.15.60:0/1398327489 conn(0x55f0d316c500 msgr2=0x55f0d3176760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.376+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 shutdown_connections 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.376+0000 7f74b1182740 1 -- 172.21.15.60:0/1398327489 wait complete. 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.376+0000 7f74b1182740 20 client.27536 trim_cache size 0 max 16384 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 msgr2=0x7f74980b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 0x7f74980b6300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009a60 tx=0x7f74a0009540 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 msgr2=0x7f74a8197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f749400b4a0 tx=0x7f749400b960 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 shutdown_connections 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f74980b3e40 0x7f74980b6300 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a8107a00 0x7f74a819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8107080 0x7f74a8197150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 --2- 172.21.15.60:0/3577767492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8101670 0x7f74a8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 >> 172.21.15.60:0/3577767492 conn(0x7f74a80fd100 msgr2=0x7f74a80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 shutdown_connections 2024-07-24T21:59:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:34.377+0000 7f74aedd1640 1 -- 172.21.15.60:0/3577767492 wait complete. 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.947+0000 7f80dc641640 1 -- 172.21.15.60:0/1864240629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 msgr2=0x7f80d410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.947+0000 7f80dc641640 1 --2- 172.21.15.60:0/1864240629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d410ef20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009ca0 tx=0x7f80cc02a8c0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.947+0000 7f80dc641640 1 -- 172.21.15.60:0/1864240629 shutdown_connections 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.947+0000 7f80dc641640 1 --2- 172.21.15.60:0/1864240629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d410ef20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 --2- 172.21.15.60:0/1864240629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d410c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 --2- 172.21.15.60:0/1864240629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 0x7f80d4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 -- 172.21.15.60:0/1864240629 >> 172.21.15.60:0/1864240629 conn(0x7f80d40fd080 msgr2=0x7f80d40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 -- 172.21.15.60:0/1864240629 shutdown_connections 2024-07-24T21:59:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 -- 172.21.15.60:0/1864240629 wait complete. 2024-07-24T21:59:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 Processor -- start 2024-07-24T21:59:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.948+0000 7f80dc641640 1 -- start start 2024-07-24T21:59:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 0x7f80d4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4112760 con 0x7f80d410caa0 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41125e0 con 0x7f80d4103f70 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dc641640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41128e0 con 0x7f80d4103650 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55790/0 (socket says 172.21.15.60:55790) 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 -- 172.21.15.60:0/1172830794 learned_addr learned my addr 172.21.15.60:0/1172830794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80da3dd640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 0x7f80d4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80dabde640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 -- 172.21.15.60:0/1172830794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 msgr2=0x7f80d4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 0x7f80d4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 -- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 msgr2=0x7f80d419e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d419e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.949+0000 7f80d9bdc640 1 -- 172.21.15.60:0/1172830794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d419ecc0 con 0x7f80d4103f70 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9c0 tx=0x7f80c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80cb7fe640 1 -- 172.21.15.60:0/1172830794 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400ccc0 con 0x7f80d4103f70 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80cb7fe640 1 -- 172.21.15.60:0/1172830794 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c40035f0 con 0x7f80d4103f70 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d419efb0 con 0x7f80d4103f70 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80cb7fe640 1 -- 172.21.15.60:0/1172830794 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400ce60 con 0x7f80d4103f70 2024-07-24T21:59:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.950+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41a5850 con 0x7f80d4103f70 2024-07-24T21:59:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.954+0000 7f80df39a740 1 Processor -- start 2024-07-24T21:59:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.954+0000 7f80cb7fe640 1 -- 172.21.15.60:0/1172830794 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f80c4004000 con 0x7f80d4103f70 2024-07-24T21:59:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.954+0000 7f80df39a740 1 -- start start 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f7e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 0x558559f80970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558559ee3280 con 0x558559f6e1e0 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558559ed2780 con 0x558559f6d8c0 2024-07-24T21:59:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80df39a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558559ed67c0 con 0x558559f7e580 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:55804/0 (socket says 172.21.15.60:55804) 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 -- 172.21.15.60:0/4092186398 learned_addr learned my addr 172.21.15.60:0/4092186398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80da3dd640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f7e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 0x558559f80970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 -- 172.21.15.60:0/4092186398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 msgr2=0x558559f80970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 0x558559f80970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 -- 172.21.15.60:0/4092186398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 msgr2=0x558559f7e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f7e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80dabde640 1 -- 172.21.15.60:0/4092186398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558559c36790 con 0x558559f6d8c0 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 0x558559f80970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.955+0000 7f80da3dd640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f7e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80cb7fe640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 0x7f80c00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80dabde640 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009ca0 tx=0x7f80cc0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6df3b70582edfd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80da3dd640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 0x7f80c00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80cb7fe640 1 -- 172.21.15.60:0/1172830794 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f80c40e4270 con 0x7f80d4103f70 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80c8bfe640 1 -- 172.21.15.60:0/4092186398 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc031cf0 con 0x558559f6d8c0 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80c8bfe640 1 -- 172.21.15.60:0/4092186398 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80cc036030 con 0x558559f6d8c0 2024-07-24T21:59:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.956+0000 7f80c8bfe640 1 -- 172.21.15.60:0/4092186398 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0042a0 con 0x558559f6d8c0 2024-07-24T21:59:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 -- 172.21.15.60:0/4092186398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 msgr2=0x558559f6dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009ca0 tx=0x7f80cc0093e0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80da3dd640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 0x7f80c00b6280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009db0 tx=0x7f80d0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 -- 172.21.15.60:0/4092186398 shutdown_connections 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f7e580 0x558559f80970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f7e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 --2- 172.21.15.60:0/4092186398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f6d8c0 0x558559f6dca0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.957+0000 7f80df39a740 1 -- 172.21.15.60:0/4092186398 >> 172.21.15.60:0/4092186398 conn(0x558559f6b6f0 msgr2=0x558559f6c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.958+0000 7f80df39a740 1 -- 172.21.15.60:0/4092186398 shutdown_connections 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.958+0000 7f80df39a740 1 -- 172.21.15.60:0/4092186398 wait complete. 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.958+0000 7f80df39a740 1 -- start start 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.958+0000 7f80df39a740 1 Processor -- start 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.958+0000 7f80df39a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f7dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f7b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f7b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558559e0cab0 con 0x558559f6d8c0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558559e5a9e0 con 0x558559f7e580 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558559f81080 con 0x558559f6e1e0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80da3dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f7dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80da3dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f7dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51756/0 (socket says 172.21.15.60:51756) 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80da3dd640 1 -- 172.21.15.60:0/3656330770 learned_addr learned my addr 172.21.15.60:0/3656330770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f7b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f7b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80df39a740 10 client.0 _ll_register_callbacks cb 0x558559be2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 -- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 msgr2=0x558559f7b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f7b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 -- 172.21.15.60:0/3656330770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 msgr2=0x558559f7dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f7dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 -- 172.21.15.60:0/3656330770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558559caf620 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80dabde640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f7b740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009bf0 tx=0x7f80cc031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5d8fe6e9af7d8e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.959+0000 7f80bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc003970 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558559e81ce0 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f80cc02f610 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f80cc05d050 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 10 client.27548 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558559f8c790 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558559e81ce0 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f80cc04e070 con 0x558559f7e580 2024-07-24T21:59:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 10 client.27548 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 10 client.27548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 20 client.27548 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559f8dae0 0x558559f8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 4 client.27548 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.960+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558559ee51d0 con 0x558559f8dae0 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.961+0000 7f80da3dd640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559f8dae0 0x558559f8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.961+0000 7f80da3dd640 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559f8dae0 0x558559f8fed0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376fc4b337f7728d server_cookie=504897d1daadb1a7 in_seq=0 out_seq=0 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.961+0000 7f80bdffb640 10 client.27548 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80bdffb640 1 -- 172.21.15.60:0/3656330770 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558559ee51d0 con 0x558559f8dae0 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80bdffb640 10 client.27548 handle_command_reply: tid=0 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 client.27548 shutdown 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 msgr2=0x558559f7b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f7b740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009bf0 tx=0x7f80cc031680 comp rx=0 tx=0).stop 2024-07-24T21:59:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 shutdown_connections 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559f8dae0 0x558559f8fed0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f7b740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f7b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.962+0000 7f80df39a740 1 --2- 172.21.15.60:0/3656330770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f7dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.963+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 >> 172.21.15.60:0/3656330770 conn(0x558559f6b6f0 msgr2=0x558559f71290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.963+0000 7f80bdffb640 0 client.27548 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.963+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 shutdown_connections 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.963+0000 7f80df39a740 1 -- 172.21.15.60:0/3656330770 wait complete. 2024-07-24T21:59:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.963+0000 7f80df39a740 20 client.27548 trim_cache size 0 max 16384 2024-07-24T21:59:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.985+0000 7f80df39a740 1 Processor -- start 2024-07-24T21:59:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 -- start start 2024-07-24T21:59:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80da3dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80da3dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51768/0 (socket says 172.21.15.60:51768) 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80da3dd640 1 -- 172.21.15.60:0/2064613787 learned_addr learned my addr 172.21.15.60:0/2064613787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f79fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ae8e0 con 0x558559f6d8c0 2024-07-24T21:59:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ac0f0 con 0x558559f7e580 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41a9f60 con 0x558559f6e1e0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.986+0000 7f80dabde640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f79fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 -- 172.21.15.60:0/2064613787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 msgr2=0x558559f79a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f79a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 -- 172.21.15.60:0/2064613787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 msgr2=0x558559f79fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f79fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 -- 172.21.15.60:0/2064613787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558559f6b520 con 0x558559f6d8c0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f80d001d920 tx=0x7f80d001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2331cd08a851455b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80bffff640 1 -- 172.21.15.60:0/2064613787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0033430 con 0x558559f6d8c0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80bffff640 1 -- 172.21.15.60:0/2064613787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80d002a8a0 con 0x558559f6d8c0 2024-07-24T21:59:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.987+0000 7f80bffff640 1 -- 172.21.15.60:0/2064613787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0032700 con 0x558559f6d8c0 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 msgr2=0x558559f8a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f80d001d920 tx=0x7f80d001b000 comp rx=0 tx=0).stop 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 shutdown_connections 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6e1e0 0x558559f79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 --2- 172.21.15.60:0/2064613787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6d8c0 0x558559f8a1a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 >> 172.21.15.60:0/2064613787 conn(0x558559f6b6f0 msgr2=0x558559f6c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 shutdown_connections 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.988+0000 7f80df39a740 1 -- 172.21.15.60:0/2064613787 wait complete. 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 -- start start 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 Processor -- start 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6d8c0 0x558559f95a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f86850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41aa5b0 con 0x558559f6e1e0 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41aba20 con 0x558559f7e580 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41aac80 con 0x558559f6d8c0 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80df39a740 10 client.0 _ll_register_callbacks cb 0x7f80cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80dabde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f86850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51774/0 (socket says 172.21.15.60:51774) 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 -- 172.21.15.60:0/2482422806 learned_addr learned my addr 172.21.15.60:0/2482422806 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6d8c0 0x558559f95a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 -- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6d8c0 msgr2=0x558559f95a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6d8c0 0x558559f95a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 -- 172.21.15.60:0/2482422806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 msgr2=0x558559f86850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f86850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80d9bdc640 1 -- 172.21.15.60:0/2482422806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558559f86e90 con 0x558559f6e1e0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.989+0000 7f80dabde640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f86850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80d9bdc640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c6a0 tx=0x7f80c4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc9401fee0131a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40ab520 con 0x558559f6e1e0 2024-07-24T21:59:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7f80c40aa080 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d002a8a0 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f80c40aae20 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80df39a740 10 client.18174 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558559ff7620 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.990+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80d002a8a0 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f80c40a9af0 con 0x558559f6e1e0 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 10 client.18174 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 10 client.18174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 20 client.18174 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559ff7e80 0x558559ffa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 4 client.18174 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:59:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558559ee0c70 con 0x558559ff7e80 2024-07-24T21:59:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559ff7e80 0x558559ffa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80da3dd640 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559ff7e80 0x558559ffa270 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47700dc0fd6f0caa server_cookie=73bf6530e61d1223 in_seq=0 out_seq=0 2024-07-24T21:59:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:34.991+0000 7f80bdffb640 10 client.18174 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.545+0000 7f80bdffb640 1 -- 172.21.15.60:0/2482422806 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558559ee0c70 con 0x558559ff7e80 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.545+0000 7f80bdffb640 10 client.18174 handle_command_reply: tid=0 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 client.18174 shutdown 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 msgr2=0x558559f86310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c6a0 tx=0x7f80c4003c80 comp rx=0 tx=0).stop 2024-07-24T21:59:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 shutdown_connections 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558559ff7e80 0x558559ffa270 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558559f7e580 0x558559f86850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558559f6e1e0 0x558559f86310 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80bdffb640 0 client.18174 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 --2- 172.21.15.60:0/2482422806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558559f6d8c0 0x558559f95a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 >> 172.21.15.60:0/2482422806 conn(0x558559f6b6f0 msgr2=0x558559f798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 shutdown_connections 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 1 -- 172.21.15.60:0/2482422806 wait complete. 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.546+0000 7f80df39a740 20 client.18174 trim_cache size 0 max 16384 2024-07-24T21:59:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:59:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:59:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:59:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 msgr2=0x7f80c00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 0x7f80c00b6280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009db0 tx=0x7f80d0005f90 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 msgr2=0x7f80d4199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9c0 tx=0x7f80c400ee80 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 shutdown_connections 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f80c00b3dc0 0x7f80c00b6280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d410caa0 0x7f80d419e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80d4103f70 0x7f80d4199160 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 --2- 172.21.15.60:0/1172830794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80d4103650 0x7f80d4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 >> 172.21.15.60:0/1172830794 conn(0x7f80d40fd080 msgr2=0x7f80d410ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 shutdown_connections 2024-07-24T21:59:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:35.547+0000 7f80dc641640 1 -- 172.21.15.60:0/1172830794 wait complete. 2024-07-24T21:59:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:35 smithi060 ceph-mon[33000]: pgmap v779: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 959 KiB/s wr, 196 op/s 2024-07-24T21:59:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:35 smithi123 ceph-mon[37143]: pgmap v779: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 959 KiB/s wr, 196 op/s 2024-07-24T21:59:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:35 smithi086 ceph-mon[37128]: pgmap v779: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 959 KiB/s wr, 196 op/s 2024-07-24T21:59:36.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T21:59:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:37 smithi060 ceph-mon[33000]: pgmap v780: 97 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 66 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 815 KiB/s wr, 210 op/s 2024-07-24T21:59:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:37 smithi123 ceph-mon[37143]: pgmap v780: 97 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 66 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 815 KiB/s wr, 210 op/s 2024-07-24T21:59:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:37 smithi086 ceph-mon[37128]: pgmap v780: 97 pgs: 8 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 66 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 815 KiB/s wr, 210 op/s 2024-07-24T21:59:39.575 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:39 smithi060 ceph-mon[33000]: pgmap v781: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-07-24T21:59:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.564+0000 7fb0f1826640 1 -- 172.21.15.60:0/383589269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec108b20 msgr2=0x7fb0ec10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.564+0000 7fb0f1826640 1 --2- 172.21.15.60:0/383589269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec108b20 0x7fb0ec10f320 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e402a8c0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 -- 172.21.15.60:0/383589269 shutdown_connections 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 --2- 172.21.15.60:0/383589269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec108b20 0x7fb0ec10f320 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 --2- 172.21.15.60:0/383589269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 --2- 172.21.15.60:0/383589269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 0x7fb0ec102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 -- 172.21.15.60:0/383589269 >> 172.21.15.60:0/383589269 conn(0x7fb0ec0fe200 msgr2=0x7fb0ec100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 -- 172.21.15.60:0/383589269 shutdown_connections 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 -- 172.21.15.60:0/383589269 wait complete. 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.565+0000 7fb0f1826640 1 Processor -- start 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 -- start start 2024-07-24T21:59:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 0x7fb0ec074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec108b20 0x7fb0ec075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec112740 con 0x7fb0ec1081a0 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1125c0 con 0x7fb0ec108b20 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0f1826640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1128c0 con 0x7fb0ec102790 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41084/0 (socket says 172.21.15.60:41084) 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/3164648122 learned_addr learned my addr 172.21.15.60:0/3164648122 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 0x7fb0ec074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec108b20 0x7fb0ec075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/3164648122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 msgr2=0x7fb0ec074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 0x7fb0ec074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/3164648122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec108b20 msgr2=0x7fb0ec075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec108b20 0x7fb0ec075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.566+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/3164648122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec1a6ff0 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.567+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000ea60 tx=0x7fb0e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.569+0000 7fb0f0824640 1 -- 172.21.15.60:0/3164648122 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000cde0 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.569+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec1a72e0 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.569+0000 7fb0f0824640 1 -- 172.21.15.60:0/3164648122 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00035f0 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.569+0000 7fb0f0824640 1 -- 172.21.15.60:0/3164648122 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0010690 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.569+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec1a7820 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.571+0000 7fb0f0824640 1 -- 172.21.15.60:0/3164648122 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fb0e0010830 con 0x7fb0ec1081a0 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.571+0000 7fb0f3bd7740 1 Processor -- start 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.571+0000 7fb0f3bd7740 1 -- start start 2024-07-24T21:59:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.571+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7db2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7db5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557da7d1d9b0 con 0x557da7da66d0 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557da7d0ceb0 con 0x557da7db2f80 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557da7d10ef0 con 0x557da7da6ff0 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7db2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41100/0 (socket says 172.21.15.60:41100) 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eb7fe640 1 -- 172.21.15.60:0/784650799 learned_addr learned my addr 172.21.15.60:0/784650799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7db5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 -- 172.21.15.60:0/784650799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 msgr2=0x557da7db5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7db5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 -- 172.21.15.60:0/784650799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 msgr2=0x557da7da6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 -- 172.21.15.60:0/784650799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da7c96320 con 0x557da7da6ff0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f0824640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 0x7fb0d40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7db2a40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00bbf0 tx=0x7fb0dc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c107582b0e40d3eb server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 0x7fb0d40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0f0824640 1 -- 172.21.15.60:0/3164648122 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fb0e00e4330 con 0x7fb0ec1081a0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0d1ffb640 1 -- 172.21.15.60:0/784650799 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc004030 con 0x557da7da6ff0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0d1ffb640 1 -- 172.21.15.60:0/784650799 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0dc0035f0 con 0x557da7da6ff0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.572+0000 7fb0d1ffb640 1 -- 172.21.15.60:0/784650799 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011660 con 0x557da7da6ff0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/784650799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 msgr2=0x557da7db2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 0x7fb0d40b6200 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e40099a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7db2a40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00bbf0 tx=0x7fb0dc0107a0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/784650799 shutdown_connections 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7db5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7db2a40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/784650799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/784650799 >> 172.21.15.60:0/784650799 conn(0x557da7da4500 msgr2=0x557da7da56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.573+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/784650799 shutdown_connections 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.574+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/784650799 wait complete. 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.582+0000 7fb0f3bd7740 1 -- start start 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.583+0000 7fb0f3bd7740 1 Processor -- start 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.583+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.583+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7dafe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.583+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557da7b70700 con 0x557da7da66d0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557da7caf5f0 con 0x557da7da6ff0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557da7db5c00 con 0x557da7db2f80 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0f3bd7740 10 client.0 _ll_register_callbacks cb 0x557da7ccf960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41104/0 (socket says 172.21.15.60:41104) 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1831041390 learned_addr learned my addr 172.21.15.60:0/1831041390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7dafe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 msgr2=0x557da7db0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1831041390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 msgr2=0x557da7dafe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7dafe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1831041390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da7ca2680 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011010 tx=0x7fb0dc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5d8514382309bc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.584+0000 7fb0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc02a060 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fb0dc0039b0 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0dc0035f0 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc003fe0 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0f3bd7740 10 client.18186 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557da7a6d650 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.585+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0dc0035f0 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb0dc0046c0 con 0x557da7da66d0 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 10 client.18186 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:39.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 20 client.18186 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7dcb0a0 0x557da7dcd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 4 client.18186 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.586+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557da7d1f900 con 0x557da7dcb0a0 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.587+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7dcb0a0 0x557da7dcd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.587+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7dcb0a0 0x557da7dcd490 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f5ed3c0f7ca299 server_cookie=a30f579b7c4699e6 in_seq=0 out_seq=0 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.587+0000 7fb0caffd640 10 client.18186 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0caffd640 1 -- 172.21.15.60:0/1831041390 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557da7d1f900 con 0x557da7dcb0a0 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0caffd640 10 client.18186 handle_command_reply: tid=0 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 client.18186 shutdown 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 msgr2=0x557da7db2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011010 tx=0x7fb0dc00bfd0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 shutdown_connections 2024-07-24T21:59:39.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7dcb0a0 0x557da7dcd490 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7dafe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0caffd640 0 client.18186 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1831041390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7db2a20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 >> 172.21.15.60:0/1831041390 conn(0x557da7da4500 msgr2=0x557da7daa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 shutdown_connections 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1831041390 wait complete. 2024-07-24T21:59:39.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.588+0000 7fb0f3bd7740 20 client.18186 trim_cache size 0 max 16384 2024-07-24T21:59:39.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 Processor -- start 2024-07-24T21:59:39.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 -- start start 2024-07-24T21:59:39.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7dd2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7dd2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1ae9c0 con 0x557da7da66d0 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1ac1d0 con 0x557da7db2f80 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.613+0000 7fb0f3bd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1aa040 con 0x557da7da6ff0 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41110/0 (socket says 172.21.15.60:41110) 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0eaffd640 1 -- 172.21.15.60:0/2031798227 learned_addr learned my addr 172.21.15.60:0/2031798227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7dd2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7dd2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/2031798227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 msgr2=0x557da7dd2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7dd2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/2031798227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 msgr2=0x557da7da3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 -- 172.21.15.60:0/2031798227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da7da6e20 con 0x557da7da6ff0 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.614+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7dd2a90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00e1a20 tx=0x7fb0e0003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f08fe5912c1b3dc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.615+0000 7fb0d17fa640 1 -- 172.21.15.60:0/2031798227 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00e3030 con 0x557da7da6ff0 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.615+0000 7fb0d17fa640 1 -- 172.21.15.60:0/2031798227 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00aa9f0 con 0x557da7da6ff0 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.615+0000 7fb0d17fa640 1 -- 172.21.15.60:0/2031798227 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00af4e0 con 0x557da7da6ff0 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.615+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/2031798227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 msgr2=0x557da7dd2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.615+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7dd2a90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00e1a20 tx=0x7fb0e0003e90 comp rx=0 tx=0).stop 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/2031798227 shutdown_connections 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7db2f80 0x557da7dd2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7da6ff0 0x557da7dd2a90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/2031798227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7da3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/2031798227 >> 172.21.15.60:0/2031798227 conn(0x557da7da4500 msgr2=0x557da7da2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/2031798227 shutdown_connections 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/2031798227 wait complete. 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.616+0000 7fb0f3bd7740 1 -- start start 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 Processor -- start 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7db40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41120/0 (socket says 172.21.15.60:41120) 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1317593817 learned_addr learned my addr 172.21.15.60:0/1317593817 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1aa690 con 0x557da7da66d0 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1abb00 con 0x557da7da6ff0 2024-07-24T21:59:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1aad60 con 0x557da7db2f80 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.617+0000 7fb0f3bd7740 10 client.0 _ll_register_callbacks cb 0x7fb0e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7db40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eb7fe640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 msgr2=0x557da7db45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1317593817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 msgr2=0x557da7db40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7db40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da7db4c30 con 0x557da7da66d0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0eaffd640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011010 tx=0x7fb0dc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d758b04221d82675 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc03a060 con 0x557da7da66d0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8609+0+0 (secure 0 0 0) 0x7fb0dc004390 con 0x557da7da66d0 2024-07-24T21:59:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.618+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e00aa9f0 con 0x557da7da66d0 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc02a040 con 0x557da7da66d0 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 10 client.18192 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557da7dc2550 con 0x557da7da66d0 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00aa9f0 con 0x557da7da66d0 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb0dc005240 con 0x557da7da66d0 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 10 client.18192 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 20 client.18192 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7e36020 0x557da7e38440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 4 client.18192 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557da7d1b3a0 con 0x557da7e36020 2024-07-24T21:59:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.619+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7e36020 0x557da7e38440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.620+0000 7fb0caffd640 10 client.18192 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.620+0000 7fb0ea7fc640 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7e36020 0x557da7e38440 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c86fb6791acac0b server_cookie=351a6e0adff429f3 in_seq=0 out_seq=0 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.620+0000 7fb0caffd640 1 -- 172.21.15.60:0/1317593817 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557da7d1b3a0 con 0x557da7e36020 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.620+0000 7fb0caffd640 10 client.18192 handle_command_reply: tid=0 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.620+0000 7fb0f3bd7740 1 client.18192 shutdown 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 msgr2=0x557da7dc3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011010 tx=0x7fb0dc00bab0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 shutdown_connections 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557da7e36020 0x557da7e38440 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557da7db2f80 0x557da7db45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557da7da6ff0 0x557da7db40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 --2- 172.21.15.60:0/1317593817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557da7da66d0 0x557da7dc3ba0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0caffd640 0 client.18192 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 >> 172.21.15.60:0/1317593817 conn(0x557da7da4500 msgr2=0x557da7da3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 shutdown_connections 2024-07-24T21:59:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 1 -- 172.21.15.60:0/1317593817 wait complete. 2024-07-24T21:59:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.621+0000 7fb0f3bd7740 20 client.18192 trim_cache size 0 max 16384 2024-07-24T21:59:39.632 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 msgr2=0x7fb0d40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 0x7fb0d40b6200 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e40099a0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 msgr2=0x7fb0ec0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000ea60 tx=0x7fb0e000ef20 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 shutdown_connections 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fb0d40b3d40 0x7fb0d40b6200 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0ec108b20 0x7fb0ec075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ec1081a0 0x7fb0ec0753d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 --2- 172.21.15.60:0/3164648122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0ec102790 0x7fb0ec074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.622+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 >> 172.21.15.60:0/3164648122 conn(0x7fb0ec0fe200 msgr2=0x7fb0ec1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:39.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.623+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 shutdown_connections 2024-07-24T21:59:39.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:39.623+0000 7fb0f1826640 1 -- 172.21.15.60:0/3164648122 wait complete. 2024-07-24T21:59:39.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:39 smithi086 ceph-mon[37128]: pgmap v781: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-07-24T21:59:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:39 smithi123 ceph-mon[37143]: pgmap v781: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-07-24T21:59:41.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T21:59:41.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T21:59:41.694 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:41.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:41 smithi060 ceph-mon[33000]: pgmap v782: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1023 KiB/s wr, 220 op/s 2024-07-24T21:59:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:41 smithi123 ceph-mon[37143]: pgmap v782: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1023 KiB/s wr, 220 op/s 2024-07-24T21:59:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:41 smithi086 ceph-mon[37128]: pgmap v782: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1023 KiB/s wr, 220 op/s 2024-07-24T21:59:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 -- 172.21.15.60:0/4075880070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 msgr2=0x7fd46010c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 --2- 172.21.15.60:0/4075880070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 0x7fd46010c490 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd450009900 tx=0x7fd45002a410 comp rx=0 tx=0).stop 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 -- 172.21.15.60:0/4075880070 shutdown_connections 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 --2- 172.21.15.60:0/4075880070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46010cae0 0x7fd46010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 --2- 172.21.15.60:0/4075880070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 0x7fd46010c490 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 --2- 172.21.15.60:0/4075880070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd460103680 0x7fd460103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 -- 172.21.15.60:0/4075880070 >> 172.21.15.60:0/4075880070 conn(0x7fd4600fd120 msgr2=0x7fd4600ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 -- 172.21.15.60:0/4075880070 shutdown_connections 2024-07-24T21:59:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.632+0000 7fd465c79640 1 -- 172.21.15.60:0/4075880070 wait complete. 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 Processor -- start 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 -- start start 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd460103680 0x7fd460102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 0x7fd460101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd460112790 con 0x7fd460103fa0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd460112610 con 0x7fd460103680 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.633+0000 7fd465c79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd460112910 con 0x7fd46010cae0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38694/0 (socket says 172.21.15.60:38694) 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 -- 172.21.15.60:0/545135760 learned_addr learned my addr 172.21.15.60:0/545135760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 -- 172.21.15.60:0/545135760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd460103680 msgr2=0x7fd460102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd460103680 0x7fd460102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 -- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 msgr2=0x7fd460101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 0x7fd460101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.634+0000 7fd45ffff640 1 -- 172.21.15.60:0/545135760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4601ab5e0 con 0x7fd46010cae0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd45ffff640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd45800eb10 tx=0x7fd45800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd45cff9640 1 -- 172.21.15.60:0/545135760 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458004b20 con 0x7fd46010cae0 2024-07-24T21:59:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd45cff9640 1 -- 172.21.15.60:0/545135760 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd458004cc0 con 0x7fd46010cae0 2024-07-24T21:59:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd45cff9640 1 -- 172.21.15.60:0/545135760 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458010690 con 0x7fd46010cae0 2024-07-24T21:59:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4601ab810 con 0x7fd46010cae0 2024-07-24T21:59:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.635+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4601abd50 con 0x7fd46010cae0 2024-07-24T21:59:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.639+0000 7fd46802a740 1 Processor -- start 2024-07-24T21:59:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.640+0000 7fd45cff9640 1 -- 172.21.15.60:0/545135760 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fd458005000 con 0x7fd46010cae0 2024-07-24T21:59:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.640+0000 7fd46802a740 1 -- start start 2024-07-24T21:59:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db81bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55884daea9f0 con 0x55884db7f7d0 2024-07-24T21:59:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55884dad9ef0 con 0x55884db73840 2024-07-24T21:59:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd46802a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55884daddf30 con 0x55884db72f20 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49126/0 (socket says 172.21.15.60:49126) 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2093744933 learned_addr learned my addr 172.21.15.60:0/2093744933 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2093744933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 msgr2=0x55884db73300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.641+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2093744933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 msgr2=0x55884db81bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45effd640 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db81bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db81bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2093744933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884d83a930 con 0x55884db73840 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45effd640 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db81bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T21:59:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45cff9640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 0x7fd4480b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45cff9640 1 -- 172.21.15.60:0/545135760 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fd4580e46f0 con 0x7fd46010cae0 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.642+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd454009820 tx=0x7fd454009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afa366cfe2dae97a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.643+0000 7fd445ffb640 1 -- 172.21.15.60:0/2093744933 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454015020 con 0x55884db73840 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.643+0000 7fd45effd640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 0x7fd4480b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.643+0000 7fd445ffb640 1 -- 172.21.15.60:0/2093744933 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454011420 con 0x55884db73840 2024-07-24T21:59:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.643+0000 7fd445ffb640 1 -- 172.21.15.60:0/2093744933 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540105e0 con 0x55884db73840 2024-07-24T21:59:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.644+0000 7fd46802a740 1 -- 172.21.15.60:0/2093744933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 msgr2=0x55884db7f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.644+0000 7fd46802a740 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd454009820 tx=0x7fd454009ce0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd45effd640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 0x7fd4480b6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd450005ab0 tx=0x7fd450000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd46802a740 1 -- 172.21.15.60:0/2093744933 shutdown_connections 2024-07-24T21:59:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd46802a740 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db81bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd46802a740 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db7f290 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd46802a740 1 --2- 172.21.15.60:0/2093744933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.645+0000 7fd46802a740 1 -- 172.21.15.60:0/2093744933 >> 172.21.15.60:0/2093744933 conn(0x55884db70d50 msgr2=0x55884db71ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.647+0000 7fd46802a740 1 -- 172.21.15.60:0/2093744933 shutdown_connections 2024-07-24T21:59:43.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.647+0000 7fd46802a740 1 -- 172.21.15.60:0/2093744933 wait complete. 2024-07-24T21:59:43.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.647+0000 7fd46802a740 1 -- start start 2024-07-24T21:59:43.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 Processor -- start 2024-07-24T21:59:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db73840 0x55884db7c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 0x55884db7cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55884d93d890 con 0x55884db73840 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55884da7c390 con 0x55884db7f7d0 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd46802a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55884db82450 con 0x55884db72f20 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.648+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db7f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38718/0 (socket says 172.21.15.60:38718) 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2443041904 learned_addr learned my addr 172.21.15.60:0/2443041904 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd46802a740 10 client.0 _ll_register_callbacks cb 0x55884da86500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45effd640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db73840 0x55884db7c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 0x55884db7cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 -- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 msgr2=0x55884db7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 -- 172.21.15.60:0/2443041904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db73840 msgr2=0x55884db7c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db73840 0x55884db7c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 -- 172.21.15.60:0/2443041904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884da76d60 con 0x55884db7f7d0 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd45ffff640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 0x55884db7cea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4580e1e10 tx=0x7fd4580aab70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cea939aa557bd97 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.649+0000 7fd43effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.650+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580af8d0 con 0x55884db7f7d0 2024-07-24T21:59:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.650+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd454011420 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.650+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fd4580a78c0 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.651+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd4580cc050 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.651+0000 7fd46802a740 10 client.27581 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.651+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55884db8e370 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.651+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454011420 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd4580a9fa0 con 0x55884db7f7d0 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd46802a740 10 client.27581 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd46802a740 10 client.27581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd46802a740 20 client.27581 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884db8fc20 0x55884db92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.652+0000 7fd46802a740 4 client.27581 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.653+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55884daec940 con 0x55884db8fc20 2024-07-24T21:59:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.653+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884db8fc20 0x55884db92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.653+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884db8fc20 0x55884db92040 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca603f73c073845 server_cookie=78ad20caf9d51fb5 in_seq=0 out_seq=0 2024-07-24T21:59:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.653+0000 7fd43effd640 10 client.27581 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.654+0000 7fd43effd640 1 -- 172.21.15.60:0/2443041904 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55884daec940 con 0x55884db8fc20 2024-07-24T21:59:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.654+0000 7fd43effd640 10 client.27581 handle_command_reply: tid=0 2024-07-24T21:59:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.655+0000 7fd46802a740 1 client.27581 shutdown 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.655+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 msgr2=0x55884db7cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.655+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 0x55884db7cea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4580e1e10 tx=0x7fd4580aab70 comp rx=0 tx=0).stop 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 shutdown_connections 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884db8fc20 0x55884db92040 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db7f7d0 0x55884db7cea0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db73840 0x55884db7c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 --2- 172.21.15.60:0/2443041904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 >> 172.21.15.60:0/2443041904 conn(0x55884db70d50 msgr2=0x55884db768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd43effd640 0 client.27581 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 shutdown_connections 2024-07-24T21:59:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 1 -- 172.21.15.60:0/2443041904 wait complete. 2024-07-24T21:59:43.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.656+0000 7fd46802a740 20 client.27581 trim_cache size 0 max 16384 2024-07-24T21:59:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.680+0000 7fd46802a740 1 Processor -- start 2024-07-24T21:59:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.680+0000 7fd46802a740 1 -- start start 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db73840 0x55884db7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b7200 con 0x55884db7f7d0 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49134/0 (socket says 172.21.15.60:49134) 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd45f7fe640 1 -- 172.21.15.60:0/1543289523 learned_addr learned my addr 172.21.15.60:0/1543289523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b4a10 con 0x55884db72f20 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd45effd640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db73840 0x55884db7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd45ffff640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.681+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b2880 con 0x55884db73840 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 -- 172.21.15.60:0/1543289523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db73840 msgr2=0x55884db7b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db73840 0x55884db7b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 -- 172.21.15.60:0/1543289523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 msgr2=0x55884db97950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db97950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 -- 172.21.15.60:0/1543289523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884db73670 con 0x55884db72f20 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45ffff640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd454009140 tx=0x7fd45401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10df133b07531303 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd4457fa640 1 -- 172.21.15.60:0/1543289523 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454015040 con 0x55884db72f20 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd4457fa640 1 -- 172.21.15.60:0/1543289523 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45400bd80 con 0x55884db72f20 2024-07-24T21:59:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.682+0000 7fd4457fa640 1 -- 172.21.15.60:0/1543289523 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454011e60 con 0x55884db72f20 2024-07-24T21:59:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 msgr2=0x55884db7af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd454009140 tx=0x7fd45401c040 comp rx=0 tx=0).stop 2024-07-24T21:59:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 shutdown_connections 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db73840 0x55884db7b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 --2- 172.21.15.60:0/1543289523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db72f20 0x55884db7af50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 >> 172.21.15.60:0/1543289523 conn(0x55884db70d50 msgr2=0x55884db77980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 shutdown_connections 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.683+0000 7fd46802a740 1 -- 172.21.15.60:0/1543289523 wait complete. 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 -- start start 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 Processor -- start 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db83030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db83570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.684+0000 7fd46802a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b2ed0 con 0x55884db7f7d0 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd46802a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b4340 con 0x55884db73840 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd46802a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601b35a0 con 0x55884db72f20 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:38740/0 (socket says 172.21.15.60:38740) 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2002645144 learned_addr learned my addr 172.21.15.60:0/2002645144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45effd640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db83030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd46802a740 10 client.0 _ll_register_callbacks cb 0x7fd4580aacf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45ffff640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db83570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2002645144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 msgr2=0x55884db83030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db83030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2002645144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 msgr2=0x55884db83570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db83570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 -- 172.21.15.60:0/2002645144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884db83b30 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd45f7fe640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd454009d20 tx=0x7fd454011420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c8b921aac559b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd43effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45400bad0 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fd454025880 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.685+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45400bd80 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd454025a80 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 10 client.27523 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55884dbfa730 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45400bd80 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd45402faa0 con 0x55884db72f20 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 10 client.27523 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 10 client.27523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 20 client.27523 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884dbfac50 0x55884dbfd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 4 client.27523 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T21:59:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.686+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55884dae83e0 con 0x55884dbfac50 2024-07-24T21:59:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.687+0000 7fd45effd640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884dbfac50 0x55884dbfd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.687+0000 7fd45effd640 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884dbfac50 0x55884dbfd070 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75d53eec2f0595d server_cookie=fae0633ed7a9c0af in_seq=0 out_seq=0 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.687+0000 7fd43effd640 10 client.27523 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd43effd640 1 -- 172.21.15.60:0/2002645144 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55884dae83e0 con 0x55884dbfac50 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd43effd640 10 client.27523 handle_command_reply: tid=0 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 client.27523 shutdown 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 msgr2=0x55884db87b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd454009d20 tx=0x7fd454011420 comp rx=0 tx=0).stop 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 shutdown_connections 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55884dbfac50 0x55884dbfd070 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55884db7f7d0 0x55884db83570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd43effd640 0 client.27523 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55884db73840 0x55884db83030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 --2- 172.21.15.60:0/2002645144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55884db72f20 0x55884db87b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 >> 172.21.15.60:0/2002645144 conn(0x55884db70d50 msgr2=0x55884db7af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 shutdown_connections 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.688+0000 7fd46802a740 1 -- 172.21.15.60:0/2002645144 wait complete. 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd46802a740 20 client.27523 trim_cache size 0 max 16384 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T21:59:43.694 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "23ab1024-0bdd-4ae1-bdda-e0063126a57b", 2024-07-24T21:59:43.695 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T21:59:43.695 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:59:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 msgr2=0x7fd4480b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 0x7fd4480b6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd450005ab0 tx=0x7fd450000bc0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 msgr2=0x7fd4601019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd45800eb10 tx=0x7fd45800c2d0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 shutdown_connections 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fd4480b3dc0 0x7fd4480b6280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46010cae0 0x7fd4601019d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460103fa0 0x7fd460101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 --2- 172.21.15.60:0/545135760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd460103680 0x7fd460102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 >> 172.21.15.60:0/545135760 conn(0x7fd4600fd120 msgr2=0x7fd4600fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 shutdown_connections 2024-07-24T21:59:43.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:43.689+0000 7fd465c79640 1 -- 172.21.15.60:0/545135760 wait complete. 2024-07-24T21:59:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:43 smithi060 ceph-mon[33000]: pgmap v783: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-07-24T21:59:43.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:43 smithi060 ceph-mon[33000]: osdmap e103: 12 total, 12 up, 12 in 2024-07-24T21:59:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:43 smithi123 ceph-mon[37143]: pgmap v783: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-07-24T21:59:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:43 smithi123 ceph-mon[37143]: osdmap e103: 12 total, 12 up, 12 in 2024-07-24T21:59:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:43 smithi086 ceph-mon[37128]: pgmap v783: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-07-24T21:59:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:43 smithi086 ceph-mon[37128]: osdmap e103: 12 total, 12 up, 12 in 2024-07-24T21:59:44.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T21:59:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:44.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:44 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:59:44.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:44 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:59:45.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:44 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:59:45.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:44 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:59:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:44 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:59:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:44 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: pgmap v785: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-07-24T21:59:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:45 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T21:59:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T21:59:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: pgmap v785: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-07-24T21:59:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:45 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T21:59:46.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T21:59:46.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.065 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: pgmap v785: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-07-24T21:59:46.065 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:45 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T21:59:46.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 -- 172.21.15.60:0/968203758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1027b0 msgr2=0x7fc78c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 --2- 172.21.15.60:0/968203758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1027b0 0x7fc78c102b90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc780009900 tx=0x7fc78002a410 comp rx=0 tx=0).stop 2024-07-24T21:59:46.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 -- 172.21.15.60:0/968203758 shutdown_connections 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 --2- 172.21.15.60:0/968203758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c108b40 0x7fc78c10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 --2- 172.21.15.60:0/968203758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c1081c0 0x7fc78c108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 --2- 172.21.15.60:0/968203758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1027b0 0x7fc78c102b90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.780+0000 7fc79089b640 1 -- 172.21.15.60:0/968203758 >> 172.21.15.60:0/968203758 conn(0x7fc78c0fe200 msgr2=0x7fc78c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.781+0000 7fc79089b640 1 -- 172.21.15.60:0/968203758 shutdown_connections 2024-07-24T21:59:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.781+0000 7fc79089b640 1 -- 172.21.15.60:0/968203758 wait complete. 2024-07-24T21:59:46.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.782+0000 7fc79089b640 1 Processor -- start 2024-07-24T21:59:46.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.782+0000 7fc79089b640 1 -- start start 2024-07-24T21:59:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c1027b0 0x7fc78c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 0x7fc78c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c112790 con 0x7fc78c1081c0 2024-07-24T21:59:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c112610 con 0x7fc78c108b40 2024-07-24T21:59:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc79089b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c112910 con 0x7fc78c1027b0 2024-07-24T21:59:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.783+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c1027b0 0x7fc78c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc78ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 0x7fc78c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33118/0 (socket says 172.21.15.60:33118) 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 -- 172.21.15.60:0/4145213619 learned_addr learned my addr 172.21.15.60:0/4145213619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 -- 172.21.15.60:0/4145213619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c1027b0 msgr2=0x7fc78c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c1027b0 0x7fc78c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 -- 172.21.15.60:0/4145213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 msgr2=0x7fc78c19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 0x7fc78c19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc789d9b640 1 -- 172.21.15.60:0/4145213619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c19ecd0 con 0x7fc78c1081c0 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.784+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 0x7fc78c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.785+0000 7fc789d9b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00e9f0 tx=0x7fc77c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.785+0000 7fc77b7fe640 1 -- 172.21.15.60:0/4145213619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00cd80 con 0x7fc78c1081c0 2024-07-24T21:59:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.785+0000 7fc77b7fe640 1 -- 172.21.15.60:0/4145213619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c0108e0 con 0x7fc78c1081c0 2024-07-24T21:59:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.786+0000 7fc77b7fe640 1 -- 172.21.15.60:0/4145213619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c018980 con 0x7fc78c1081c0 2024-07-24T21:59:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.786+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c19efc0 con 0x7fc78c1081c0 2024-07-24T21:59:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.786+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c074d70 con 0x7fc78c1081c0 2024-07-24T21:59:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.790+0000 7fc792c4c740 1 Processor -- start 2024-07-24T21:59:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.790+0000 7fc792c4c740 1 -- start start 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc78ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc78ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33128/0 (socket says 172.21.15.60:33128) 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc78ad9d640 1 -- 172.21.15.60:0/1622790910 learned_addr learned my addr 172.21.15.60:0/1622790910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.791+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffbdf60 0x55723ffc0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.792+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55723ff28990 con 0x55723ffb16b0 2024-07-24T21:59:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.792+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55723ff17e90 con 0x55723ffb1fd0 2024-07-24T21:59:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.792+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55723ff1bed0 con 0x55723ffbdf60 2024-07-24T21:59:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.794+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 -- 172.21.15.60:0/1622790910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffbdf60 msgr2=0x55723ffc0350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffbdf60 0x55723ffc0350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 -- 172.21.15.60:0/1622790910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 msgr2=0x55723ffb1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 -- 172.21.15.60:0/1622790910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723fea1300 con 0x55723ffb1fd0 2024-07-24T21:59:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbda20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc78002aa00 tx=0x7fc780005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1455e3b1d4820ab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.795+0000 7fc77b7fe640 1 -- 172.21.15.60:0/4145213619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc77c010430 con 0x7fc78c1081c0 2024-07-24T21:59:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.796+0000 7fc778ff9640 1 -- 172.21.15.60:0/1622790910 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780035040 con 0x55723ffb1fd0 2024-07-24T21:59:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.796+0000 7fc778ff9640 1 -- 172.21.15.60:0/1622790910 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800054f0 con 0x55723ffb1fd0 2024-07-24T21:59:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.796+0000 7fc778ff9640 1 -- 172.21.15.60:0/1622790910 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800035f0 con 0x55723ffb1fd0 2024-07-24T21:59:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.796+0000 7fc77b7fe640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 0x7fc7740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.796+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 0x7fc7740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.797+0000 7fc77b7fe640 1 -- 172.21.15.60:0/4145213619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fc77c0e4440 con 0x7fc78c1081c0 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.797+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 msgr2=0x55723ffbda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.797+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbda20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc78002aa00 tx=0x7fc780005c00 comp rx=0 tx=0).stop 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.797+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 0x7fc7740b6200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc784005c40 tx=0x7fc784009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.798+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 shutdown_connections 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.798+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffbdf60 0x55723ffc0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.798+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbda20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.798+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1622790910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.798+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 >> 172.21.15.60:0/1622790910 conn(0x55723ffaf4e0 msgr2=0x55723ffb0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.799+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 shutdown_connections 2024-07-24T21:59:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.799+0000 7fc792c4c740 1 -- 172.21.15.60:0/1622790910 wait complete. 2024-07-24T21:59:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.800+0000 7fc792c4c740 1 -- start start 2024-07-24T21:59:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.800+0000 7fc792c4c740 1 Processor -- start 2024-07-24T21:59:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.801+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.801+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.801+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 0x55723ffbbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.802+0000 7fc792c4c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55723fd7b700 con 0x55723ffbdf60 2024-07-24T21:59:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.802+0000 7fc792c4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55723feba5d0 con 0x55723ffb1fd0 2024-07-24T21:59:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.802+0000 7fc792c4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55723ffc0bd0 con 0x55723ffb16b0 2024-07-24T21:59:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.802+0000 7fc792c4c740 10 client.0 _ll_register_callbacks cb 0x55723feda940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39186/0 (socket says 172.21.15.60:39186) 2024-07-24T21:59:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78a59c640 1 -- 172.21.15.60:0/1824648023 learned_addr learned my addr 172.21.15.60:0/1824648023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 0x55723ffbbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78a59c640 1 -- 172.21.15.60:0/1824648023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 msgr2=0x55723ffbb560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.803+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbb560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.804+0000 7fc78a59c640 1 -- 172.21.15.60:0/1824648023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 msgr2=0x55723ffbbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.804+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 0x55723ffbbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.804+0000 7fc78a59c640 1 -- 172.21.15.60:0/1824648023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723fead660 con 0x55723ffb16b0 2024-07-24T21:59:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.804+0000 7fc78ad9d640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 0x55723ffbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.805+0000 7fc771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:59:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.805+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc780004be0 tx=0x7fc7800047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3d16ba164b1597 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.807+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780053060 con 0x55723ffb16b0 2024-07-24T21:59:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.807+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fc780004c80 con 0x55723ffb16b0 2024-07-24T21:59:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.808+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7800054f0 con 0x55723ffb16b0 2024-07-24T21:59:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.808+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc780035050 con 0x55723ffb16b0 2024-07-24T21:59:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.808+0000 7fc792c4c740 10 client.27532 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.808+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55723fc78650 con 0x55723ffb16b0 2024-07-24T21:59:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.809+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800054f0 con 0x55723ffb16b0 2024-07-24T21:59:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc780003990 con 0x55723ffb16b0 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 10 client.27532 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 20 client.27532 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55723ffcda10 0x55723ffcfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 4 client.27532 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.810+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55723ff2a8e0 con 0x55723ffcda10 2024-07-24T21:59:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.811+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55723ffcda10 0x55723ffcfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.811+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55723ffcda10 0x55723ffcfe00 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6ecc1bb67b6bfd server_cookie=ce6df5137035365f in_seq=0 out_seq=0 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.811+0000 7fc771ffb640 10 client.27532 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc771ffb640 1 -- 172.21.15.60:0/1824648023 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55723ff2a8e0 con 0x55723ffcda10 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc771ffb640 10 client.27532 handle_command_reply: tid=0 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 client.27532 shutdown 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 msgr2=0x55723ffbb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc780004be0 tx=0x7fc7800047e0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 shutdown_connections 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55723ffcda10 0x55723ffcfe00 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffbdf60 0x55723ffbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc771ffb640 0 client.27532 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffb1fd0 0x55723ffbb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1824648023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb16b0 0x55723ffbb020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.812+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 >> 172.21.15.60:0/1824648023 conn(0x55723ffaf4e0 msgr2=0x55723ffb5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.813+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 shutdown_connections 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.813+0000 7fc792c4c740 1 -- 172.21.15.60:0/1824648023 wait complete. 2024-07-24T21:59:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.813+0000 7fc792c4c740 20 client.27532 trim_cache size 0 max 16384 2024-07-24T21:59:46.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.840+0000 7fc792c4c740 1 Processor -- start 2024-07-24T21:59:46.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.840+0000 7fc792c4c740 1 -- start start 2024-07-24T21:59:46.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffb8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b2da0 con 0x55723ffb16b0 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1b05b0 con 0x55723ffbdf60 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc792c4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1ae3e0 con 0x55723ffb1fd0 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc789d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffb8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc789d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39196/0 (socket says 172.21.15.60:39196) 2024-07-24T21:59:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc789d9b640 1 -- 172.21.15.60:0/4005023706 learned_addr learned my addr 172.21.15.60:0/4005023706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 -- 172.21.15.60:0/4005023706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 msgr2=0x55723ffc9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 -- 172.21.15.60:0/4005023706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 msgr2=0x55723ffb8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffb8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 -- 172.21.15.60:0/4005023706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723ffaae70 con 0x55723ffb16b0 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc789d9b640 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc78a59c640 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb5800 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc780003f00 tx=0x7fc780005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46675ab8bd4df1a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.841+0000 7fc773fff640 1 -- 172.21.15.60:0/4005023706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780035050 con 0x55723ffb16b0 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc773fff640 1 -- 172.21.15.60:0/4005023706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800054f0 con 0x55723ffb16b0 2024-07-24T21:59:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc773fff640 1 -- 172.21.15.60:0/4005023706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800317a0 con 0x55723ffb16b0 2024-07-24T21:59:46.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 -- 172.21.15.60:0/4005023706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 msgr2=0x55723ffb5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb5800 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc780003f00 tx=0x7fc780005a20 comp rx=0 tx=0).stop 2024-07-24T21:59:46.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 -- 172.21.15.60:0/4005023706 shutdown_connections 2024-07-24T21:59:46.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffb8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffc9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 --2- 172.21.15.60:0/4005023706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffb5800 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 -- 172.21.15.60:0/4005023706 >> 172.21.15.60:0/4005023706 conn(0x55723ffaf4e0 msgr2=0x55723ffc8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 -- 172.21.15.60:0/4005023706 shutdown_connections 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.842+0000 7fc792c4c740 1 -- 172.21.15.60:0/4005023706 wait complete. 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 -- start start 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 Processor -- start 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffd1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffd1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc78a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33152/0 (socket says 172.21.15.60:33152) 2024-07-24T21:59:46.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc78a59c640 1 -- 172.21.15.60:0/1034311406 learned_addr learned my addr 172.21.15.60:0/1034311406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffd1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1aea70 con 0x55723ffb16b0 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.843+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1afee0 con 0x55723ffbdf60 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1af140 con 0x55723ffb1fd0 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc792c4c740 10 client.0 _ll_register_callbacks cb 0x7fc784009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 -- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 msgr2=0x55723ffd1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffd1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 -- 172.21.15.60:0/1034311406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 msgr2=0x55723ffd1740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffd1740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572400373d0 con 0x55723ffb16b0 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffd1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:46.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc78a59c640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc780005ab0 tx=0x7fc7800314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5c7ee247ad6360 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780035070 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fc78004a040 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.844+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7800054f0 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc780009520 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 10 client.18222 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5572400386a0 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800054f0 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc780031ad0 con 0x55723ffb16b0 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 10 client.18222 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 10 client.18222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 20 client.18222 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:46.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557240038d90 0x55724003b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 4 client.18222 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.845+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55723ff26380 con 0x557240038d90 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557240038d90 0x55724003b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc789d9b640 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557240038d90 0x55724003b1b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bef8fb229ca4c98 server_cookie=1c5f678d8591b628 in_seq=0 out_seq=0 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc771ffb640 10 client.18222 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc771ffb640 1 -- 172.21.15.60:0/1034311406 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55723ff26380 con 0x557240038d90 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc771ffb640 10 client.18222 handle_command_reply: tid=0 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc792c4c740 1 client.18222 shutdown 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 msgr2=0x55723ffd0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc780005ab0 tx=0x7fc7800314a0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.846+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 shutdown_connections 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557240038d90 0x55724003b1b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55723ffbdf60 0x55723ffd1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc771ffb640 0 client.18222 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55723ffb1fd0 0x55723ffd1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 --2- 172.21.15.60:0/1034311406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723ffb16b0 0x55723ffd0cc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 >> 172.21.15.60:0/1034311406 conn(0x55723ffaf4e0 msgr2=0x55723ffc9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 shutdown_connections 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 1 -- 172.21.15.60:0/1034311406 wait complete. 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.847+0000 7fc792c4c740 20 client.18222 trim_cache size 0 max 16384 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T21:59:46.857 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: "23ab1024-0bdd-4ae1-bdda-e0063126a57b": { 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "23ab1024-0bdd-4ae1-bdda-e0063126a57b", 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:46.858 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:59:46.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 msgr2=0x7fc7740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 0x7fc7740b6200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc784005c40 tx=0x7fc784009500 comp rx=0 tx=0).stop 2024-07-24T21:59:46.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 msgr2=0x7fc78c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:46.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00e9f0 tx=0x7fc77c00eeb0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 shutdown_connections 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc7740b3d40 0x7fc7740b6200 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc78c108b40 0x7fc78c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c1081c0 0x7fc78c199160 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 --2- 172.21.15.60:0/4145213619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc78c1027b0 0x7fc78c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 >> 172.21.15.60:0/4145213619 conn(0x7fc78c0fe200 msgr2=0x7fc78c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 shutdown_connections 2024-07-24T21:59:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:46.848+0000 7fc79089b640 1 -- 172.21.15.60:0/4145213619 wait complete. 2024-07-24T21:59:47.679 INFO:tasks.cephfs.filesystem:scrub status for tag:23ab1024-0bdd-4ae1-bdda-e0063126a57b - {'path': '/', 'tag': '23ab1024-0bdd-4ae1-bdda-e0063126a57b', 'options': 'recursive,force'} 2024-07-24T21:59:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:47 smithi123 ceph-mon[37143]: pgmap v786: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-07-24T21:59:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:47 smithi086 ceph-mon[37128]: pgmap v786: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-07-24T21:59:48.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:47 smithi060 ceph-mon[33000]: pgmap v786: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2024-07-24T21:59:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 -- 172.21.15.60:0/866788925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101aa0 msgr2=0x7ff330105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 --2- 172.21.15.60:0/866788925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101aa0 0x7ff330105f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff320009900 tx=0x7ff32002a480 comp rx=0 tx=0).stop 2024-07-24T21:59:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 -- 172.21.15.60:0/866788925 shutdown_connections 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 --2- 172.21.15.60:0/866788925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 0x7ff330107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 --2- 172.21.15.60:0/866788925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101aa0 0x7ff330105f60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 --2- 172.21.15.60:0/866788925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101180 0x7ff330101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 -- 172.21.15.60:0/866788925 >> 172.21.15.60:0/866788925 conn(0x7ff330076860 msgr2=0x7ff330076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 -- 172.21.15.60:0/866788925 shutdown_connections 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.654+0000 7ff3354c6640 1 -- 172.21.15.60:0/866788925 wait complete. 2024-07-24T21:59:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 Processor -- start 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 -- start start 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 0x7ff33019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 0x7ff3301a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff330118cd0 con 0x7ff330101aa0 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff330118b50 con 0x7ff330101180 2024-07-24T21:59:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff3354c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff330118e50 con 0x7ff330106710 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff32e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.655+0000 7ff32effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 0x7ff33019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33174/0 (socket says 172.21.15.60:33174) 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 -- 172.21.15.60:0/844846334 learned_addr learned my addr 172.21.15.60:0/844846334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 0x7ff3301a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 -- 172.21.15.60:0/844846334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 msgr2=0x7ff3301a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 0x7ff3301a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 -- 172.21.15.60:0/844846334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 msgr2=0x7ff33019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 0x7ff33019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 -- 172.21.15.60:0/844846334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3301a5290 con 0x7ff330101aa0 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32effd640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 0x7ff33019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff320031440 tx=0x7ff320003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff327fff640 1 -- 172.21.15.60:0/844846334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32003f070 con 0x7ff330101aa0 2024-07-24T21:59:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff327fff640 1 -- 172.21.15.60:0/844846334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff320031b50 con 0x7ff330101aa0 2024-07-24T21:59:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3301a5520 con 0x7ff330101aa0 2024-07-24T21:59:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff327fff640 1 -- 172.21.15.60:0/844846334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320035030 con 0x7ff330101aa0 2024-07-24T21:59:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.656+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3301abc40 con 0x7ff330101aa0 2024-07-24T21:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.660+0000 7ff33821b740 1 Processor -- start 2024-07-24T21:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.660+0000 7ff33821b740 1 -- start start 2024-07-24T21:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56444d01c280 con 0x56444d0b3170 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56444d00b780 con 0x56444d0a68c0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff33821b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56444d00f7c0 con 0x56444d0a71e0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff327fff640 1 -- 172.21.15.60:0/844846334 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff320042070 con 0x7ff330101aa0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39060/0 (socket says 172.21.15.60:39060) 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 -- 172.21.15.60:0/3033419408 learned_addr learned my addr 172.21.15.60:0/3033419408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32effd640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 -- 172.21.15.60:0/3033419408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 msgr2=0x56444d0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 -- 172.21.15.60:0/3033419408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 msgr2=0x56444d0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32f7fe640 1 -- 172.21.15.60:0/3033419408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444cd6f790 con 0x56444d0a68c0 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.661+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff328009fd0 tx=0x7ff3280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08513bfdf965be1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff3257fa640 1 -- 172.21.15.60:0/3033419408 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328004d20 con 0x56444d0a68c0 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff3257fa640 1 -- 172.21.15.60:0/3033419408 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff328004050 con 0x56444d0a68c0 2024-07-24T21:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff3257fa640 1 -- 172.21.15.60:0/3033419408 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328005280 con 0x56444d0a68c0 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff327fff640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 0x7ff31c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.662+0000 7ff32effd640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 0x7ff31c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff327fff640 1 -- 172.21.15.60:0/844846334 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7ff320104670 con 0x7ff330101aa0 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 -- 172.21.15.60:0/3033419408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 msgr2=0x56444d0a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff328009fd0 tx=0x7ff3280118d0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff32effd640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 0x7ff31c0b62b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x56444d0a9a40 tx=0x7ff318005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 -- 172.21.15.60:0/3033419408 shutdown_connections 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 --2- 172.21.15.60:0/3033419408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0a6ca0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 -- 172.21.15.60:0/3033419408 >> 172.21.15.60:0/3033419408 conn(0x56444d0a46f0 msgr2=0x56444d0a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 -- 172.21.15.60:0/3033419408 shutdown_connections 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.663+0000 7ff33821b740 1 -- 172.21.15.60:0/3033419408 wait complete. 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 -- start start 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 Processor -- start 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56444cf45ab0 con 0x56444d0b3170 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56444cf939e0 con 0x56444d0a68c0 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56444d0b5e50 con 0x56444d0a71e0 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff33821b740 10 client.0 _ll_register_callbacks cb 0x56444cd1ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff32f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff32f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33186/0 (socket says 172.21.15.60:33186) 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff32f7fe640 1 -- 172.21.15.60:0/240702135 learned_addr learned my addr 172.21.15.60:0/240702135 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff32effd640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.664+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 -- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 msgr2=0x56444d0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 -- 172.21.15.60:0/240702135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 msgr2=0x56444d0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 -- 172.21.15.60:0/240702135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56444cde8620 con 0x56444d0b3170 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32effd640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff328005c10 tx=0x7ff328004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b137add6b82929b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff3167fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328028020 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56444cfbace0 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7ff328004260 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff328033050 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff33821b740 10 client.18234 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56444d0c15c0 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.665+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56444cfbace0 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff328005660 con 0x56444d0b3170 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 10 client.18234 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 20 client.18234 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 4 client.18234 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56444d01e1d0 con 0x56444d0c2a30 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff32effd640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff32effd640 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0c4e20 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af56cf7d9b47f3d2 server_cookie=26473be984434ac in_seq=0 out_seq=0 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.666+0000 7ff3167fc640 10 client.18234 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff3167fc640 1 -- 172.21.15.60:0/240702135 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56444d01e1d0 con 0x56444d0c2a30 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff3167fc640 10 client.18234 handle_command_reply: tid=0 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 client.18234 shutdown 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 msgr2=0x56444d0b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff328005c10 tx=0x7ff328004c70 comp rx=0 tx=0).stop 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 shutdown_connections 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0c4e20 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0b3170 0x56444d0b0210 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff3167fc640 0 client.18234 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 --2- 172.21.15.60:0/240702135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a68c0 0x56444d0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 >> 172.21.15.60:0/240702135 conn(0x56444d0a46f0 msgr2=0x56444d0aa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 shutdown_connections 2024-07-24T21:59:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 1 -- 172.21.15.60:0/240702135 wait complete. 2024-07-24T21:59:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.667+0000 7ff33821b740 20 client.18234 trim_cache size 0 max 16384 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 Processor -- start 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 -- start start 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a71e0 0x56444d0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 0x56444d0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b7140 con 0x56444d0a68c0 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b4950 con 0x56444d0a71e0 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff33821b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b27c0 con 0x56444d0b3170 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33196/0 (socket says 172.21.15.60:33196) 2024-07-24T21:59:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 -- 172.21.15.60:0/3203360079 learned_addr learned my addr 172.21.15.60:0/3203360079 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a71e0 0x56444d0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 0x56444d0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 -- 172.21.15.60:0/3203360079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 msgr2=0x56444d0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 0x56444d0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 -- 172.21.15.60:0/3203360079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a71e0 msgr2=0x56444d0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.684+0000 7ff32effd640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a71e0 0x56444d0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff32effd640 1 -- 172.21.15.60:0/3203360079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444d0a7010 con 0x56444d0a68c0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 0x56444d0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff32effd640 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff31800ff90 tx=0x7ff31801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f3c55196711cd2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff324ff9640 1 -- 172.21.15.60:0/3203360079 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31802e070 con 0x56444d0a68c0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff324ff9640 1 -- 172.21.15.60:0/3203360079 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff318004020 con 0x56444d0a68c0 2024-07-24T21:59:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff324ff9640 1 -- 172.21.15.60:0/3203360079 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318005620 con 0x56444d0a68c0 2024-07-24T21:59:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff33821b740 1 -- 172.21.15.60:0/3203360079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 msgr2=0x56444d0ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.685+0000 7ff33821b740 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff31800ff90 tx=0x7ff31801b000 comp rx=0 tx=0).stop 2024-07-24T21:59:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- 172.21.15.60:0/3203360079 shutdown_connections 2024-07-24T21:59:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0b3170 0x56444d0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0a71e0 0x56444d0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- 172.21.15.60:0/3203360079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ab2f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- 172.21.15.60:0/3203360079 >> 172.21.15.60:0/3203360079 conn(0x56444d0a46f0 msgr2=0x56444d0aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- 172.21.15.60:0/3203360079 shutdown_connections 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- 172.21.15.60:0/3203360079 wait complete. 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- start start 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 Processor -- start 2024-07-24T21:59:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0b3170 0x56444d0ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b2e10 con 0x56444d0a68c0 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b4280 con 0x56444d0b3170 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301b34e0 con 0x56444d0a71e0 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff33821b740 10 client.0 _ll_register_callbacks cb 0x7ff318009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff32effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff32effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33210/0 (socket says 172.21.15.60:33210) 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff32effd640 1 -- 172.21.15.60:0/4185071039 learned_addr learned my addr 172.21.15.60:0/4185071039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.686+0000 7ff32f7fe640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0b3170 0x56444d0ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 -- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 msgr2=0x56444d0abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 -- 172.21.15.60:0/4185071039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0b3170 msgr2=0x56444d0ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0b3170 0x56444d0ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 -- 172.21.15.60:0/4185071039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56444d0acbd0 con 0x56444d0a68c0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff32effd640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff318000c00 tx=0x7ff31800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114231bc5c962958 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31802e040 con 0x56444d0a68c0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7ff3180041d0 con 0x56444d0a68c0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff318004020 con 0x56444d0a68c0 2024-07-24T21:59:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff3180043d0 con 0x56444d0a68c0 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff33821b740 10 client.18240 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56444d0bb160 con 0x56444d0a68c0 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff318004020 con 0x56444d0a68c0 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7ff318004b00 con 0x56444d0a68c0 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.687+0000 7ff33821b740 10 client.18240 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff33821b740 10 client.18240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff33821b740 20 client.18240 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff33821b740 4 client.18240 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56444d019c70 con 0x56444d0c2a30 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff32e7fc640 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0bb910 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c464db8062d77f server_cookie=39cd6b2b5338fbaa in_seq=0 out_seq=0 2024-07-24T21:59:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.688+0000 7ff3167fc640 10 client.18240 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.748+0000 7ff3167fc640 1 -- 172.21.15.60:0/4185071039 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56444d019c70 con 0x56444d0c2a30 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.748+0000 7ff3167fc640 10 client.18240 handle_command_reply: tid=0 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.748+0000 7ff33821b740 1 client.18240 shutdown 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 msgr2=0x56444d0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff318000c00 tx=0x7ff31800fc50 comp rx=0 tx=0).stop 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 shutdown_connections 2024-07-24T21:59:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56444d0c2a30 0x56444d0bb910 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56444d0b3170 0x56444d0ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56444d0a71e0 0x56444d0abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3167fc640 0 client.18240 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 --2- 172.21.15.60:0/4185071039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56444d0a68c0 0x56444d0ad6d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 >> 172.21.15.60:0/4185071039 conn(0x56444d0a46f0 msgr2=0x56444d0ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 shutdown_connections 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 1 -- 172.21.15.60:0/4185071039 wait complete. 2024-07-24T21:59:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff33821b740 20 client.18240 trim_cache size 0 max 16384 2024-07-24T21:59:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T21:59:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T21:59:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T21:59:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 msgr2=0x7ff31c0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 0x7ff31c0b62b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x56444d0a9a40 tx=0x7ff318005f90 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 msgr2=0x7ff33019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff320031440 tx=0x7ff320003b90 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 shutdown_connections 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff31c0b3df0 0x7ff31c0b62b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff330106710 0x7ff3301a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff330101aa0 0x7ff33019f730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.749+0000 7ff3354c6640 1 --2- 172.21.15.60:0/844846334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff330101180 0x7ff33019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.750+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 >> 172.21.15.60:0/844846334 conn(0x7ff330076860 msgr2=0x7ff330103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.750+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 shutdown_connections 2024-07-24T21:59:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T21:59:48.750+0000 7ff3354c6640 1 -- 172.21.15.60:0/844846334 wait complete. 2024-07-24T21:59:49.282 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.3 2024-07-24T21:59:49.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T21:59:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:49.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:49 smithi060 ceph-mon[33000]: pgmap v787: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 731 op/s 2024-07-24T21:59:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:49 smithi123 ceph-mon[37143]: pgmap v787: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 731 op/s 2024-07-24T21:59:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:49 smithi086 ceph-mon[37128]: pgmap v787: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 731 op/s 2024-07-24T21:59:51.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 -- 172.21.15.60:0/2013035882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08010ee20 msgr2=0x7fc080111210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 --2- 172.21.15.60:0/2013035882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08010ee20 0x7fc080111210 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc078009ca0 tx=0x7fc07802a950 comp rx=0 tx=0).stop 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 -- 172.21.15.60:0/2013035882 shutdown_connections 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 --2- 172.21.15.60:0/2013035882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08010ee20 0x7fc080111210 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 --2- 172.21.15.60:0/2013035882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc080101b90 0x7fc08010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 --2- 172.21.15.60:0/2013035882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101270 0x7fc080101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 -- 172.21.15.60:0/2013035882 >> 172.21.15.60:0/2013035882 conn(0x7fc0800fd0c0 msgr2=0x7fc0800ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 -- 172.21.15.60:0/2013035882 shutdown_connections 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 -- 172.21.15.60:0/2013035882 wait complete. 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.390+0000 7fc085b95640 1 Processor -- start 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 -- start start 2024-07-24T21:59:51.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc080101270 0x7fc08019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101b90 0x7fc08019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc080114930 con 0x7fc080101270 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0801147b0 con 0x7fc08010ee20 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc085b95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc080114ab0 con 0x7fc080101b90 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc080101270 0x7fc08019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39120/0 (socket says 172.21.15.60:39120) 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 -- 172.21.15.60:0/3549188472 learned_addr learned my addr 172.21.15.60:0/3549188472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07effd640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101b90 0x7fc08019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 -- 172.21.15.60:0/3549188472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101b90 msgr2=0x7fc08019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101b90 0x7fc08019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 -- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc080101270 msgr2=0x7fc08019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc080101270 0x7fc08019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.391+0000 7fc07ffff640 1 -- 172.21.15.60:0/3549188472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801a0f10 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc07ffff640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc078004af0 tx=0x7fc07802f5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078034040 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0801a11a0 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07802fe80 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0780046a0 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.392+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a78c0 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.393+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080101650 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.395+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc078037020 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.395+0000 7fc07cff9640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 0x7fc0680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.396+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7fc078104f70 con 0x7fc08010ee20 2024-07-24T21:59:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.396+0000 7fc07f7fe640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 0x7fc0680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.396+0000 7fc07f7fe640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 0x7fc0680b6200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc070005d00 tx=0x7fc07000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:51.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.402+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc0780825e0 con 0x7fc08010ee20 2024-07-24T21:59:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.663+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08019bf90 con 0x7fc08010ee20 2024-07-24T21:59:51.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.664+0000 7fc07cff9640 1 -- 172.21.15.60:0/3549188472 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fc0780cc300 con 0x7fc08010ee20 2024-07-24T21:59:51.666 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T21:59:51.666 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T21:59:51.666 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T21:59:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 msgr2=0x7fc0680b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 0x7fc0680b6200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc070005d00 tx=0x7fc07000b410 comp rx=0 tx=0).stop 2024-07-24T21:59:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 msgr2=0x7fc0801a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc078004af0 tx=0x7fc07802f5d0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 shutdown_connections 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0680b3d40 0x7fc0680b6200 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08010ee20 0x7fc0801a0740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc080101b90 0x7fc08019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 --2- 172.21.15.60:0/3549188472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc080101270 0x7fc08019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 >> 172.21.15.60:0/3549188472 conn(0x7fc0800fd0c0 msgr2=0x7fc0801101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 shutdown_connections 2024-07-24T21:59:51.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:51.666+0000 7fc085b95640 1 -- 172.21.15.60:0/3549188472 wait complete. 2024-07-24T21:59:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:51 smithi123 ceph-mon[37143]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 1.14k op/s 2024-07-24T21:59:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:51 smithi086 ceph-mon[37128]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 1.14k op/s 2024-07-24T21:59:52.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:51 smithi060 ceph-mon[33000]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 1.14k op/s 2024-07-24T21:59:52.603 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 134.3 --await-for 45.0' 2024-07-24T21:59:52.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 134.3 --await-for 45.0 2024-07-24T21:59:52.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:52 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3549188472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:52.925 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T21:59:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:52 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3549188472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:53.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:52 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3549188472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T21:59:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:53 smithi123 ceph-mon[37143]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 1.62k op/s 2024-07-24T21:59:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:53 smithi086 ceph-mon[37128]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 1.62k op/s 2024-07-24T21:59:54.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:53 smithi060 ceph-mon[33000]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 1.62k op/s 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 -- 172.21.15.60:0/2274859535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c810cb00 msgr2=0x7f28c810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 --2- 172.21.15.60:0/2274859535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c810cb00 0x7f28c810ef60 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009ca0 tx=0x7f28c002a8c0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 -- 172.21.15.60:0/2274859535 shutdown_connections 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 --2- 172.21.15.60:0/2274859535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c810cb00 0x7f28c810ef60 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 --2- 172.21.15.60:0/2274859535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 0x7f28c810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 --2- 172.21.15.60:0/2274859535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c81036a0 0x7f28c8103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 -- 172.21.15.60:0/2274859535 >> 172.21.15.60:0/2274859535 conn(0x7f28c80fd100 msgr2=0x7f28c80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:54.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 -- 172.21.15.60:0/2274859535 shutdown_connections 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.726+0000 7f28cf483640 1 -- 172.21.15.60:0/2274859535 wait complete. 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 Processor -- start 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 -- start start 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 0x7f28c8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c810cb00 0x7f28c819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8112760 con 0x7f28c81036a0 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81125e0 con 0x7f28c8103fc0 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cf483640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81128e0 con 0x7f28c810cb00 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33240/0 (socket says 172.21.15.60:33240) 2024-07-24T21:59:54.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cca1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 0x7f28c8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 -- 172.21.15.60:0/2984105574 learned_addr learned my addr 172.21.15.60:0/2984105574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cda20640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c810cb00 0x7f28c819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 -- 172.21.15.60:0/2984105574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c810cb00 msgr2=0x7f28c819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c810cb00 0x7f28c819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 -- 172.21.15.60:0/2984105574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 msgr2=0x7f28c8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 0x7f28c8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.727+0000 7f28cd21f640 1 -- 172.21.15.60:0/2984105574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c819ecf0 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28cd21f640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e9f0 tx=0x7f28c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28be7fc640 1 -- 172.21.15.60:0/2984105574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c400cd80 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28be7fc640 1 -- 172.21.15.60:0/2984105574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c40054d0 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c819efe0 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28be7fc640 1 -- 172.21.15.60:0/2984105574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4010690 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.728+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c81a56a0 con 0x7f28c81036a0 2024-07-24T21:59:54.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28be7fc640 1 -- 172.21.15.60:0/2984105574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f28c4004e90 con 0x7f28c81036a0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 Processor -- start 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 -- start start 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bcfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d454b336a0 con 0x55d454bcd9e0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d454b22ba0 con 0x55d454bbd7a0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28d1834740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d454b26be0 con 0x55d454bbecd0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28cda20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28cca1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bcfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28cda20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39306/0 (socket says 172.21.15.60:39306) 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28cda20640 1 -- 172.21.15.60:0/4194779395 learned_addr learned my addr 172.21.15.60:0/4194779395 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.731+0000 7f28cd21f640 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bbdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 -- 172.21.15.60:0/4194779395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 msgr2=0x55d454bbdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bbdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 -- 172.21.15.60:0/4194779395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 msgr2=0x55d454bcfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bcfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 -- 172.21.15.60:0/4194779395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d454abad60 con 0x55d454bbecd0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28be7fc640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 0x7f28b40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cda20640 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28c0005cc0 tx=0x7f28c0009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976e75e14c44688 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28cd21f640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 0x7f28b40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28b37fe640 1 -- 172.21.15.60:0/4194779395 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0031cf0 con 0x55d454bbecd0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28b37fe640 1 -- 172.21.15.60:0/4194779395 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0036040 con 0x55d454bbecd0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28b37fe640 1 -- 172.21.15.60:0/4194779395 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003e60 con 0x55d454bbecd0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.732+0000 7f28be7fc640 1 -- 172.21.15.60:0/2984105574 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f28c40e44c0 con 0x7f28c81036a0 2024-07-24T21:59:54.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- 172.21.15.60:0/4194779395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 msgr2=0x55d454bbd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28cd21f640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 0x7f28b40b6200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f28c40e1c90 tx=0x7f28c40e7f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28c0005cc0 tx=0x7f28c0009360 comp rx=0 tx=0).stop 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- 172.21.15.60:0/4194779395 shutdown_connections 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bcfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bbdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 --2- 172.21.15.60:0/4194779395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbecd0 0x55d454bbd260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- 172.21.15.60:0/4194779395 >> 172.21.15.60:0/4194779395 conn(0x55d454b87750 msgr2=0x55d454bbb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- 172.21.15.60:0/4194779395 shutdown_connections 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- 172.21.15.60:0/4194779395 wait complete. 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.733+0000 7f28d1834740 1 -- start start 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 Processor -- start 2024-07-24T21:59:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 0x55d454bcae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bcb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d4549867a0 con 0x55d454bbecd0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d454ac5020 con 0x55d454bbd7a0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d454bd0810 con 0x55d454bcd9e0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28d1834740 10 client.0 _ll_register_callbacks cb 0x55d454830090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28cda20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28cda20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39162/0 (socket says 172.21.15.60:39162) 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28cda20640 1 -- 172.21.15.60:0/443189941 learned_addr learned my addr 172.21.15.60:0/443189941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.734+0000 7f28cda20640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bcb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cca1e640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 0x55d454bcae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cda20640 1 -- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 msgr2=0x55d454bcb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cda20640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bcb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cda20640 1 -- 172.21.15.60:0/443189941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 msgr2=0x55d454bcae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cda20640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 0x55d454bcae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cda20640 1 -- 172.21.15.60:0/443189941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d454ac90d0 con 0x55d454bbd7a0 2024-07-24T21:59:54.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.735+0000 7f28cca1e640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 0x55d454bcae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T21:59:54.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28cda20640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f28c0006010 tx=0x7f28c003e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf113527292fb729 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0004d40 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f28c0031af0 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c0036040 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f28c0003970 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28d1834740 10 client.27635 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.741+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d454892b90 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0036040 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f28c0004640 con 0x55d454bbd7a0 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 10 client.27635 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 10 client.27635 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 20 client.27635 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bdd8a0 0x55d454bdfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 4 client.27635 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.742+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d454b355f0 con 0x55d454bdd8a0 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.743+0000 7f28cca1e640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bdd8a0 0x55d454bdfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.743+0000 7f28b0ff9640 10 client.27635 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.743+0000 7f28cca1e640 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bdd8a0 0x55d454bdfc90 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104ef45d9ddecde8 server_cookie=bb9e40d1b6ed0058 in_seq=0 out_seq=0 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.744+0000 7f28b0ff9640 1 -- 172.21.15.60:0/443189941 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d454b355f0 con 0x55d454bdd8a0 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.744+0000 7f28b0ff9640 10 client.27635 handle_command_reply: tid=0 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.744+0000 7f28d1834740 1 client.27635 shutdown 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 msgr2=0x55d454bca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f28c0006010 tx=0x7f28c003e040 comp rx=0 tx=0).stop 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 shutdown_connections 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bdd8a0 0x55d454bdfc90 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bcb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28b0ff9640 0 client.27635 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbecd0 0x55d454bcae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 --2- 172.21.15.60:0/443189941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbd7a0 0x55d454bca920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 >> 172.21.15.60:0/443189941 conn(0x55d454b87750 msgr2=0x55d454bc0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 shutdown_connections 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 1 -- 172.21.15.60:0/443189941 wait complete. 2024-07-24T21:59:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.745+0000 7f28d1834740 20 client.27635 trim_cache size 0 max 16384 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 Processor -- start 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 -- start start 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 0x55d454bcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bc79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81ae9e0 con 0x55d454bcd9e0 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81ac1f0 con 0x55d454bbecd0 2024-07-24T21:59:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28d1834740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81aa060 con 0x55d454bbd7a0 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28cda20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 0x55d454bcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28cda20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 0x55d454bcf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39312/0 (socket says 172.21.15.60:39312) 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28cda20640 1 -- 172.21.15.60:0/1399498189 learned_addr learned my addr 172.21.15.60:0/1399498189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28cca1e640 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.778+0000 7f28cda20640 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bc79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 -- 172.21.15.60:0/1399498189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 msgr2=0x55d454bcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 0x55d454bcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 -- 172.21.15.60:0/1399498189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 msgr2=0x55d454bc79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bc79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 -- 172.21.15.60:0/1399498189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d454bc98a0 con 0x55d454bbecd0 2024-07-24T21:59:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28cca1e640 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bcf9d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55d454be0200 tx=0x7f28b800b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c15fdf8ff52857f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28b2ffd640 1 -- 172.21.15.60:0/1399498189 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b800b740 con 0x55d454bbecd0 2024-07-24T21:59:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28b2ffd640 1 -- 172.21.15.60:0/1399498189 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b80054d0 con 0x55d454bbecd0 2024-07-24T21:59:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.779+0000 7f28b2ffd640 1 -- 172.21.15.60:0/1399498189 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8011440 con 0x55d454bbecd0 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.780+0000 7f28d1834740 1 -- 172.21.15.60:0/1399498189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 msgr2=0x55d454bcf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.780+0000 7f28d1834740 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bcf9d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55d454be0200 tx=0x7f28b800b360 comp rx=0 tx=0).stop 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 -- 172.21.15.60:0/1399498189 shutdown_connections 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bcd9e0 0x55d454bc79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bcf9d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 --2- 172.21.15.60:0/1399498189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bbd7a0 0x55d454bcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 -- 172.21.15.60:0/1399498189 >> 172.21.15.60:0/1399498189 conn(0x55d454b87750 msgr2=0x55d454bbaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 -- 172.21.15.60:0/1399498189 shutdown_connections 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.781+0000 7f28d1834740 1 -- 172.21.15.60:0/1399498189 wait complete. 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.782+0000 7f28d1834740 1 -- start start 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.782+0000 7f28d1834740 1 Processor -- start 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbd7a0 0x55d454bbf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bbfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81aa6b0 con 0x55d454bbd7a0 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81abb20 con 0x55d454bbecd0 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81aad80 con 0x55d454bcd9e0 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28d1834740 10 client.0 _ll_register_callbacks cb 0x7f28b8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28cca1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bbfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28cca1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bbfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39170/0 (socket says 172.21.15.60:39170) 2024-07-24T21:59:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.783+0000 7f28cca1e640 1 -- 172.21.15.60:0/928219742 learned_addr learned my addr 172.21.15.60:0/928219742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T21:59:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.786+0000 7f28cda20640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.786+0000 7f28cda20640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbd7a0 0x55d454bbf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.788+0000 7f28cda20640 1 -- 172.21.15.60:0/928219742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 msgr2=0x55d454bbfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.788+0000 7f28cda20640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bbfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.788+0000 7f28cda20640 1 -- 172.21.15.60:0/928219742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbd7a0 msgr2=0x55d454bbf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T21:59:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.788+0000 7f28cda20640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbd7a0 0x55d454bbf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.788+0000 7f28cda20640 1 -- 172.21.15.60:0/928219742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d454bd6040 con 0x55d454bcd9e0 2024-07-24T21:59:54.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.789+0000 7f28cda20640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bd5b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003c90 tx=0x7f28c0003f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863babcb562adee9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T21:59:54.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.789+0000 7f28b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T21:59:54.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.789+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0060040 con 0x55d454bcd9e0 2024-07-24T21:59:54.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.789+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f28c0031650 con 0x55d454bcd9e0 2024-07-24T21:59:54.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.790+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b80054d0 con 0x55d454bcd9e0 2024-07-24T21:59:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.795+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f28c0031850 con 0x55d454bcd9e0 2024-07-24T21:59:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.795+0000 7f28d1834740 10 client.27580 fetch_fsmap learned FSMap version 44 2024-07-24T21:59:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.795+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d454be4aa0 con 0x55d454bcd9e0 2024-07-24T21:59:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.795+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0036030 con 0x55d454bcd9e0 2024-07-24T21:59:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.795+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f28c0030a00 con 0x55d454bcd9e0 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 10 client.27580 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 10 client.27580 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 20 client.27580 populate_metadata read hostname 'smithi060' 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bc3590 0x55d454bc59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 4 client.27580 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.3, "await_for": 45.0}] 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.3, "await_for": 45.0}) -- 0x7f28b80054d0 con 0x55d454bc3590 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.796+0000 7f28cca1e640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bc3590 0x55d454bc59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.797+0000 7f28cca1e640 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bc3590 0x55d454bc59b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df23bf83cb5f01d server_cookie=8372a79eaaece1b1 in_seq=0 out_seq=0 2024-07-24T21:59:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:54.797+0000 7f28b0ff9640 10 client.27580 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:55 smithi123 ceph-mon[37143]: pgmap v790: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 1.58k op/s 2024-07-24T21:59:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:55 smithi086 ceph-mon[37128]: pgmap v790: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 1.58k op/s 2024-07-24T21:59:56.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:55 smithi060 ceph-mon[33000]: pgmap v790: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 1.58k op/s 2024-07-24T21:59:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:57 smithi123 ceph-mon[37143]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 2.01k op/s 2024-07-24T21:59:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:57 smithi086 ceph-mon[37128]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 2.01k op/s 2024-07-24T21:59:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:57 smithi060 ceph-mon[33000]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 2.01k op/s 2024-07-24T21:59:59.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28b0ff9640 1 -- 172.21.15.60:0/928219742 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f28b80054d0 con 0x55d454bc3590 2024-07-24T21:59:59.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28b0ff9640 10 client.27580 handle_command_reply: tid=0 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 client.27580 shutdown 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 msgr2=0x55d454bd5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bd5b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003c90 tx=0x7f28c0003f20 comp rx=0 tx=0).stop 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 shutdown_connections 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55d454bc3590 0x55d454bc59b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d454bcd9e0 0x55d454bd5b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d454bbecd0 0x55d454bbfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 --2- 172.21.15.60:0/928219742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d454bbd7a0 0x55d454bbf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28b0ff9640 0 client.27580 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 >> 172.21.15.60:0/928219742 conn(0x55d454b87750 msgr2=0x55d454bcf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 shutdown_connections 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 1 -- 172.21.15.60:0/928219742 wait complete. 2024-07-24T21:59:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.525+0000 7f28d1834740 20 client.27580 trim_cache size 0 max 16384 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 38, 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "2e120f66": { 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 38, 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T21:59:59.530 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 134.3, 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:59.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:59.532 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:59.532 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T21:59:59.532 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T21:59:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 msgr2=0x7f28b40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 0x7f28b40b6200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f28c40e1c90 tx=0x7f28c40e7f90 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 msgr2=0x7f28c8198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e9f0 tx=0x7f28c400edf0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 shutdown_connections 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28b40b3d40 0x7f28b40b6200 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c810cb00 0x7f28c819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c8103fc0 0x7f28c8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 --2- 172.21.15.60:0/2984105574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28c81036a0 0x7f28c8198c40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 >> 172.21.15.60:0/2984105574 conn(0x7f28c80fd100 msgr2=0x7f28c80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 shutdown_connections 2024-07-24T21:59:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T21:59:59.526+0000 7f28cf483640 1 -- 172.21.15.60:0/2984105574 wait complete. 2024-07-24T21:59:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 21:59:59 smithi060 ceph-mon[33000]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.06k op/s 2024-07-24T21:59:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 21:59:59 smithi123 ceph-mon[37143]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.06k op/s 2024-07-24T22:00:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 21:59:59 smithi086 ceph-mon[37128]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.06k op/s 2024-07-24T22:00:00.213 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2e120f66', quiesce took 10.9 seconds. Will release after: 3.4000000000000004 2024-07-24T22:00:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:00 smithi123 ceph-mon[37143]: overall HEALTH_OK 2024-07-24T22:00:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:00 smithi086 ceph-mon[37128]: overall HEALTH_OK 2024-07-24T22:00:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:01.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:00 smithi060 ceph-mon[33000]: overall HEALTH_OK 2024-07-24T22:00:01.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 -- 172.21.15.60:0/3911993215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01026b0 msgr2=0x7f26d0102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 --2- 172.21.15.60:0/3911993215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01026b0 0x7f26d0102a90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009ca0 tx=0x7f26c002a8c0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 -- 172.21.15.60:0/3911993215 shutdown_connections 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 --2- 172.21.15.60:0/3911993215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 --2- 172.21.15.60:0/3911993215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01080b0 0x7f26d01084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 --2- 172.21.15.60:0/3911993215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01026b0 0x7f26d0102a90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 -- 172.21.15.60:0/3911993215 >> 172.21.15.60:0/3911993215 conn(0x7f26d00fe0f0 msgr2=0x7f26d0100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 -- 172.21.15.60:0/3911993215 shutdown_connections 2024-07-24T22:00:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.851+0000 7f26d6006640 1 -- 172.21.15.60:0/3911993215 wait complete. 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 Processor -- start 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 -- start start 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01026b0 0x7f26d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01126d0 con 0x7f26d01080b0 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26d0112550 con 0x7f26d0108a30 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26d6006640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26d0112850 con 0x7f26d01026b0 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42466/0 (socket says 172.21.15.60:42466) 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26ceffd640 1 -- 172.21.15.60:0/763691658 learned_addr learned my addr 172.21.15.60:0/763691658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26cffff640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.852+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01026b0 0x7f26d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 -- 172.21.15.60:0/763691658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01026b0 msgr2=0x7f26d0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01026b0 0x7f26d0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 -- 172.21.15.60:0/763691658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 msgr2=0x7f26d019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 -- 172.21.15.60:0/763691658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d019ecf0 con 0x7f26d01080b0 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26cffff640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ceffd640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b4a0 tx=0x7f26c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ccff9640 1 -- 172.21.15.60:0/763691658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4014070 con 0x7f26d01080b0 2024-07-24T22:00:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d019efe0 con 0x7f26d01080b0 2024-07-24T22:00:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ccff9640 1 -- 172.21.15.60:0/763691658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c4010ac0 con 0x7f26d01080b0 2024-07-24T22:00:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26ccff9640 1 -- 172.21.15.60:0/763691658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4017c30 con 0x7f26d01080b0 2024-07-24T22:00:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.853+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d01a5720 con 0x7f26d01080b0 2024-07-24T22:00:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 Processor -- start 2024-07-24T22:00:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26ccff9640 1 -- 172.21.15.60:0/763691658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f26c4017070 con 0x7f26d01080b0 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 -- start start 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13954ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae13967970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ae138ca280 con 0x55ae139551e0 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae138b9780 con 0x55ae139548c0 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26d8d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae138bd7c0 con 0x55ae13965580 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42468/0 (socket says 172.21.15.60:42468) 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.856+0000 7f26cf7fe640 1 -- 172.21.15.60:0/3694886049 learned_addr learned my addr 172.21.15.60:0/3694886049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cffff640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13954ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 -- 172.21.15.60:0/3694886049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 msgr2=0x55ae13967970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26ceffd640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae13967970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae13967970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 -- 172.21.15.60:0/3694886049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 msgr2=0x55ae13954ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13954ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 -- 172.21.15.60:0/3694886049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1361d790 con 0x55ae139551e0 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cffff640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13954ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26ceffd640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae13967970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26ccff9640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 0x7f26b80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f26c002f480 tx=0x7f26c002f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d514216964ee100d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26cffff640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 0x7f26b80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26b5ffb640 1 -- 172.21.15.60:0/3694886049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0031450 con 0x55ae139551e0 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26ccff9640 1 -- 172.21.15.60:0/763691658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f26c40e3650 con 0x7f26d01080b0 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26b5ffb640 1 -- 172.21.15.60:0/3694886049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c0036040 con 0x55ae139551e0 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.857+0000 7f26b5ffb640 1 -- 172.21.15.60:0/3694886049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00043c0 con 0x55ae139551e0 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- 172.21.15.60:0/3694886049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 msgr2=0x55ae13965040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26cffff640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 0x7f26b80b6200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f26c8012560 tx=0x7f26c800b2e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f26c002f480 tx=0x7f26c002f4b0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- 172.21.15.60:0/3694886049 shutdown_connections 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae13967970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae13965040 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 --2- 172.21.15.60:0/3694886049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13954ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- 172.21.15.60:0/3694886049 >> 172.21.15.60:0/3694886049 conn(0x55ae139526f0 msgr2=0x55ae13953890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- 172.21.15.60:0/3694886049 shutdown_connections 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- 172.21.15.60:0/3694886049 wait complete. 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.858+0000 7f26d8d64740 1 -- start start 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 Processor -- start 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13964ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae139551e0 0x55ae139621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ae137f3ab0 con 0x55ae13965580 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae138419e0 con 0x55ae139548c0 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae13968200 con 0x55ae139551e0 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26d8d64740 10 client.0 _ll_register_callbacks cb 0x55ae135c9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42484/0 (socket says 172.21.15.60:42484) 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 -- 172.21.15.60:0/1086963979 learned_addr learned my addr 172.21.15.60:0/1086963979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26ceffd640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae139551e0 0x55ae139621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13964ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 -- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae139551e0 msgr2=0x55ae139621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae139551e0 0x55ae139621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 -- 172.21.15.60:0/1086963979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 msgr2=0x55ae13964ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.859+0000 7f26cffff640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13964ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26cffff640 1 -- 172.21.15.60:0/1086963979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae13696620 con 0x55ae13965580 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26cffff640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f26c8010820 tx=0x7f26c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3249dcb65573236 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c802c060 con 0x55ae13965580 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f26c801d990 con 0x55ae13965580 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae13868ce0 con 0x55ae13965580 2024-07-24T22:00:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26c801db90 con 0x55ae13965580 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26d8d64740 10 client.18276 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ae13973930 con 0x55ae13965580 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ae13868ce0 con 0x55ae13965580 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.860+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f26c802f3e0 con 0x55ae13965580 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 10 client.18276 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 10 client.18276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 20 client.18276 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae13974e60 0x55ae13977270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 4 client.18276 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae138cc1d0 con 0x55ae13974e60 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae13974e60 0x55ae13977270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26aeffd640 10 client.18276 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.861+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae13974e60 0x55ae13977270 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297107c851a8459e server_cookie=2cf18d779937bc71 in_seq=0 out_seq=0 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26aeffd640 1 -- 172.21.15.60:0/1086963979 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae138cc1d0 con 0x55ae13974e60 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26aeffd640 10 client.18276 handle_command_reply: tid=0 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 client.18276 shutdown 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 msgr2=0x55ae13962710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f26c8010820 tx=0x7f26c80046c0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 shutdown_connections 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae13974e60 0x55ae13977270 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae13965580 0x55ae13962710 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae139551e0 0x55ae139621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 --2- 172.21.15.60:0/1086963979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13964ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26aeffd640 0 client.18276 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 >> 172.21.15.60:0/1086963979 conn(0x55ae139526f0 msgr2=0x55ae13958290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.862+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 shutdown_connections 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.863+0000 7f26d8d64740 1 -- 172.21.15.60:0/1086963979 wait complete. 2024-07-24T22:00:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.863+0000 7f26d8d64740 20 client.18276 trim_cache size 0 max 16384 2024-07-24T22:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 Processor -- start 2024-07-24T22:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 -- start start 2024-07-24T22:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13971320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae139610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01ae9e0 con 0x55ae139551e0 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01ac1f0 con 0x55ae139548c0 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26d8d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01aa060 con 0x55ae13965580 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42494/0 (socket says 172.21.15.60:42494) 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26ceffd640 1 -- 172.21.15.60:0/2122177324 learned_addr learned my addr 172.21.15.60:0/2122177324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.878+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13971320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 -- 172.21.15.60:0/2122177324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 msgr2=0x55ae139610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26cffff640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae139610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae139610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 -- 172.21.15.60:0/2122177324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 msgr2=0x55ae13971320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13971320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 -- 172.21.15.60:0/2122177324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae13965000 con 0x55ae139551e0 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13971320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26cffff640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae139610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aa040 tx=0x7f26c40e5bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749096372eab925b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26b57fa640 1 -- 172.21.15.60:0/2122177324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c401e040 con 0x55ae139551e0 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26b57fa640 1 -- 172.21.15.60:0/2122177324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40aa340 con 0x55ae139551e0 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.879+0000 7f26b57fa640 1 -- 172.21.15.60:0/2122177324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40b2f60 con 0x55ae139551e0 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- 172.21.15.60:0/2122177324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 msgr2=0x55ae139628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aa040 tx=0x7f26c40e5bd0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- 172.21.15.60:0/2122177324 shutdown_connections 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae139610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139551e0 0x55ae139628b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2122177324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139548c0 0x55ae13971320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- 172.21.15.60:0/2122177324 >> 172.21.15.60:0/2122177324 conn(0x55ae139526f0 msgr2=0x55ae139621a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- 172.21.15.60:0/2122177324 shutdown_connections 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- 172.21.15.60:0/2122177324 wait complete. 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 -- start start 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 Processor -- start 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.880+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139551e0 0x55ae1396d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae1396db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01aa6b0 con 0x55ae139548c0 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01abb20 con 0x55ae139551e0 2024-07-24T22:00:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01aad80 con 0x55ae13965580 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 10 client.0 _ll_register_callbacks cb 0x7f26c800bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42504/0 (socket says 172.21.15.60:42504) 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 -- 172.21.15.60:0/2886980126 learned_addr learned my addr 172.21.15.60:0/2886980126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cffff640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae1396db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139551e0 0x55ae1396d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 -- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 msgr2=0x55ae1396db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae1396db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 -- 172.21.15.60:0/2886980126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139551e0 msgr2=0x55ae1396d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139551e0 0x55ae1396d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 -- 172.21.15.60:0/2886980126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae1397bc70 con 0x55ae139548c0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26cf7fe640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f26c0031af0 tx=0x7f26c002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9eaae421526eb7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0009b20 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f26c0003970 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c40aa340 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.881+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26c0003b90 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 10 client.18282 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ae139deec0 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c40aa340 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f26c002fbf0 con 0x55ae139548c0 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 10 client.18282 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 10 client.18282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 20 client.18282 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae139df2c0 0x55ae139e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 4 client.18282 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:00:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae138c7c70 con 0x55ae139df2c0 2024-07-24T22:00:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae139df2c0 0x55ae139e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26aeffd640 10 client.18282 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.882+0000 7f26ceffd640 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae139df2c0 0x55ae139e16b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0803f3b257726d server_cookie=96e7094358889e58 in_seq=0 out_seq=0 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26aeffd640 1 -- 172.21.15.60:0/2886980126 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae138c7c70 con 0x55ae139df2c0 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26aeffd640 10 client.18282 handle_command_reply: tid=0 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 client.18282 shutdown 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 msgr2=0x55ae1397d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f26c0031af0 tx=0x7f26c002a8c0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 shutdown_connections 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ae139df2c0 0x55ae139e16b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae13965580 0x55ae1396db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26aeffd640 0 client.18282 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae139551e0 0x55ae1396d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 --2- 172.21.15.60:0/2886980126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae139548c0 0x55ae1397d400 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.891+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 >> 172.21.15.60:0/2886980126 conn(0x55ae139526f0 msgr2=0x55ae13959a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 shutdown_connections 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d8d64740 1 -- 172.21.15.60:0/2886980126 wait complete. 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d8d64740 20 client.18282 trim_cache size 0 max 16384 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:00:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:00:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 msgr2=0x7f26b80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 0x7f26b80b6200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f26c8012560 tx=0x7f26c800b2e0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 msgr2=0x7f26d0199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b4a0 tx=0x7f26c400b960 comp rx=0 tx=0).stop 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 shutdown_connections 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26b80b3d40 0x7f26b80b6200 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26d0108a30 0x7f26d019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01080b0 0x7f26d0199190 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 --2- 172.21.15.60:0/763691658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d01026b0 0x7f26d0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 >> 172.21.15.60:0/763691658 conn(0x7f26d00fe0f0 msgr2=0x7f26d01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 shutdown_connections 2024-07-24T22:00:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:01.892+0000 7f26d6006640 1 -- 172.21.15.60:0/763691658 wait complete. 2024-07-24T22:00:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:01 smithi123 ceph-mon[37143]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 1.98k op/s 2024-07-24T22:00:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:01 smithi086 ceph-mon[37128]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 1.98k op/s 2024-07-24T22:00:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:01 smithi060 ceph-mon[33000]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 1.98k op/s 2024-07-24T22:00:03.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:00:03.893 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:03 smithi123 ceph-mon[37143]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.86k op/s 2024-07-24T22:00:04.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:03 smithi060 ceph-mon[33000]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.86k op/s 2024-07-24T22:00:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:03 smithi086 ceph-mon[37128]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.86k op/s 2024-07-24T22:00:05.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:05 smithi060 ceph-mon[33000]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- 172.21.15.60:0/4086506781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45401070a0 msgr2=0x7f45401074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 --2- 172.21.15.60:0/4086506781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45401070a0 0x7f45401074e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4534009900 tx=0x7f453402a410 comp rx=0 tx=0).stop 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- 172.21.15.60:0/4086506781 shutdown_connections 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 --2- 172.21.15.60:0/4086506781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4540107a20 0x7f454010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 --2- 172.21.15.60:0/4086506781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45401070a0 0x7f45401074e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 --2- 172.21.15.60:0/4086506781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- 172.21.15.60:0/4086506781 >> 172.21.15.60:0/4086506781 conn(0x7f45400fd100 msgr2=0x7f45400ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:05.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- 172.21.15.60:0/4086506781 shutdown_connections 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- 172.21.15.60:0/4086506781 wait complete. 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 Processor -- start 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.854+0000 7f4546411640 1 -- start start 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 0x7f45401980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4540111650 con 0x7f4540107a20 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45401114d0 con 0x7f4540101690 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f4546411640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45401117d0 con 0x7f45401070a0 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36074/0 (socket says 172.21.15.60:36074) 2024-07-24T22:00:05.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 -- 172.21.15.60:0/4052885337 learned_addr learned my addr 172.21.15.60:0/4052885337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f453f7fe640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 0x7f45401980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f453ffff640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 -- 172.21.15.60:0/4052885337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 msgr2=0x7f45401980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 0x7f45401980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 -- 172.21.15.60:0/4052885337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 msgr2=0x7f4540197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f45449ae640 1 -- 172.21.15.60:0/4052885337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454019dc30 con 0x7f4540107a20 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f453f7fe640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 0x7f45401980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.855+0000 7f453ffff640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f45449ae640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f453800cad0 tx=0x7f453800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453800ae20 con 0x7f4540107a20 2024-07-24T22:00:05.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4538005020 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454019df20 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453800f6b0 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.856+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45401a45e0 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.857+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4540101a70 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.858+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f45380051d0 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.859+0000 7f453d7fa640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 0x7f45280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.859+0000 7f453ffff640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 0x7f45280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.859+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8565+0+0 (secure 0 0 0) 0x7f45380e4570 con 0x7f4540107a20 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.860+0000 7f453ffff640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 0x7f45280b6200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45300058c0 tx=0x7f4530005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:05.862+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4538014030 con 0x7f4540107a20 2024-07-24T22:00:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:05 smithi123 ceph-mon[37143]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-07-24T22:00:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:05 smithi086 ceph-mon[37128]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-07-24T22:00:06.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.096+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4540198cb0 con 0x7f4540107a20 2024-07-24T22:00:06.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.097+0000 7f453d7fa640 1 -- 172.21.15.60:0/4052885337 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f45380ab7f0 con 0x7f4540107a20 2024-07-24T22:00:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:00:06.100 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:00:06.100 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 msgr2=0x7f45280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 0x7f45280b6200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f45300058c0 tx=0x7f4530005ec0 comp rx=0 tx=0).stop 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 msgr2=0x7f454019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f453800cad0 tx=0x7f453800cf90 comp rx=0 tx=0).stop 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 shutdown_connections 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f45280b3d40 0x7f45280b6200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4540107a20 0x7f454019d460 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45401070a0 0x7f45401980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 --2- 172.21.15.60:0/4052885337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4540101690 0x7f4540197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 >> 172.21.15.60:0/4052885337 conn(0x7f45400fd100 msgr2=0x7f45400ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 shutdown_connections 2024-07-24T22:00:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:06.099+0000 7f4546411640 1 -- 172.21.15.60:0/4052885337 wait complete. 2024-07-24T22:00:06.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:06 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:00:06.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:06 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:00:06.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:06 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/4052885337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:06.844 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 2e120f66 --release --await' 2024-07-24T22:00:06.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 2e120f66 --release --await 2024-07-24T22:00:06.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:06 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:00:06.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:06 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:00:06.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:06 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/4052885337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:07.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:06 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:00:07.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:06 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:00:07.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:06 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/4052885337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:07 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:00:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:07 smithi060 ceph-mon[33000]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-07-24T22:00:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:07 smithi060 ceph-mon[33000]: osdmap e104: 12 total, 12 up, 12 in 2024-07-24T22:00:07.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:07 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:00:07.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:07 smithi123 ceph-mon[37143]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-07-24T22:00:07.894 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:07 smithi123 ceph-mon[37143]: osdmap e104: 12 total, 12 up, 12 in 2024-07-24T22:00:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:07 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:00:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:07 smithi086 ceph-mon[37128]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-07-24T22:00:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:07 smithi086 ceph-mon[37128]: osdmap e104: 12 total, 12 up, 12 in 2024-07-24T22:00:09.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:09 smithi060 ceph-mon[33000]: pgmap v798: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 167 KiB/s wr, 828 op/s 2024-07-24T22:00:09.843 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:09 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:00:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:09 smithi123 ceph-mon[37143]: pgmap v798: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 167 KiB/s wr, 828 op/s 2024-07-24T22:00:09.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:09 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:00:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:09 smithi086 ceph-mon[37128]: pgmap v798: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 167 KiB/s wr, 828 op/s 2024-07-24T22:00:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:09 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:00:10.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.114+0000 7f270e8f6640 1 -- 172.21.15.60:0/3520114021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 msgr2=0x7f270810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:10.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.114+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3520114021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 0x7f270810a360 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2700009ca0 tx=0x7f270002a980 comp rx=0 tx=0).stop 2024-07-24T22:00:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.116+0000 7f270e8f6640 1 -- 172.21.15.60:0/3520114021 shutdown_connections 2024-07-24T22:00:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.116+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3520114021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f270811ba90 0x7f270811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.116+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3520114021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 0x7f270810a360 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.116+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3520114021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2708109b40 0x7f2708110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.116+0000 7f270e8f6640 1 -- 172.21.15.60:0/3520114021 >> 172.21.15.60:0/3520114021 conn(0x7f270806d980 msgr2=0x7f270806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:10.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.117+0000 7f270e8f6640 1 -- 172.21.15.60:0/3520114021 shutdown_connections 2024-07-24T22:00:10.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.117+0000 7f270e8f6640 1 -- 172.21.15.60:0/3520114021 wait complete. 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.118+0000 7f270e8f6640 1 Processor -- start 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.118+0000 7f270e8f6640 1 -- start start 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708109b40 0x7f270811afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 0x7f270811b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2708111b70 con 0x7f2708109f20 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27081119f0 con 0x7f270811ba90 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270e8f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2708111cf0 con 0x7f2708109b40 2024-07-24T22:00:10.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39676/0 (socket says 172.21.15.60:39676) 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 -- 172.21.15.60:0/3627745024 learned_addr learned my addr 172.21.15.60:0/3627745024 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 -- 172.21.15.60:0/3627745024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708109b40 msgr2=0x7f270811afa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708109b40 0x7f270811afa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 -- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 msgr2=0x7f270811b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 0x7f270811b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.119+0000 7f270ce93640 1 -- 172.21.15.60:0/3627745024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2708115d00 con 0x7f270811ba90 2024-07-24T22:00:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.120+0000 7f270ce93640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009fd0 tx=0x7f26fc00b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.121+0000 7f27057fa640 1 -- 172.21.15.60:0/3627745024 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc011830 con 0x7f270811ba90 2024-07-24T22:00:10.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.121+0000 7f27057fa640 1 -- 172.21.15.60:0/3627745024 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc016070 con 0x7f270811ba90 2024-07-24T22:00:10.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.121+0000 7f27057fa640 1 -- 172.21.15.60:0/3627745024 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc010de0 con 0x7f270811ba90 2024-07-24T22:00:10.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.121+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2708115ff0 con 0x7f270811ba90 2024-07-24T22:00:10.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.121+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270807b060 con 0x7f270811ba90 2024-07-24T22:00:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.125+0000 7f2710ca7740 1 Processor -- start 2024-07-24T22:00:10.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.129+0000 7f27057fa640 1 -- 172.21.15.60:0/3627745024 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f26fc010380 con 0x7f270811ba90 2024-07-24T22:00:10.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.131+0000 7f27057fa640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 0x7f26f00b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.131+0000 7f27057fa640 1 -- 172.21.15.60:0/3627745024 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f26fc0e3550 con 0x7f270811ba90 2024-07-24T22:00:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.131+0000 7f2710ca7740 1 -- start start 2024-07-24T22:00:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564ab7948dd0 con 0x564ab79d1a30 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ab79382d0 con 0x564ab79de2e0 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f2710ca7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ab793c310 con 0x564ab79d2350 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f270ce93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f270ce93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39680/0 (socket says 172.21.15.60:39680) 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.132+0000 7f270ce93640 1 -- 172.21.15.60:0/4045516514 learned_addr learned my addr 172.21.15.60:0/4045516514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f270ce93640 1 -- 172.21.15.60:0/4045516514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 msgr2=0x564ab79ddda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f270ce93640 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79ddda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f270ce93640 1 -- 172.21.15.60:0/4045516514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 msgr2=0x564ab79d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f270ce93640 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f270ce93640 1 -- 172.21.15.60:0/4045516514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab7698b00 con 0x564ab79de2e0 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.135+0000 7f27077fe640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 0x7f26f00b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.136+0000 7f270ce93640 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0a6230 tx=0x7f26fc0a9a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbbb90dd3f76f36 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.136+0000 7f26ee7fc640 1 -- 172.21.15.60:0/4045516514 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0ae7b0 con 0x564ab79de2e0 2024-07-24T22:00:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.136+0000 7f26ee7fc640 1 -- 172.21.15.60:0/4045516514 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0b3260 con 0x564ab79de2e0 2024-07-24T22:00:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.136+0000 7f26ee7fc640 1 -- 172.21.15.60:0/4045516514 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0a9700 con 0x564ab79de2e0 2024-07-24T22:00:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.137+0000 7f27077fe640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 0x7f26f00b6410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2700006010 tx=0x7f2700000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.137+0000 7f2710ca7740 1 -- 172.21.15.60:0/4045516514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 msgr2=0x564ab79e06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:10.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.137+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0a6230 tx=0x7f26fc0a9a50 comp rx=0 tx=0).stop 2024-07-24T22:00:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.138+0000 7f2710ca7740 1 -- 172.21.15.60:0/4045516514 shutdown_connections 2024-07-24T22:00:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.138+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79e06d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.138+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.138+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4045516514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.138+0000 7f2710ca7740 1 -- 172.21.15.60:0/4045516514 >> 172.21.15.60:0/4045516514 conn(0x564ab79cf860 msgr2=0x564ab79d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:10.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.145+0000 7f2710ca7740 1 -- 172.21.15.60:0/4045516514 shutdown_connections 2024-07-24T22:00:10.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.148+0000 7f2710ca7740 1 -- 172.21.15.60:0/4045516514 wait complete. 2024-07-24T22:00:10.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.148+0000 7f2710ca7740 1 -- start start 2024-07-24T22:00:10.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.149+0000 7f2710ca7740 1 Processor -- start 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d2350 0x564ab79db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564ab779bbb0 con 0x564ab79d2350 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ab78da900 con 0x564ab79de2e0 2024-07-24T22:00:10.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ab79e0f60 con 0x564ab79d1a30 2024-07-24T22:00:10.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.150+0000 7f2710ca7740 10 client.0 _ll_register_callbacks cb 0x564ab78e4c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:10.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.151+0000 7f2707fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.151+0000 7f2707fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:39000/0 (socket says 172.21.15.60:39000) 2024-07-24T22:00:10.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.151+0000 7f2707fff640 1 -- 172.21.15.60:0/2112858401 learned_addr learned my addr 172.21.15.60:0/2112858401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:10.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f270ce93640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f2707fff640 1 -- 172.21.15.60:0/2112858401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 msgr2=0x564ab79dbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f2707fff640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79dbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f2707fff640 1 -- 172.21.15.60:0/2112858401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d2350 msgr2=0x564ab79db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f2707fff640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d2350 0x564ab79db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.152+0000 7f2707fff640 1 -- 172.21.15.60:0/2112858401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab770f960 con 0x564ab79d1a30 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.153+0000 7f2707fff640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e3a0 tx=0x7f26f800e860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732ab207a3730b0b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.153+0000 7f26e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.153+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8010900 con 0x564ab79d1a30 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.153+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f26f8010aa0 con 0x564ab79d1a30 2024-07-24T22:00:10.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.154+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc0b3260 con 0x564ab79d1a30 2024-07-24T22:00:10.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.157+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26f8010ca0 con 0x564ab79d1a30 2024-07-24T22:00:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.164+0000 7f2710ca7740 10 client.27595 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.164+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564ab79ecb60 con 0x564ab79d1a30 2024-07-24T22:00:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.165+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0b3260 con 0x564ab79d1a30 2024-07-24T22:00:10.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.165+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f26f8017430 con 0x564ab79d1a30 2024-07-24T22:00:10.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 10 client.27595 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:10.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 10 client.27595 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:00:10.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 20 client.27595 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:10.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab79f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 4 client.27595 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:10.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.167+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab794ad20 con 0x564ab79ee090 2024-07-24T22:00:10.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.169+0000 7f27077fe640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab79f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.174+0000 7f27077fe640 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab79f0530 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95929d2f42d32333 server_cookie=d7d96233a21a6cee in_seq=0 out_seq=0 2024-07-24T22:00:10.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.174+0000 7f26e77fe640 10 client.27595 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:10.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.178+0000 7f26e77fe640 1 -- 172.21.15.60:0/2112858401 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ab794ad20 con 0x564ab79ee090 2024-07-24T22:00:10.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.178+0000 7f26e77fe640 10 client.27595 handle_command_reply: tid=0 2024-07-24T22:00:10.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.179+0000 7f2710ca7740 1 client.27595 shutdown 2024-07-24T22:00:10.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.181+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 msgr2=0x564ab79db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:10.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.181+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26f800e3a0 tx=0x7f26f800e860 comp rx=0 tx=0).stop 2024-07-24T22:00:10.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.187+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 shutdown_connections 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.187+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab79f0530 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.187+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79de2e0 0x564ab79dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.187+0000 7f26e77fe640 0 client.27595 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.187+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d2350 0x564ab79db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.188+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2112858401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d1a30 0x564ab79db220 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.188+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 >> 172.21.15.60:0/2112858401 conn(0x564ab79cf860 msgr2=0x564ab79d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.188+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 shutdown_connections 2024-07-24T22:00:10.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.188+0000 7f2710ca7740 1 -- 172.21.15.60:0/2112858401 wait complete. 2024-07-24T22:00:10.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.188+0000 7f2710ca7740 20 client.27595 trim_cache size 0 max 16384 2024-07-24T22:00:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.227+0000 7f2710ca7740 1 Processor -- start 2024-07-24T22:00:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.227+0000 7f2710ca7740 1 -- start start 2024-07-24T22:00:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.228+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d1a30 0x564ab79dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.228+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.228+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.228+0000 7f2710ca7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26f40113e0 con 0x564ab79de2e0 2024-07-24T22:00:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.228+0000 7f2710ca7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26f400ebf0 con 0x564ab79d1a30 2024-07-24T22:00:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f2710ca7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26f400ca60 con 0x564ab79d2350 2024-07-24T22:00:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f2707fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d1a30 0x564ab79dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36126/0 (socket says 172.21.15.60:36126) 2024-07-24T22:00:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 -- 172.21.15.60:0/2805592440 learned_addr learned my addr 172.21.15.60:0/2805592440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 -- 172.21.15.60:0/2805592440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 msgr2=0x564ab79f5e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79f5e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 -- 172.21.15.60:0/2805592440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d1a30 msgr2=0x564ab79dd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.229+0000 7f270ce93640 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d1a30 0x564ab79dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.230+0000 7f270ce93640 1 -- 172.21.15.60:0/2805592440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab79d9720 con 0x564ab79de2e0 2024-07-24T22:00:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.230+0000 7f270ce93640 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0aa0f0 tx=0x7f26fc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bb52a70372a766 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.231+0000 7f26edffb640 1 -- 172.21.15.60:0/2805592440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0c9070 con 0x564ab79de2e0 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.231+0000 7f26edffb640 1 -- 172.21.15.60:0/2805592440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0a67a0 con 0x564ab79de2e0 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.231+0000 7f26edffb640 1 -- 172.21.15.60:0/2805592440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0c3030 con 0x564ab79de2e0 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 -- 172.21.15.60:0/2805592440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 msgr2=0x564ab79f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0aa0f0 tx=0x7f26fc0047d0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 -- 172.21.15.60:0/2805592440 shutdown_connections 2024-07-24T22:00:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79de2e0 0x564ab79f6380 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79d2350 0x564ab79f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 --2- 172.21.15.60:0/2805592440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d1a30 0x564ab79dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.232+0000 7f2710ca7740 1 -- 172.21.15.60:0/2805592440 >> 172.21.15.60:0/2805592440 conn(0x564ab79cf860 msgr2=0x564ab79d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.234+0000 7f2710ca7740 1 -- 172.21.15.60:0/2805592440 shutdown_connections 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.235+0000 7f2710ca7740 1 -- 172.21.15.60:0/2805592440 wait complete. 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.235+0000 7f2710ca7740 1 -- start start 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.236+0000 7f2710ca7740 1 Processor -- start 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.236+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.236+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d2350 0x564ab79e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.236+0000 7f2710ca7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79de2e0 0x564ab79e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.237+0000 7f2710ca7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26f400d0b0 con 0x564ab79d1a30 2024-07-24T22:00:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.237+0000 7f2710ca7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26f400e520 con 0x564ab79d2350 2024-07-24T22:00:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.237+0000 7f2710ca7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26f400d780 con 0x564ab79de2e0 2024-07-24T22:00:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.237+0000 7f2710ca7740 10 client.0 _ll_register_callbacks cb 0x7f26f8009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:10.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.238+0000 7f2707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.238+0000 7f2707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36130/0 (socket says 172.21.15.60:36130) 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.238+0000 7f2707fff640 1 -- 172.21.15.60:0/4285679260 learned_addr learned my addr 172.21.15.60:0/4285679260 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.238+0000 7f27077fe640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d2350 0x564ab79e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f270ce93640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79de2e0 0x564ab79e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f2707fff640 1 -- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79de2e0 msgr2=0x564ab79e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f2707fff640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79de2e0 0x564ab79e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f2707fff640 1 -- 172.21.15.60:0/4285679260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d2350 msgr2=0x564ab79e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f2707fff640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d2350 0x564ab79e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.239+0000 7f2707fff640 1 -- 172.21.15.60:0/4285679260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab79e1930 con 0x564ab79d1a30 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.242+0000 7f2707fff640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26f8001eb0 tx=0x7f26f8017600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bfc2e8022ccbfc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.242+0000 7f26e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.242+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f802e020 con 0x564ab79d1a30 2024-07-24T22:00:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.242+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f26f8010910 con 0x564ab79d1a30 2024-07-24T22:00:10.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.243+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc0a67a0 con 0x564ab79d1a30 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.243+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26f8039050 con 0x564ab79d1a30 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.243+0000 7f2710ca7740 10 client.18297 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.243+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564ab7a59830 con 0x564ab79d1a30 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f8005850 con 0x564ab79d1a30 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f26f8003b70 con 0x564ab79d1a30 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 10 client.18297 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 10 client.18297 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:00:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 20 client.18297 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab7a5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 4 client.18297 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2e120f66", "release": true, "await": true}] 2024-07-24T22:00:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.244+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2e120f66", "release": true, "await": true}) -- 0x7f26fc0a67a0 con 0x564ab79ee090 2024-07-24T22:00:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.246+0000 7f27077fe640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab7a5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.260+0000 7f27077fe640 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab7a5c020 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7527e92fc9deb84b server_cookie=6582f2c9da18fb1d in_seq=0 out_seq=0 2024-07-24T22:00:10.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:10.260+0000 7f26e77fe640 10 client.18297 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:00:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:00:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:00:11.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:00:11.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:00:11.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:00:11.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:00:11.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:00:11.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:00:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:11 smithi123 ceph-mon[37143]: pgmap v799: 97 pgs: 18 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 48 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 87 KiB/s wr, 617 op/s 2024-07-24T22:00:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:11 smithi086 ceph-mon[37128]: pgmap v799: 97 pgs: 18 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 48 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 87 KiB/s wr, 617 op/s 2024-07-24T22:00:12.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:11 smithi060 ceph-mon[33000]: pgmap v799: 97 pgs: 18 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 48 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 87 KiB/s wr, 617 op/s 2024-07-24T22:00:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.421+0000 7f26e77fe640 1 -- 172.21.15.60:0/4285679260 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f26fc0a67a0 con 0x564ab79ee090 2024-07-24T22:00:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.422+0000 7f26e77fe640 10 client.18297 handle_command_reply: tid=0 2024-07-24T22:00:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.422+0000 7f2710ca7740 1 client.18297 shutdown 2024-07-24T22:00:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.422+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 msgr2=0x564ab79e6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.422+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26f8001eb0 tx=0x7f26f8017600 comp rx=0 tx=0).stop 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.422+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 shutdown_connections 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564ab79ee090 0x564ab7a5c020 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ab79de2e0 0x564ab79e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ab79d2350 0x564ab79e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 --2- 172.21.15.60:0/4285679260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ab79d1a30 0x564ab79e6470 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.426 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 40, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "2e120f66": { 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 40, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 134.3, 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:00:12.427 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:12.428 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 >> 172.21.15.60:0/4285679260 conn(0x564ab79cf860 msgr2=0x564ab79dd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f26e77fe640 0 client.18297 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 shutdown_connections 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 1 -- 172.21.15.60:0/4285679260 wait complete. 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.423+0000 7f2710ca7740 20 client.18297 trim_cache size 0 max 16384 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 msgr2=0x7f26f00b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:12.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 0x7f26f00b6410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2700006010 tx=0x7f2700000f40 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 msgr2=0x7f2708115440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009fd0 tx=0x7f26fc00b8e0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 shutdown_connections 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f26f00b3f50 0x7f26f00b6410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f270811ba90 0x7f2708115440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2708109f20 0x7f270811b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 --2- 172.21.15.60:0/3627745024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2708109b40 0x7f270811afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.424+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 >> 172.21.15.60:0/3627745024 conn(0x7f270806d980 msgr2=0x7f2708072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.425+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 shutdown_connections 2024-07-24T22:00:12.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:12.425+0000 7f270e8f6640 1 -- 172.21.15.60:0/3627745024 wait complete. 2024-07-24T22:00:13.299 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2e120f66', total seconds elapsed: 24.0 2024-07-24T22:00:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:13 smithi123 ceph-mon[37143]: pgmap v800: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 59 KiB/s wr, 266 op/s 2024-07-24T22:00:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:13 smithi086 ceph-mon[37128]: pgmap v800: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 59 KiB/s wr, 266 op/s 2024-07-24T22:00:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:13 smithi060 ceph-mon[33000]: pgmap v800: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 59 KiB/s wr, 266 op/s 2024-07-24T22:00:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.045+0000 7fc6575ed640 1 -- 172.21.15.60:0/2263778269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65010f580 msgr2=0x7fc650115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.045+0000 7fc6575ed640 1 --2- 172.21.15.60:0/2263778269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65010f580 0x7fc650115cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc648009cd0 tx=0x7fc64802a930 comp rx=0 tx=0).stop 2024-07-24T22:00:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.048+0000 7fc6575ed640 1 -- 172.21.15.60:0/2263778269 shutdown_connections 2024-07-24T22:00:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.048+0000 7fc6575ed640 1 --2- 172.21.15.60:0/2263778269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc65010f580 0x7fc650115cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.048+0000 7fc6575ed640 1 --2- 172.21.15.60:0/2263778269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010ec00 0x7fc65010f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.048+0000 7fc6575ed640 1 --2- 172.21.15.60:0/2263778269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc650109db0 0x7fc65010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.048+0000 7fc6575ed640 1 -- 172.21.15.60:0/2263778269 >> 172.21.15.60:0/2263778269 conn(0x7fc65006db80 msgr2=0x7fc65006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.051+0000 7fc6575ed640 1 -- 172.21.15.60:0/2263778269 shutdown_connections 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.051+0000 7fc6575ed640 1 -- 172.21.15.60:0/2263778269 wait complete. 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.052+0000 7fc6575ed640 1 Processor -- start 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 -- start start 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc650109db0 0x7fc650114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010f580 0x7fc65011cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501184c0 con 0x7fc65010ec00 2024-07-24T22:00:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc650118340 con 0x7fc65010f580 2024-07-24T22:00:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.055+0000 7fc6575ed640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc650118640 con 0x7fc650109db0 2024-07-24T22:00:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.057+0000 7fc654b88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.057+0000 7fc654b88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36138/0 (socket says 172.21.15.60:36138) 2024-07-24T22:00:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.057+0000 7fc654b88640 1 -- 172.21.15.60:0/4277314488 learned_addr learned my addr 172.21.15.60:0/4277314488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.057+0000 7fc655389640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc650109db0 0x7fc650114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.057+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010f580 0x7fc65011cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 -- 172.21.15.60:0/4277314488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc650109db0 msgr2=0x7fc650114b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc650109db0 0x7fc650114b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 -- 172.21.15.60:0/4277314488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010f580 msgr2=0x7fc65011cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010f580 0x7fc65011cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 -- 172.21.15.60:0/4277314488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65011d420 con 0x7fc65010ec00 2024-07-24T22:00:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.058+0000 7fc654b88640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9f0 tx=0x7fc64400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.061+0000 7fc6427fc640 1 -- 172.21.15.60:0/4277314488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64400cd80 con 0x7fc65010ec00 2024-07-24T22:00:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.062+0000 7fc6427fc640 1 -- 172.21.15.60:0/4277314488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6440108e0 con 0x7fc65010ec00 2024-07-24T22:00:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.062+0000 7fc6427fc640 1 -- 172.21.15.60:0/4277314488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644018860 con 0x7fc65010ec00 2024-07-24T22:00:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.062+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65011d6b0 con 0x7fc65010ec00 2024-07-24T22:00:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.062+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc650123030 con 0x7fc65010ec00 2024-07-24T22:00:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.067+0000 7fc65a429740 1 Processor -- start 2024-07-24T22:00:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.068+0000 7fc6427fc640 1 -- 172.21.15.60:0/4277314488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc644018a00 con 0x7fc65010ec00 2024-07-24T22:00:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 -- start start 2024-07-24T22:00:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e880000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e882930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c71e7e9280 con 0x55c71e880540 2024-07-24T22:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c71e7d8780 con 0x55c71e873c90 2024-07-24T22:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc65a429740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c71e7dc7c0 con 0x55c71e8745b0 2024-07-24T22:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc655b8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc655b8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39734/0 (socket says 172.21.15.60:39734) 2024-07-24T22:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc655b8a640 1 -- 172.21.15.60:0/4181884275 learned_addr learned my addr 172.21.15.60:0/4181884275 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.070+0000 7fc6427fc640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 0x7fc63c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 0x7fc63c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc6427fc640 1 -- 172.21.15.60:0/4277314488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc6440e4180 con 0x7fc65010ec00 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 -- 172.21.15.60:0/4181884275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 msgr2=0x55c71e880000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e880000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 -- 172.21.15.60:0/4181884275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 msgr2=0x55c71e882930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e882930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 -- 172.21.15.60:0/4181884275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71e53c790 con 0x55c71e873c90 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.071+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc648006010 tx=0x7fc648003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5038d74246d0cfd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.072+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 0x7fc63c0b6350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc648048000 tx=0x7fc64804a000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.072+0000 7fc63b7fe640 1 -- 172.21.15.60:0/4181884275 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480307f0 con 0x55c71e873c90 2024-07-24T22:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.072+0000 7fc63b7fe640 1 -- 172.21.15.60:0/4181884275 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648036070 con 0x55c71e873c90 2024-07-24T22:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.072+0000 7fc63b7fe640 1 -- 172.21.15.60:0/4181884275 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64804a470 con 0x55c71e873c90 2024-07-24T22:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.074+0000 7fc65a429740 1 -- 172.21.15.60:0/4181884275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 msgr2=0x55c71e874070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.074+0000 7fc65a429740 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc648006010 tx=0x7fc648003e40 comp rx=0 tx=0).stop 2024-07-24T22:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.075+0000 7fc65a429740 1 -- 172.21.15.60:0/4181884275 shutdown_connections 2024-07-24T22:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.075+0000 7fc65a429740 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e882930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.075+0000 7fc65a429740 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e880000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.075+0000 7fc65a429740 1 --2- 172.21.15.60:0/4181884275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e874070 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.075+0000 7fc65a429740 1 -- 172.21.15.60:0/4181884275 >> 172.21.15.60:0/4181884275 conn(0x55c71e871ac0 msgr2=0x55c71e872c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.076+0000 7fc65a429740 1 -- 172.21.15.60:0/4181884275 shutdown_connections 2024-07-24T22:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.076+0000 7fc65a429740 1 -- 172.21.15.60:0/4181884275 wait complete. 2024-07-24T22:00:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.077+0000 7fc65a429740 1 -- start start 2024-07-24T22:00:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.077+0000 7fc65a429740 1 Processor -- start 2024-07-24T22:00:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.078+0000 7fc65a429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e87fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.078+0000 7fc65a429740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e87d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.078+0000 7fc65a429740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc65a429740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c71e712ab0 con 0x55c71e880540 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc655b8a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc655b8a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36156/0 (socket says 172.21.15.60:36156) 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc655b8a640 1 -- 172.21.15.60:0/4185177977 learned_addr learned my addr 172.21.15.60:0/4185177977 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c71e7609e0 con 0x55c71e873c90 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c71e883160 con 0x55c71e8745b0 2024-07-24T22:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.079+0000 7fc65a429740 10 client.0 _ll_register_callbacks cb 0x55c71e4e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 -- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 msgr2=0x55c71e87d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e87d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 -- 172.21.15.60:0/4185177977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 msgr2=0x55c71e87fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e87fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71e5b5620 con 0x55c71e880540 2024-07-24T22:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc655b8a640 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc64804a930 tx=0x7fc64804a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bee3d02fc29429 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.080+0000 7fc638ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.081+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648057030 con 0x55c71e880540 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.081+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc64804ad90 con 0x55c71e880540 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.081+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c71e787ce0 con 0x55c71e880540 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.082+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc648062050 con 0x55c71e880540 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.082+0000 7fc65a429740 10 client.18303 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.082+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c71e88e980 con 0x55c71e880540 2024-07-24T22:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648036050 con 0x55c71e880540 2024-07-24T22:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc64805e4e0 con 0x55c71e880540 2024-07-24T22:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 10 client.18303 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 10 client.18303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 20 client.18303 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e88fcd0 0x55c71e8920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 4 client.18303 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.083+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c71e787ce0 con 0x55c71e88fcd0 2024-07-24T22:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.084+0000 7fc655389640 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e88fcd0 0x55c71e8920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.091+0000 7fc638ff9640 10 client.18303 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.091+0000 7fc655389640 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e88fcd0 0x55c71e8920c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663382ae66c4815f server_cookie=87301e39b3627f55 in_seq=0 out_seq=0 2024-07-24T22:00:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.093+0000 7fc638ff9640 1 -- 172.21.15.60:0/4185177977 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c71e787ce0 con 0x55c71e88fcd0 2024-07-24T22:00:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.093+0000 7fc638ff9640 10 client.18303 handle_command_reply: tid=0 2024-07-24T22:00:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.093+0000 7fc65a429740 1 client.18303 shutdown 2024-07-24T22:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 msgr2=0x55c71e87d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc64804a930 tx=0x7fc64804a960 comp rx=0 tx=0).stop 2024-07-24T22:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 shutdown_connections 2024-07-24T22:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e88fcd0 0x55c71e8920c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e880540 0x55c71e87d6c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e8745b0 0x55c71e87d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 --2- 172.21.15.60:0/4185177977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e87fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.095+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 >> 172.21.15.60:0/4185177977 conn(0x55c71e871ac0 msgr2=0x55c71e877660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.097+0000 7fc638ff9640 0 client.18303 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.100+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 shutdown_connections 2024-07-24T22:00:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.100+0000 7fc65a429740 1 -- 172.21.15.60:0/4185177977 wait complete. 2024-07-24T22:00:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.105+0000 7fc65a429740 20 client.18303 trim_cache size 0 max 16384 2024-07-24T22:00:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.134+0000 7fc65a429740 1 Processor -- start 2024-07-24T22:00:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.134+0000 7fc65a429740 1 -- start start 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e891ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e892220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e87b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340113e0 con 0x55c71e8745b0 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc63400ebf0 con 0x55c71e873c90 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc65a429740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc63400ca60 con 0x55c71e880540 2024-07-24T22:00:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc655389640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e891ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc655389640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e891ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39746/0 (socket says 172.21.15.60:39746) 2024-07-24T22:00:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc655389640 1 -- 172.21.15.60:0/2172427108 learned_addr learned my addr 172.21.15.60:0/2172427108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.135+0000 7fc655b8a640 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e87b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc654b88640 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e892220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc655b8a640 1 -- 172.21.15.60:0/2172427108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 msgr2=0x55c71e891ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc655b8a640 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e891ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc655b8a640 1 -- 172.21.15.60:0/2172427108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 msgr2=0x55c71e892220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc655b8a640 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e892220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.137+0000 7fc655b8a640 1 -- 172.21.15.60:0/2172427108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71e87cf00 con 0x55c71e880540 2024-07-24T22:00:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.139+0000 7fc655b8a640 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e87b9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc64804a9d0 tx=0x7fc648031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e47a67faaeb6ac7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.143+0000 7fc63affd640 1 -- 172.21.15.60:0/2172427108 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648057030 con 0x55c71e880540 2024-07-24T22:00:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.143+0000 7fc63affd640 1 -- 172.21.15.60:0/2172427108 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648036040 con 0x55c71e880540 2024-07-24T22:00:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.143+0000 7fc63affd640 1 -- 172.21.15.60:0/2172427108 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64805e110 con 0x55c71e880540 2024-07-24T22:00:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 -- 172.21.15.60:0/2172427108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 msgr2=0x55c71e87b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e87b9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc64804a9d0 tx=0x7fc648031000 comp rx=0 tx=0).stop 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 -- 172.21.15.60:0/2172427108 shutdown_connections 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e87b9e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e892220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 --2- 172.21.15.60:0/2172427108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e891ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 -- 172.21.15.60:0/2172427108 >> 172.21.15.60:0/2172427108 conn(0x55c71e871ac0 msgr2=0x55c71e87d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 -- 172.21.15.60:0/2172427108 shutdown_connections 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.145+0000 7fc65a429740 1 -- 172.21.15.60:0/2172427108 wait complete. 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.146+0000 7fc65a429740 1 -- start start 2024-07-24T22:00:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.146+0000 7fc65a429740 1 Processor -- start 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.146+0000 7fc65a429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e8853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e8f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc63400d0b0 con 0x55c71e8745b0 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc63400e520 con 0x55c71e873c90 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc63400d780 con 0x55c71e880540 2024-07-24T22:00:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc655389640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc655389640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39754/0 (socket says 172.21.15.60:39754) 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc655389640 1 -- 172.21.15.60:0/707619542 learned_addr learned my addr 172.21.15.60:0/707619542 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.147+0000 7fc65a429740 10 client.0 _ll_register_callbacks cb 0x7fc648003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655b8a640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e8f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc654b88640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e8853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655389640 1 -- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 msgr2=0x55c71e8f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655389640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e8f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655389640 1 -- 172.21.15.60:0/707619542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 msgr2=0x55c71e8853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655389640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e8853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.148+0000 7fc655389640 1 -- 172.21.15.60:0/707619542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71e8f20c0 con 0x55c71e873c90 2024-07-24T22:00:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.153+0000 7fc655389640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009ad0 tx=0x7fc64c009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0b277d4e726a1d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.153+0000 7fc638ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c022020 con 0x55c71e873c90 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc648036040 con 0x55c71e873c90 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc64c00f040 con 0x55c71e873c90 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc64c02d050 con 0x55c71e873c90 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc65a429740 10 client.27689 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.155+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c71e8f9590 con 0x55c71e873c90 2024-07-24T22:00:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.157+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648036040 con 0x55c71e873c90 2024-07-24T22:00:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.157+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc64c01ebd0 con 0x55c71e873c90 2024-07-24T22:00:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 10 client.27689 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 10 client.27689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 20 client.27689 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e8f9da0 0x55c71e8fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 4 client.27689 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:00:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.159+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c71e7eb1d0 con 0x55c71e8f9da0 2024-07-24T22:00:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.164+0000 7fc654b88640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e8f9da0 0x55c71e8fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.175+0000 7fc654b88640 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e8f9da0 0x55c71e8fc1c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576f37bb2fac6efc server_cookie=4fdbb1ba0174e647 in_seq=0 out_seq=0 2024-07-24T22:00:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.175+0000 7fc638ff9640 10 client.27689 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.255+0000 7fc638ff9640 1 -- 172.21.15.60:0/707619542 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c71e7eb1d0 con 0x55c71e8f9da0 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.255+0000 7fc638ff9640 10 client.27689 handle_command_reply: tid=0 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 client.27689 shutdown 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 msgr2=0x55c71e884e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009ad0 tx=0x7fc64c009bc0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 shutdown_connections 2024-07-24T22:00:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c71e8f9da0 0x55c71e8fc1c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c71e880540 0x55c71e8f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c71e8745b0 0x55c71e8853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 --2- 172.21.15.60:0/707619542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c71e873c90 0x55c71e884e30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 >> 172.21.15.60:0/707619542 conn(0x55c71e871ac0 msgr2=0x55c71e891ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc638ff9640 0 client.27689 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 shutdown_connections 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 1 -- 172.21.15.60:0/707619542 wait complete. 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.256+0000 7fc65a429740 20 client.27689 trim_cache size 0 max 16384 2024-07-24T22:00:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:00:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:00:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:00:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:00:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 msgr2=0x7fc63c0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 0x7fc63c0b6350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc648048000 tx=0x7fc64804a000 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 msgr2=0x7fc650115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e9f0 tx=0x7fc64400eeb0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 shutdown_connections 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc63c0b3e90 0x7fc63c0b6350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc65010f580 0x7fc65011cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65010ec00 0x7fc650115080 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 --2- 172.21.15.60:0/4277314488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc650109db0 0x7fc650114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 >> 172.21.15.60:0/4277314488 conn(0x7fc65006db80 msgr2=0x7fc65010ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 shutdown_connections 2024-07-24T22:00:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:15.257+0000 7fc6575ed640 1 -- 172.21.15.60:0/4277314488 wait complete. 2024-07-24T22:00:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:15 smithi123 ceph-mon[37143]: pgmap v801: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 32 KiB/s wr, 128 op/s 2024-07-24T22:00:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:15 smithi086 ceph-mon[37128]: pgmap v801: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 32 KiB/s wr, 128 op/s 2024-07-24T22:00:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:15 smithi060 ceph-mon[33000]: pgmap v801: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 32 KiB/s wr, 128 op/s 2024-07-24T22:00:17.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:00:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:17 smithi123 ceph-mon[37143]: pgmap v802: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 263 KiB/s wr, 27 op/s 2024-07-24T22:00:18.021 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:17 smithi060 ceph-mon[33000]: pgmap v802: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 263 KiB/s wr, 27 op/s 2024-07-24T22:00:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:17 smithi086 ceph-mon[37128]: pgmap v802: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 263 KiB/s wr, 27 op/s 2024-07-24T22:00:18.217 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:19.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:19 smithi123 ceph-mon[37143]: pgmap v803: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 522 KiB/s wr, 42 op/s 2024-07-24T22:00:19.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:19 smithi060 ceph-mon[33000]: pgmap v803: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 522 KiB/s wr, 42 op/s 2024-07-24T22:00:19.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:19 smithi086 ceph-mon[37128]: pgmap v803: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 522 KiB/s wr, 42 op/s 2024-07-24T22:00:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.222+0000 7f5af759e640 1 -- 172.21.15.60:0/3490240774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00a6f10 msgr2=0x7f5af00a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.222+0000 7f5af759e640 1 --2- 172.21.15.60:0/3490240774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00a6f10 0x7f5af00a7390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009900 tx=0x7f5ae802a420 comp rx=0 tx=0).stop 2024-07-24T22:00:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.223+0000 7f5af759e640 1 -- 172.21.15.60:0/3490240774 shutdown_connections 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.223+0000 7f5af759e640 1 --2- 172.21.15.60:0/3490240774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00098b0 0x7f5af000bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.223+0000 7f5af759e640 1 --2- 172.21.15.60:0/3490240774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00a6f10 0x7f5af00a7390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.223+0000 7f5af759e640 1 --2- 172.21.15.60:0/3490240774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a4e50 0x7f5af00a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.223+0000 7f5af759e640 1 -- 172.21.15.60:0/3490240774 >> 172.21.15.60:0/3490240774 conn(0x7f5af001a050 msgr2=0x7f5af001c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.224+0000 7f5af759e640 1 -- 172.21.15.60:0/3490240774 shutdown_connections 2024-07-24T22:00:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.224+0000 7f5af759e640 1 -- 172.21.15.60:0/3490240774 wait complete. 2024-07-24T22:00:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 Processor -- start 2024-07-24T22:00:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 -- start start 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00098b0 0x7f5af0016190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00a4e50 0x7f5af00103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000f240 con 0x7f5af00a6f10 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000f0c0 con 0x7f5af00a4e50 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af759e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000f3c0 con 0x7f5af00098b0 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.225+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48198/0 (socket says 172.21.15.60:48198) 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 -- 172.21.15.60:0/3289235373 learned_addr learned my addr 172.21.15.60:0/3289235373 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00a4e50 0x7f5af00103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 -- 172.21.15.60:0/3289235373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00098b0 msgr2=0x7f5af0016190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00098b0 0x7f5af0016190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 -- 172.21.15.60:0/3289235373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00a4e50 msgr2=0x7f5af00103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00a4e50 0x7f5af00103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 -- 172.21.15.60:0/3289235373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af015c2d0 con 0x7f5af00a6f10 2024-07-24T22:00:20.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.226+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5af80685c0 tx=0x7f5af8071820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.227+0000 7f5ae77fe640 1 -- 172.21.15.60:0/3289235373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8076590 con 0x7f5af00a6f10 2024-07-24T22:00:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.227+0000 7f5ae77fe640 1 -- 172.21.15.60:0/3289235373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af8076be0 con 0x7f5af00a6f10 2024-07-24T22:00:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.228+0000 7f5ae77fe640 1 -- 172.21.15.60:0/3289235373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8077cc0 con 0x7f5af00a6f10 2024-07-24T22:00:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.228+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af015c4a0 con 0x7f5af00a6f10 2024-07-24T22:00:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.228+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af015c960 con 0x7f5af00a6f10 2024-07-24T22:00:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.237+0000 7f5aff17c740 1 Processor -- start 2024-07-24T22:00:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.239+0000 7f5ae77fe640 1 -- 172.21.15.60:0/3289235373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5af807e030 con 0x7f5af00a6f10 2024-07-24T22:00:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.240+0000 7f5ae77fe640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 0x7f5ae00b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 -- start start 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5ae77fe640 1 -- 172.21.15.60:0/3289235373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f5af8147780 con 0x7f5af00a6f10 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 0x7f5ae00b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e3529f800 0x557e3529fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557e35216960 con 0x557e352ac0b0 2024-07-24T22:00:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557e35205e60 con 0x557e3529f800 2024-07-24T22:00:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5aff17c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557e35209ea0 con 0x557e352a0120 2024-07-24T22:00:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48212/0 (socket says 172.21.15.60:48212) 2024-07-24T22:00:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1193711178 learned_addr learned my addr 172.21.15.60:0/1193711178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.241+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 0x7f5ae00b61e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5af0015ee0 tx=0x7f5ae8031db0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 -- 172.21.15.60:0/1193711178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e3529f800 msgr2=0x557e3529fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e3529f800 0x557e3529fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 -- 172.21.15.60:0/1193711178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 msgr2=0x557e352ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.242+0000 7f5af5d9b640 1 -- 172.21.15.60:0/1193711178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3518f2d0 con 0x557e352a0120 2024-07-24T22:00:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.246+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352abb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69e11232c79edde server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.250+0000 7f5ae4ff9640 1 -- 172.21.15.60:0/1193711178 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8035050 con 0x557e352a0120 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.250+0000 7f5ae4ff9640 1 -- 172.21.15.60:0/1193711178 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae803f170 con 0x557e352a0120 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.251+0000 7f5aff17c740 1 -- 172.21.15.60:0/1193711178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 msgr2=0x557e352abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.251+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352abb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8046000 comp rx=0 tx=0).stop 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.252+0000 7f5aff17c740 1 -- 172.21.15.60:0/1193711178 shutdown_connections 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.252+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.252+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352abb70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.252+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1193711178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e3529f800 0x557e3529fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.252+0000 7f5aff17c740 1 -- 172.21.15.60:0/1193711178 >> 172.21.15.60:0/1193711178 conn(0x557e3529d630 msgr2=0x557e3529e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.254+0000 7f5aff17c740 1 -- 172.21.15.60:0/1193711178 shutdown_connections 2024-07-24T22:00:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.254+0000 7f5aff17c740 1 -- 172.21.15.60:0/1193711178 wait complete. 2024-07-24T22:00:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 -- start start 2024-07-24T22:00:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 Processor -- start 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557e35069700 con 0x557e352ac0b0 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557e351a85a0 con 0x557e352a0120 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.255+0000 7f5aff17c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557e352aed20 con 0x557e3529f800 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.256+0000 7f5aff17c740 10 client.0 _ll_register_callbacks cb 0x557e351c8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.256+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.256+0000 7f5af5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.256+0000 7f5af6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48216/0 (socket says 172.21.15.60:48216) 2024-07-24T22:00:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.256+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1609092355 learned_addr learned my addr 172.21.15.60:0/1609092355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.257+0000 7f5af659c640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.258+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 msgr2=0x557e352aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.258+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.258+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1609092355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 msgr2=0x557e352a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.258+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.258+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1609092355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e3519b630 con 0x557e352ac0b0 2024-07-24T22:00:20.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.259+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5af8145470 tx=0x7f5af810df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5c6a817b37c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.259+0000 7f5addffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.259+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8077e60 con 0x557e352ac0b0 2024-07-24T22:00:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.259+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f5af810aac0 con 0x557e352ac0b0 2024-07-24T22:00:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.260+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae803f170 con 0x557e352ac0b0 2024-07-24T22:00:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.261+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5af812d050 con 0x557e352ac0b0 2024-07-24T22:00:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.262+0000 7f5aff17c740 10 client.18321 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.262+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557e34f66650 con 0x557e352ac0b0 2024-07-24T22:00:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.263+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae803f170 con 0x557e352ac0b0 2024-07-24T22:00:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5af810d730 con 0x557e352ac0b0 2024-07-24T22:00:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 10 client.18321 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 20 client.18321 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e352ccb20 0x557e352cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 4 client.18321 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e352188b0 con 0x557e352ccb20 2024-07-24T22:00:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.264+0000 7f5af659c640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e352ccb20 0x557e352cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.274+0000 7f5af659c640 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e352ccb20 0x557e352cef10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb148f763960cb7 server_cookie=565f5595405b9ce9 in_seq=0 out_seq=0 2024-07-24T22:00:20.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.274+0000 7f5addffb640 10 client.18321 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:20.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.275+0000 7f5addffb640 1 -- 172.21.15.60:0/1609092355 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e352188b0 con 0x557e352ccb20 2024-07-24T22:00:20.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.275+0000 7f5addffb640 10 client.18321 handle_command_reply: tid=0 2024-07-24T22:00:20.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.275+0000 7f5aff17c740 1 client.18321 shutdown 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 msgr2=0x557e352a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5af8145470 tx=0x7f5af810df10 comp rx=0 tx=0).stop 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 shutdown_connections 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e352ccb20 0x557e352cef10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352a94f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1609092355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 >> 172.21.15.60:0/1609092355 conn(0x557e3529d630 msgr2=0x557e352a31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.276+0000 7f5addffb640 0 client.18321 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:20.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.277+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 shutdown_connections 2024-07-24T22:00:20.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.277+0000 7f5aff17c740 1 -- 172.21.15.60:0/1609092355 wait complete. 2024-07-24T22:00:20.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.278+0000 7f5aff17c740 20 client.18321 trim_cache size 0 max 16384 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 Processor -- start 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 -- start start 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0163960 con 0x557e352ac0b0 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0161170 con 0x557e352a0120 2024-07-24T22:00:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.312+0000 7f5aff17c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af015f060 con 0x557e3529f800 2024-07-24T22:00:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.315+0000 7f5af659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.315+0000 7f5af659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37402/0 (socket says 172.21.15.60:37402) 2024-07-24T22:00:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.315+0000 7f5af659c640 1 -- 172.21.15.60:0/1734539406 learned_addr learned my addr 172.21.15.60:0/1734539406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.315+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1734539406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 msgr2=0x557e352a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1734539406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 msgr2=0x557e352a7c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a7c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 -- 172.21.15.60:0/1734539406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e352cb650 con 0x557e352ac0b0 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.316+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352d47a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5af810e1d0 tx=0x7f5af810de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb539a8e6e446608 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.317+0000 7f5adffff640 1 -- 172.21.15.60:0/1734539406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8077e60 con 0x557e352ac0b0 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.317+0000 7f5adffff640 1 -- 172.21.15.60:0/1734539406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af810e4d0 con 0x557e352ac0b0 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.318+0000 7f5aff17c740 1 -- 172.21.15.60:0/1734539406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 msgr2=0x557e352d47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.318+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352d47a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5af810e1d0 tx=0x7f5af810de20 comp rx=0 tx=0).stop 2024-07-24T22:00:20.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.320+0000 7f5aff17c740 1 -- 172.21.15.60:0/1734539406 shutdown_connections 2024-07-24T22:00:20.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.320+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e352ac0b0 0x557e352d47a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.320+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352a0120 0x557e352a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.320+0000 7f5aff17c740 1 --2- 172.21.15.60:0/1734539406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e3529f800 0x557e352a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.320+0000 7f5aff17c740 1 -- 172.21.15.60:0/1734539406 >> 172.21.15.60:0/1734539406 conn(0x557e3529d630 msgr2=0x557e352a4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.323+0000 7f5aff17c740 1 -- 172.21.15.60:0/1734539406 shutdown_connections 2024-07-24T22:00:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.325+0000 7f5aff17c740 1 -- 172.21.15.60:0/1734539406 wait complete. 2024-07-24T22:00:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.326+0000 7f5aff17c740 1 -- start start 2024-07-24T22:00:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.326+0000 7f5aff17c740 1 Processor -- start 2024-07-24T22:00:20.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 0x557e352b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352ac0b0 0x557e352ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5af015f6b0 con 0x557e3529f800 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0160aa0 con 0x557e352ac0b0 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af015fd00 con 0x557e352a0120 2024-07-24T22:00:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.328+0000 7f5aff17c740 10 client.0 _ll_register_callbacks cb 0x7f5af8076fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:20.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.329+0000 7f5af5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.329+0000 7f5af5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37414/0 (socket says 172.21.15.60:37414) 2024-07-24T22:00:20.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.329+0000 7f5af5d9b640 1 -- 172.21.15.60:0/995993434 learned_addr learned my addr 172.21.15.60:0/995993434 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:20.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.329+0000 7f5af6d9d640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352ac0b0 0x557e352ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.330+0000 7f5af659c640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 0x557e352b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.332+0000 7f5af659c640 1 -- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 msgr2=0x557e352aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.332+0000 7f5af659c640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.332+0000 7f5af659c640 1 -- 172.21.15.60:0/995993434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352ac0b0 msgr2=0x557e352ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.332+0000 7f5af659c640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352ac0b0 0x557e352ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.332+0000 7f5af659c640 1 -- 172.21.15.60:0/995993434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e352adb80 con 0x557e3529f800 2024-07-24T22:00:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.333+0000 7f5af659c640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 0x557e352b4630 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0057c0 tx=0x7f5aec004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad2db27671c6356 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.333+0000 7f5addffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.334+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00bbf0 con 0x557e3529f800 2024-07-24T22:00:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.334+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af810e4d0 con 0x557e3529f800 2024-07-24T22:00:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.334+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f5aec00bd90 con 0x557e3529f800 2024-07-24T22:00:20.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5aec01a300 con 0x557e3529f800 2024-07-24T22:00:20.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 10 client.18327 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:20.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557e35337770 con 0x557e3529f800 2024-07-24T22:00:20.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af810e4d0 con 0x557e3529f800 2024-07-24T22:00:20.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5aec0118d0 con 0x557e3529f800 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 10 client.18327 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 20 client.18327 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e35337f30 0x557e3533a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 4 client.18327 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.335+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e35298d50 con 0x557e35337f30 2024-07-24T22:00:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.337+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e35337f30 0x557e3533a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.347+0000 7f5addffb640 10 client.18327 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.347+0000 7f5af5d9b640 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e35337f30 0x557e3533a350 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6c36713527a98 server_cookie=909c7535dd36e37d in_seq=0 out_seq=0 2024-07-24T22:00:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.348+0000 7f5addffb640 1 -- 172.21.15.60:0/995993434 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e35298d50 con 0x557e35337f30 2024-07-24T22:00:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.348+0000 7f5addffb640 10 client.18327 handle_command_reply: tid=0 2024-07-24T22:00:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.348+0000 7f5aff17c740 1 client.18327 shutdown 2024-07-24T22:00:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.353+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 msgr2=0x557e352b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.353+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 0x557e352b4630 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0057c0 tx=0x7f5aec004c40 comp rx=0 tx=0).stop 2024-07-24T22:00:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 shutdown_connections 2024-07-24T22:00:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557e35337f30 0x557e3533a350 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e352ac0b0 0x557e352ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e352a0120 0x557e352aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 --2- 172.21.15.60:0/995993434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e3529f800 0x557e352b4630 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5addffb640 0 client.18327 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.354+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 >> 172.21.15.60:0/995993434 conn(0x557e3529d630 msgr2=0x557e352a76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.355+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 shutdown_connections 2024-07-24T22:00:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.355+0000 7f5aff17c740 1 -- 172.21.15.60:0/995993434 wait complete. 2024-07-24T22:00:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.356+0000 7f5aff17c740 20 client.18327 trim_cache size 0 max 16384 2024-07-24T22:00:20.358 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:00:20.358 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:00:20.358 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:00:20.358 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 msgr2=0x7f5ae00b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 0x7f5ae00b61e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5af0015ee0 tx=0x7f5ae8031db0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 msgr2=0x7f5af00109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5af80685c0 tx=0x7f5af8071820 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 shutdown_connections 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ae00b3d20 0x7f5ae00b61e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5af00a6f10 0x7f5af00109e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af00a4e50 0x7f5af00103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 --2- 172.21.15.60:0/3289235373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af00098b0 0x7f5af0016190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 >> 172.21.15.60:0/3289235373 conn(0x7f5af001a050 msgr2=0x7f5af00a8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.357+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 shutdown_connections 2024-07-24T22:00:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:20.358+0000 7f5af759e640 1 -- 172.21.15.60:0/3289235373 wait complete. 2024-07-24T22:00:21.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:00:21.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:00:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:21.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:21 smithi060 ceph-mon[33000]: pgmap v804: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 520 KiB/s wr, 55 op/s 2024-07-24T22:00:21.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:21 smithi123 ceph-mon[37143]: pgmap v804: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 520 KiB/s wr, 55 op/s 2024-07-24T22:00:21.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:21 smithi086 ceph-mon[37128]: pgmap v804: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 520 KiB/s wr, 55 op/s 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.496+0000 7f72f221e640 1 -- 172.21.15.60:0/2516465846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec10caa0 msgr2=0x7f72ec10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.496+0000 7f72f221e640 1 --2- 172.21.15.60:0/2516465846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec10caa0 0x7f72ec10ef10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009ca0 tx=0x7f72e402a8c0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 -- 172.21.15.60:0/2516465846 shutdown_connections 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 --2- 172.21.15.60:0/2516465846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec10caa0 0x7f72ec10ef10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 --2- 172.21.15.60:0/2516465846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 0x7f72ec10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 --2- 172.21.15.60:0/2516465846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec1034f0 0x7f72ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 -- 172.21.15.60:0/2516465846 >> 172.21.15.60:0/2516465846 conn(0x7f72ec0fd0c0 msgr2=0x7f72ec0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 -- 172.21.15.60:0/2516465846 shutdown_connections 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 -- 172.21.15.60:0/2516465846 wait complete. 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 Processor -- start 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.497+0000 7f72f221e640 1 -- start start 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 0x7f72ec199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec10caa0 0x7f72ec19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec112700 con 0x7f72ec1034f0 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec112580 con 0x7f72ec103ef0 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72f221e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec112880 con 0x7f72ec10caa0 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48244/0 (socket says 172.21.15.60:48244) 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 -- 172.21.15.60:0/3916985924 learned_addr learned my addr 172.21.15.60:0/3916985924 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72ebfff640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec10caa0 0x7f72ec19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72e3fff640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 0x7f72ec199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 -- 172.21.15.60:0/3916985924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec10caa0 msgr2=0x7f72ec19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec10caa0 0x7f72ec19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 -- 172.21.15.60:0/3916985924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 msgr2=0x7f72ec199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 0x7f72ec199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 -- 172.21.15.60:0/3916985924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec19ece0 con 0x7f72ec1034f0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72eb7fe640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e9f0 tx=0x7f72dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.498+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00cd80 con 0x7f72ec1034f0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.499+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec19efd0 con 0x7f72ec1034f0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.499+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc0054d0 con 0x7f72ec1034f0 2024-07-24T22:00:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.499+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc010690 con 0x7f72ec1034f0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.499+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec101250 con 0x7f72ec1034f0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.500+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec1038d0 con 0x7f72ec1034f0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.502+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f72dc004e30 con 0x7f72ec1034f0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.503+0000 7f72e97fa640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 0x7f72d80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.503+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f72dc0e46a0 con 0x7f72ec1034f0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.503+0000 7f72e3fff640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 0x7f72d80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.504+0000 7f72e3fff640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 0x7f72d80b6200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10f1b0 tx=0x7f72d4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.508+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f72dc0555e0 con 0x7f72ec1034f0 2024-07-24T22:00:23.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:23 smithi123 ceph-mon[37143]: pgmap v805: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 960 KiB/s wr, 95 op/s 2024-07-24T22:00:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:23 smithi086 ceph-mon[37128]: pgmap v805: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 960 KiB/s wr, 95 op/s 2024-07-24T22:00:23.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:23 smithi060 ceph-mon[33000]: pgmap v805: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 960 KiB/s wr, 95 op/s 2024-07-24T22:00:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.868+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72ec199d60 con 0x7f72ec1034f0 2024-07-24T22:00:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.869+0000 7f72e97fa640 1 -- 172.21.15.60:0/3916985924 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f72dc0ab950 con 0x7f72ec1034f0 2024-07-24T22:00:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:00:23.871 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:00:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:00:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 msgr2=0x7f72d80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 0x7f72d80b6200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10f1b0 tx=0x7f72d4005f90 comp rx=0 tx=0).stop 2024-07-24T22:00:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 msgr2=0x7f72ec198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e9f0 tx=0x7f72dc00edf0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 shutdown_connections 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72d80b3d40 0x7f72d80b6200 secure :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72ec10f1b0 tx=0x7f72d4005f90 comp rx=0 tx=0).stop 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.872+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ec10caa0 0x7f72ec19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.873+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec103ef0 0x7f72ec199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.873+0000 7f72f221e640 1 --2- 172.21.15.60:0/3916985924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72ec1034f0 0x7f72ec198c30 secure :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e9f0 tx=0x7f72dc00edf0 comp rx=0 tx=0).stop 2024-07-24T22:00:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.873+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 >> 172.21.15.60:0/3916985924 conn(0x7f72ec0fd0c0 msgr2=0x7f72ec0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:23.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.875+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 shutdown_connections 2024-07-24T22:00:23.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:23.875+0000 7f72f221e640 1 -- 172.21.15.60:0/3916985924 wait complete. 2024-07-24T22:00:24.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:24 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3916985924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:24.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:24 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3916985924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:24.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:24 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3916985924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:25.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:00:25.514 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:25.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:25 smithi123 ceph-mon[37143]: pgmap v806: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-24T22:00:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:25 smithi086 ceph-mon[37128]: pgmap v806: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-24T22:00:25.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:25 smithi060 ceph-mon[33000]: pgmap v806: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-07-24T22:00:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 -- 172.21.15.60:0/3654452710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f68107060 msgr2=0x7f1f681074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/3654452710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f68107060 0x7f1f681074a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009900 tx=0x7f1f5c02a410 comp rx=0 tx=0).stop 2024-07-24T22:00:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 -- 172.21.15.60:0/3654452710 shutdown_connections 2024-07-24T22:00:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/3654452710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f681079e0 0x7f1f6810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/3654452710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f68107060 0x7f1f681074a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/3654452710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68101650 0x7f1f68101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.536+0000 7f1f6efb0640 1 -- 172.21.15.60:0/3654452710 >> 172.21.15.60:0/3654452710 conn(0x7f1f680fd0c0 msgr2=0x7f1f680ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.538+0000 7f1f6efb0640 1 -- 172.21.15.60:0/3654452710 shutdown_connections 2024-07-24T22:00:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.538+0000 7f1f6efb0640 1 -- 172.21.15.60:0/3654452710 wait complete. 2024-07-24T22:00:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.539+0000 7f1f6efb0640 1 Processor -- start 2024-07-24T22:00:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 -- start start 2024-07-24T22:00:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f68101650 0x7f1f68197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68107060 0x7f1f68197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681116b0 con 0x7f1f681079e0 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68111530 con 0x7f1f68101650 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.541+0000 7f1f6efb0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68111830 con 0x7f1f68107060 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f68101650 0x7f1f68197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37164/0 (socket says 172.21.15.60:37164) 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 -- 172.21.15.60:0/684614483 learned_addr learned my addr 172.21.15.60:0/684614483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f67fff640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68107060 0x7f1f68197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 -- 172.21.15.60:0/684614483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68107060 msgr2=0x7f1f68197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68107060 0x7f1f68197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 -- 172.21.15.60:0/684614483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f68101650 msgr2=0x7f1f68197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f68101650 0x7f1f68197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.542+0000 7f1f6d54d640 1 -- 172.21.15.60:0/684614483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681a70b0 con 0x7f1f681079e0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.543+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f60013990 tx=0x7f1f60013e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.544+0000 7f1f65ffb640 1 -- 172.21.15.60:0/684614483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600110d0 con 0x7f1f681079e0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.544+0000 7f1f65ffb640 1 -- 172.21.15.60:0/684614483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f60011720 con 0x7f1f681079e0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.544+0000 7f1f65ffb640 1 -- 172.21.15.60:0/684614483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60015b50 con 0x7f1f681079e0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.544+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f68199a80 con 0x7f1f681079e0 2024-07-24T22:00:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.544+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f68199fc0 con 0x7f1f681079e0 2024-07-24T22:00:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 Processor -- start 2024-07-24T22:00:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 -- start start 2024-07-24T22:00:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea913a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea916370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea87e9b0 con 0x55b5ea907ff0 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea86deb0 con 0x55b5ea9076d0 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.549+0000 7f1f71361740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea871ef0 con 0x55b5ea913f80 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f65ffb640 1 -- 172.21.15.60:0/684614483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1f60015cf0 con 0x7f1f681079e0 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea913a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38842/0 (socket says 172.21.15.60:38842) 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea913a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37172/0 (socket says 172.21.15.60:37172) 2024-07-24T22:00:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 -- 172.21.15.60:0/932392748 learned_addr learned my addr 172.21.15.60:0/932392748 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:27.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 -- 172.21.15.60:0/932392748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 msgr2=0x55b5ea916370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea916370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 -- 172.21.15.60:0/932392748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 msgr2=0x55b5ea913a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea913a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.550+0000 7f1f67fff640 1 -- 172.21.15.60:0/932392748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ea7f7320 con 0x55b5ea9076d0 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.551+0000 7f1f67fff640 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009230 tx=0x7f1f5c004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd41530462abe4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.551+0000 7f1f65ffb640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 0x7f1f500b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.551+0000 7f1f4effd640 1 -- 172.21.15.60:0/932392748 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c041070 con 0x55b5ea9076d0 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.552+0000 7f1f4effd640 1 -- 172.21.15.60:0/932392748 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c003aa0 con 0x55b5ea9076d0 2024-07-24T22:00:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.552+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 0x7f1f500b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.552+0000 7f1f65ffb640 1 -- 172.21.15.60:0/684614483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f1f600ec2a0 con 0x7f1f681079e0 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 -- 172.21.15.60:0/932392748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 msgr2=0x55b5ea907ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009230 tx=0x7f1f5c004180 comp rx=0 tx=0).stop 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 0x7f1f500b63d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55b5ea9063e0 tx=0x7f1f5800b4d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 -- 172.21.15.60:0/932392748 shutdown_connections 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea916370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea913a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 --2- 172.21.15.60:0/932392748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea9076d0 0x55b5ea907ab0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 -- 172.21.15.60:0/932392748 >> 172.21.15.60:0/932392748 conn(0x55b5ea905500 msgr2=0x55b5ea9066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 -- 172.21.15.60:0/932392748 shutdown_connections 2024-07-24T22:00:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.553+0000 7f1f71361740 1 -- 172.21.15.60:0/932392748 wait complete. 2024-07-24T22:00:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.554+0000 7f1f71361740 1 -- start start 2024-07-24T22:00:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.554+0000 7f1f71361740 1 Processor -- start 2024-07-24T22:00:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f71361740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f71361740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea907ff0 0x55b5ea911410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37174/0 (socket says 172.21.15.60:37174) 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/612378228 learned_addr learned my addr 172.21.15.60:0/612378228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea911950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.555+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea6d1700 con 0x55b5ea9076d0 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea8105f0 con 0x55b5ea907ff0 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea927c00 con 0x55b5ea913f80 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f71361740 10 client.0 _ll_register_callbacks cb 0x55b5ea830960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea911950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 msgr2=0x55b5ea911950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea911950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/612378228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea907ff0 msgr2=0x55b5ea911410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea907ff0 0x55b5ea911410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.556+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ea803680 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.557+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5ea880900 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.557+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f58013fd0 tx=0x7f1f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e83b38136f84e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.557+0000 7f1f47fff640 10 client.18345 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.557+0000 7f1f47fff640 1 -- 172.21.15.60:0/612378228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801de60 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.557+0000 7f1f47fff640 1 -- 172.21.15.60:0/612378228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f1f5801e820 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f47fff640 1 -- 172.21.15.60:0/612378228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1f5801ea20 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 10 client.18345 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55b5ea5ce650 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f47fff640 1 -- 172.21.15.60:0/612378228 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1f5801ebc0 con 0x55b5ea9076d0 2024-07-24T22:00:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 10 client.18345 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 10 client.18345 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 20 client.18345 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea934a90 0x55b5ea936f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 4 client.18345 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.558+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ea87c3a0 con 0x55b5ea934a90 2024-07-24T22:00:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.559+0000 7f1f67fff640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea934a90 0x55b5ea936f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.560+0000 7f1f67fff640 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea934a90 0x55b5ea936f50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecf4ad96b391f632 server_cookie=832caa65c61d00b in_seq=0 out_seq=0 2024-07-24T22:00:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.560+0000 7f1f47fff640 10 client.18345 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:00:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f47fff640 1 -- 172.21.15.60:0/612378228 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5ea87c3a0 con 0x55b5ea934a90 2024-07-24T22:00:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f47fff640 10 client.18345 handle_command_reply: tid=0 2024-07-24T22:00:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 client.18345 shutdown 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 msgr2=0x55b5ea910ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f58013fd0 tx=0x7f1f580046c0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 shutdown_connections 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea934a90 0x55b5ea936f50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f47fff640 0 client.18345 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea911950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea907ff0 0x55b5ea911410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.561+0000 7f1f71361740 1 --2- 172.21.15.60:0/612378228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea9076d0 0x55b5ea910ed0 secure :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f58013fd0 tx=0x7f1f580046c0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.562+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 >> 172.21.15.60:0/612378228 conn(0x55b5ea905500 msgr2=0x55b5ea90b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.562+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 shutdown_connections 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.562+0000 7f1f71361740 1 -- 172.21.15.60:0/612378228 wait complete. 2024-07-24T22:00:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.562+0000 7f1f71361740 20 client.18345 trim_cache size 0 max 16384 2024-07-24T22:00:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.585+0000 7f1f71361740 1 Processor -- start 2024-07-24T22:00:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.585+0000 7f1f71361740 1 -- start start 2024-07-24T22:00:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea93cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea910220 0x55b5ea90e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f6cd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37190/0 (socket says 172.21.15.60:37190) 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/3783764986 learned_addr learned my addr 172.21.15.60:0/3783764986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54011280 con 0x55b5ea907ff0 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400ea90 con 0x55b5ea910220 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400c900 con 0x55b5ea913f80 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.586+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea910220 0x55b5ea90e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f67fff640 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea93cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/3783764986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 msgr2=0x55b5ea93cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea93cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/3783764986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea910220 msgr2=0x55b5ea90e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea910220 0x55b5ea90e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 -- 172.21.15.60:0/3783764986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ea9136b0 con 0x55b5ea907ff0 2024-07-24T22:00:27.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.587+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f58004700 tx=0x7f1f58020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc695086fe609f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.591+0000 7f1f4e7fc640 1 -- 172.21.15.60:0/3783764986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58041070 con 0x55b5ea907ff0 2024-07-24T22:00:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.591+0000 7f1f4e7fc640 1 -- 172.21.15.60:0/3783764986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b5ea880900 con 0x55b5ea907ff0 2024-07-24T22:00:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.591+0000 7f1f4e7fc640 1 -- 172.21.15.60:0/3783764986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801e880 con 0x55b5ea907ff0 2024-07-24T22:00:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.592+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 msgr2=0x55b5ea93c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.592+0000 7f1f71361740 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f58004700 tx=0x7f1f58020070 comp rx=0 tx=0).stop 2024-07-24T22:00:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.594+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 shutdown_connections 2024-07-24T22:00:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.594+0000 7f1f71361740 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea910220 0x55b5ea90e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.594+0000 7f1f71361740 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea913f80 0x55b5ea93cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.594+0000 7f1f71361740 1 --2- 172.21.15.60:0/3783764986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea907ff0 0x55b5ea93c8b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.594+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 >> 172.21.15.60:0/3783764986 conn(0x55b5ea905500 msgr2=0x55b5ea912d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.596+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 shutdown_connections 2024-07-24T22:00:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.597+0000 7f1f71361740 1 -- 172.21.15.60:0/3783764986 wait complete. 2024-07-24T22:00:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.597+0000 7f1f71361740 1 -- start start 2024-07-24T22:00:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.598+0000 7f1f71361740 1 Processor -- start 2024-07-24T22:00:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.598+0000 7f1f71361740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea907ff0 0x55b5ea92d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.598+0000 7f1f71361740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.598+0000 7f1f71361740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea913f80 0x55b5ea929a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f71361740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400cf50 con 0x55b5ea910220 2024-07-24T22:00:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f71361740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400e3c0 con 0x55b5ea913f80 2024-07-24T22:00:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f71361740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5400d620 con 0x55b5ea907ff0 2024-07-24T22:00:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f71361740 10 client.0 _ll_register_callbacks cb 0x7f1f5800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:27.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37194/0 (socket says 172.21.15.60:37194) 2024-07-24T22:00:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f67fff640 1 -- 172.21.15.60:0/2340755939 learned_addr learned my addr 172.21.15.60:0/2340755939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.599+0000 7f1f6d54d640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea913f80 0x55b5ea929a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 -- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea907ff0 msgr2=0x55b5ea92d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea907ff0 0x55b5ea92d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea907ff0 0x55b5ea92d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 -- 172.21.15.60:0/2340755939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea913f80 msgr2=0x55b5ea929a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea913f80 0x55b5ea929a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 -- 172.21.15.60:0/2340755939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ea92a280 con 0x55b5ea910220 2024-07-24T22:00:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f67fff640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x55b5ea904e10 tx=0x7f1f5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9088b05139beca36 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f47fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c056070 con 0x55b5ea910220 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.600+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5ea880900 con 0x55b5ea910220 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.601+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f1f5c031cb0 con 0x55b5ea910220 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.601+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1f5c060050 con 0x55b5ea910220 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.601+0000 7f1f71361740 10 client.18351 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.601+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b5ea9a0600 con 0x55b5ea910220 2024-07-24T22:00:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.601+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c003aa0 con 0x55b5ea910220 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1f5c03a040 con 0x55b5ea910220 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 10 client.18351 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 10 client.18351 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 20 client.18351 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea9a0990 0x55b5ea9a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 4 client.18351 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5ea880900 con 0x55b5ea9a0990 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.602+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea9a0990 0x55b5ea9a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.603+0000 7f1f6cd4c640 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea9a0990 0x55b5ea9a2d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db28216fe2b96baa server_cookie=266d00ce62ec5042 in_seq=0 out_seq=0 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.603+0000 7f1f47fff640 10 client.18351 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f47fff640 1 -- 172.21.15.60:0/2340755939 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5ea880900 con 0x55b5ea9a0990 2024-07-24T22:00:27.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f47fff640 10 client.18351 handle_command_reply: tid=0 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f71361740 1 client.18351 shutdown 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 msgr2=0x55b5ea92d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x55b5ea904e10 tx=0x7f1f5c000bc0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 shutdown_connections 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.604+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b5ea9a0990 0x55b5ea9a2d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f47fff640 0 client.18351 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5ea913f80 0x55b5ea929a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5ea910220 0x55b5ea92d8a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 --2- 172.21.15.60:0/2340755939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5ea907ff0 0x55b5ea92d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 >> 172.21.15.60:0/2340755939 conn(0x55b5ea905500 msgr2=0x55b5ea90a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 shutdown_connections 2024-07-24T22:00:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 1 -- 172.21.15.60:0/2340755939 wait complete. 2024-07-24T22:00:27.611 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.605+0000 7f1f71361740 20 client.18351 trim_cache size 0 max 16384 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 msgr2=0x7f1f500b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 0x7f1f500b63d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55b5ea9063e0 tx=0x7f1f5800b4d0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 msgr2=0x7f1f681982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f60013990 tx=0x7f1f60013e50 comp rx=0 tx=0).stop 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 shutdown_connections 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1f500b3f10 0x7f1f500b63d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f681079e0 0x7f1f681982f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f68107060 0x7f1f68197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 --2- 172.21.15.60:0/684614483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f68101650 0x7f1f68197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:27.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.606+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 >> 172.21.15.60:0/684614483 conn(0x7f1f680fd0c0 msgr2=0x7f1f680ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:27.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.608+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 shutdown_connections 2024-07-24T22:00:27.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:27.609+0000 7f1f6efb0640 1 -- 172.21.15.60:0/684614483 wait complete. 2024-07-24T22:00:27.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:27 smithi123 ceph-mon[37143]: pgmap v807: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-07-24T22:00:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:27 smithi086 ceph-mon[37128]: pgmap v807: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-07-24T22:00:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:27 smithi060 ceph-mon[33000]: pgmap v807: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-07-24T22:00:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 -- 172.21.15.60:0/1763421695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481080a0 msgr2=0x7f88481084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 --2- 172.21.15.60:0/1763421695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481080a0 0x7f88481084e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8830009860 tx=0x7f883002a380 comp rx=0 tx=0).stop 2024-07-24T22:00:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 -- 172.21.15.60:0/1763421695 shutdown_connections 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 --2- 172.21.15.60:0/1763421695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8848108a20 0x7f884810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 --2- 172.21.15.60:0/1763421695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481080a0 0x7f88481084e0 secure :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8830009860 tx=0x7f883002a380 comp rx=0 tx=0).stop 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 --2- 172.21.15.60:0/1763421695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88481026a0 0x7f8848102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 -- 172.21.15.60:0/1763421695 >> 172.21.15.60:0/1763421695 conn(0x7f88480fe0e0 msgr2=0x7f8848100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 -- 172.21.15.60:0/1763421695 shutdown_connections 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.397+0000 7f884dcb0640 1 -- 172.21.15.60:0/1763421695 wait complete. 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.398+0000 7f884dcb0640 1 Processor -- start 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 -- start start 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 0x7f8848074400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8848108a20 0x7f884806d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 0x7f884806e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8848112640 con 0x7f88481026a0 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88481124c0 con 0x7f884806db60 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f884dcb0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88481127c0 con 0x7f8848108a20 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f88477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 0x7f8848074400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f8847fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 0x7f884806e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f88477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 0x7f8848074400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37222/0 (socket says 172.21.15.60:37222) 2024-07-24T22:00:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.399+0000 7f88477fe640 1 -- 172.21.15.60:0/3756744868 learned_addr learned my addr 172.21.15.60:0/3756744868 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 -- 172.21.15.60:0/3756744868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8848108a20 msgr2=0x7f884806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8848108a20 0x7f884806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 -- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 msgr2=0x7f8848074400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 0x7f8848074400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 -- 172.21.15.60:0/3756744868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88480730e0 con 0x7f884806db60 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8847fff640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 0x7f884806e010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8840009fd0 tx=0x7f884000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.400+0000 7f8844ff9640 1 -- 172.21.15.60:0/3756744868 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884000ce20 con 0x7f884806db60 2024-07-24T22:00:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.401+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88480733d0 con 0x7f884806db60 2024-07-24T22:00:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.401+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8848073920 con 0x7f884806db60 2024-07-24T22:00:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.402+0000 7f8844ff9640 1 -- 172.21.15.60:0/3756744868 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88400137d0 con 0x7f884806db60 2024-07-24T22:00:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.402+0000 7f8844ff9640 1 -- 172.21.15.60:0/3756744868 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840005580 con 0x7f884806db60 2024-07-24T22:00:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.406+0000 7f8844ff9640 1 -- 172.21.15.60:0/3756744868 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f88400057e0 con 0x7f884806db60 2024-07-24T22:00:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.407+0000 7f8844ff9640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 0x7f88340b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.408+0000 7f8844ff9640 1 -- 172.21.15.60:0/3756744868 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f88400e8610 con 0x7f884806db60 2024-07-24T22:00:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.409+0000 7f88477fe640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 0x7f88340b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.413+0000 7f88477fe640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 0x7f88340b6300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f88380058c0 tx=0x7f8838009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.414+0000 7f8850a07740 1 Processor -- start 2024-07-24T22:00:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.415+0000 7f8850a07740 1 -- start start 2024-07-24T22:00:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.415+0000 7f8850a07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.415+0000 7f8850a07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b09f0 0x55ac496bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.415+0000 7f8850a07740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f88477fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f88477fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42306/0 (socket says 172.21.15.60:42306) 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f88477fe640 1 -- 172.21.15.60:0/2602342751 learned_addr learned my addr 172.21.15.60:0/2602342751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ac4961d280 con 0x55ac496bc980 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac4960c780 con 0x55ac496b09f0 2024-07-24T22:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.416+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac496107c0 con 0x55ac496b00d0 2024-07-24T22:00:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.418+0000 7f8846ffd640 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b09f0 0x55ac496bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.419+0000 7f8847fff640 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.421+0000 7f88477fe640 1 -- 172.21.15.60:0/2602342751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b09f0 msgr2=0x55ac496bc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.421+0000 7f88477fe640 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b09f0 0x55ac496bc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.421+0000 7f88477fe640 1 -- 172.21.15.60:0/2602342751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 msgr2=0x55ac496bed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.421+0000 7f88477fe640 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.421+0000 7f88477fe640 1 -- 172.21.15.60:0/2602342751 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac49370790 con 0x55ac496b00d0 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.422+0000 7f88477fe640 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8838005a50 tx=0x7f8838014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797aa18bde1f1b52 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.422+0000 7f883dffb640 1 -- 172.21.15.60:0/2602342751 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883801f620 con 0x55ac496b00d0 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.422+0000 7f883dffb640 1 -- 172.21.15.60:0/2602342751 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883801fc70 con 0x55ac496b00d0 2024-07-24T22:00:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.422+0000 7f883dffb640 1 -- 172.21.15.60:0/2602342751 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883801dcc0 con 0x55ac496b00d0 2024-07-24T22:00:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.423+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 msgr2=0x55ac496b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.423+0000 7f8850a07740 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8838005a50 tx=0x7f8838014000 comp rx=0 tx=0).stop 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.424+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 shutdown_connections 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.424+0000 7f8850a07740 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.424+0000 7f8850a07740 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b09f0 0x55ac496bc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.424+0000 7f8850a07740 1 --2- 172.21.15.60:0/2602342751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496b04b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.424+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 >> 172.21.15.60:0/2602342751 conn(0x55ac496adf00 msgr2=0x55ac496af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.430+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 shutdown_connections 2024-07-24T22:00:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.432+0000 7f8850a07740 1 -- 172.21.15.60:0/2602342751 wait complete. 2024-07-24T22:00:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.432+0000 7f8850a07740 1 -- start start 2024-07-24T22:00:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.433+0000 7f8850a07740 1 Processor -- start 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.433+0000 7f8850a07740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496b00d0 0x55ac496bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496bc980 0x55ac496b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ac49546ab0 con 0x55ac496b00d0 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac495949e0 con 0x55ac496b9a80 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8847fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8847fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38908/0 (socket says 172.21.15.60:38908) 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8847fff640 1 -- 172.21.15.60:0/591010925 learned_addr learned my addr 172.21.15.60:0/591010925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac496bf660 con 0x55ac496bc980 2024-07-24T22:00:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.434+0000 7f8850a07740 10 client.0 _ll_register_callbacks cb 0x55ac4931ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8846ffd640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496bc980 0x55ac496b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8847fff640 1 -- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496bc980 msgr2=0x55ac496b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8847fff640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496bc980 0x55ac496b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8847fff640 1 -- 172.21.15.60:0/591010925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496b00d0 msgr2=0x55ac496bc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8847fff640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496b00d0 0x55ac496bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.435+0000 7f8847fff640 1 -- 172.21.15.60:0/591010925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac493e9620 con 0x55ac496b9a80 2024-07-24T22:00:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.436+0000 7f8847fff640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88400af200 tx=0x7f88400af7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4693b249faeeb662 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.437+0000 7f8826ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.437+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400af270 con 0x55ac496b9a80 2024-07-24T22:00:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.437+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac495bbce0 con 0x55ac496b9a80 2024-07-24T22:00:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.438+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f8840019e00 con 0x55ac496b9a80 2024-07-24T22:00:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.439+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f88400b3870 con 0x55ac496b9a80 2024-07-24T22:00:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.439+0000 7f8850a07740 10 client.27725 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.439+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac496cb2b0 con 0x55ac496b9a80 2024-07-24T22:00:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.441+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ac495bbce0 con 0x55ac496b9a80 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.442+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f88400e81d0 con 0x55ac496b9a80 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.442+0000 7f8850a07740 10 client.27725 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.442+0000 7f8850a07740 10 client.27725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.443+0000 7f8850a07740 20 client.27725 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.443+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac496cccb0 0x55ac496cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.443+0000 7f8850a07740 4 client.27725 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.443+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4961f1d0 con 0x55ac496cccb0 2024-07-24T22:00:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.444+0000 7f88477fe640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac496cccb0 0x55ac496cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.447+0000 7f88477fe640 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac496cccb0 0x55ac496cf130 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6267a4cb7b83dd3 server_cookie=b6f5ea7b3a15c310 in_seq=0 out_seq=0 2024-07-24T22:00:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.447+0000 7f8826ffd640 10 client.27725 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.448+0000 7f8826ffd640 1 -- 172.21.15.60:0/591010925 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac4961f1d0 con 0x55ac496cccb0 2024-07-24T22:00:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.448+0000 7f8826ffd640 10 client.27725 handle_command_reply: tid=0 2024-07-24T22:00:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.449+0000 7f8850a07740 1 client.27725 shutdown 2024-07-24T22:00:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.449+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 msgr2=0x55ac496b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.449+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88400af200 tx=0x7f88400af7d0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:00:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 shutdown_connections 2024-07-24T22:00:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac496cccb0 0x55ac496cf130 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496b9f00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496bc980 0x55ac496b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 --2- 172.21.15.60:0/591010925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496b00d0 0x55ac496bc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 >> 172.21.15.60:0/591010925 conn(0x55ac496adf00 msgr2=0x55ac496b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8826ffd640 0 client.27725 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 shutdown_connections 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 1 -- 172.21.15.60:0/591010925 wait complete. 2024-07-24T22:00:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.450+0000 7f8850a07740 20 client.27725 trim_cache size 0 max 16384 2024-07-24T22:00:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 Processor -- start 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 -- start start 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b9a80 0x55ac496b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8828011270 con 0x55ac496bc980 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.484+0000 7f8850a07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f882800ea80 con 0x55ac496b00d0 2024-07-24T22:00:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.485+0000 7f8850a07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f882800c980 con 0x55ac496b9a80 2024-07-24T22:00:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.486+0000 7f88477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.486+0000 7f88477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38922/0 (socket says 172.21.15.60:38922) 2024-07-24T22:00:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.486+0000 7f88477fe640 1 -- 172.21.15.60:0/2396961910 learned_addr learned my addr 172.21.15.60:0/2396961910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f8846ffd640 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b9a80 0x55ac496b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f88477fe640 1 -- 172.21.15.60:0/2396961910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b9a80 msgr2=0x55ac496b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f88477fe640 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b9a80 0x55ac496b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f88477fe640 1 -- 172.21.15.60:0/2396961910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 msgr2=0x55ac496d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f88477fe640 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.487+0000 7f88477fe640 1 -- 172.21.15.60:0/2396961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac496ba810 con 0x55ac496b00d0 2024-07-24T22:00:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.488+0000 7f88477fe640 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8838000c00 tx=0x7f88380098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a8a21fe5b547289 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.488+0000 7f883d7fa640 1 -- 172.21.15.60:0/2396961910 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883801d5b0 con 0x55ac496b00d0 2024-07-24T22:00:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.489+0000 7f883d7fa640 1 -- 172.21.15.60:0/2396961910 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88380054d0 con 0x55ac496b00d0 2024-07-24T22:00:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 -- 172.21.15.60:0/2396961910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 msgr2=0x55ac496b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8838000c00 tx=0x7f88380098e0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 -- 172.21.15.60:0/2396961910 shutdown_connections 2024-07-24T22:00:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b9a80 0x55ac496b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 --2- 172.21.15.60:0/2396961910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b00d0 0x55ac496b7da0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.490+0000 7f8850a07740 1 -- 172.21.15.60:0/2396961910 >> 172.21.15.60:0/2396961910 conn(0x55ac496adf00 msgr2=0x55ac496aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.494+0000 7f8850a07740 1 -- 172.21.15.60:0/2396961910 shutdown_connections 2024-07-24T22:00:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.494+0000 7f8850a07740 1 -- 172.21.15.60:0/2396961910 wait complete. 2024-07-24T22:00:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.495+0000 7f8850a07740 1 -- start start 2024-07-24T22:00:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.495+0000 7f8850a07740 1 Processor -- start 2024-07-24T22:00:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f882800cf40 con 0x55ac496bc980 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f882800e3b0 con 0x55ac496b9a80 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f882800d610 con 0x55ac496b00d0 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8850a07740 10 client.0 _ll_register_callbacks cb 0x7f88400aecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8847fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8847fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37258/0 (socket says 172.21.15.60:37258) 2024-07-24T22:00:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8847fff640 1 -- 172.21.15.60:0/3685721312 learned_addr learned my addr 172.21.15.60:0/3685721312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.496+0000 7f8846ffd640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f88477fe640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 -- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 msgr2=0x55ac496c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 -- 172.21.15.60:0/3685721312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 msgr2=0x55ac496bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 -- 172.21.15.60:0/3685721312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac496c01f0 con 0x55ac496b9a80 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.497+0000 7f8846ffd640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496c50e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8830008000 tx=0x7f8830005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faf3f5cd6154a09 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.498+0000 7f8826ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.498+0000 7f88477fe640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.498+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830037040 con 0x55ac496b9a80 2024-07-24T22:00:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.498+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88380054d0 con 0x55ac496b9a80 2024-07-24T22:00:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.498+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f8830004ad0 con 0x55ac496b9a80 2024-07-24T22:00:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.499+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8830005c50 con 0x55ac496b9a80 2024-07-24T22:00:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.499+0000 7f8850a07740 10 client.27731 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.499+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac49736590 con 0x55ac496b9a80 2024-07-24T22:00:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.499+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88380054d0 con 0x55ac496b9a80 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.500+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f8830032cc0 con 0x55ac496b9a80 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 10 client.27731 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 10 client.27731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 20 client.27731 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac49736dc0 0x55ac497391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 4 client.27731 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:00:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac4961ac70 con 0x55ac49736dc0 2024-07-24T22:00:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f88477fe640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac49736dc0 0x55ac497391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.501+0000 7f88477fe640 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac49736dc0 0x55ac497391e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b42dac8774cc26 server_cookie=b64ea996f38aa36 in_seq=0 out_seq=0 2024-07-24T22:00:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.502+0000 7f8826ffd640 10 client.27731 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8826ffd640 1 -- 172.21.15.60:0/3685721312 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac4961ac70 con 0x55ac49736dc0 2024-07-24T22:00:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8826ffd640 10 client.27731 handle_command_reply: tid=0 2024-07-24T22:00:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 client.27731 shutdown 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 msgr2=0x55ac496c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496c50e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8830008000 tx=0x7f8830005580 comp rx=0 tx=0).stop 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 shutdown_connections 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ac49736dc0 0x55ac497391e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac496bc980 0x55ac496bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac496b9a80 0x55ac496c50e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 --2- 172.21.15.60:0/3685721312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac496b00d0 0x55ac496c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 >> 172.21.15.60:0/3685721312 conn(0x55ac496adf00 msgr2=0x55ac496b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.891+0000 7f8826ffd640 0 client.27731 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.892+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 shutdown_connections 2024-07-24T22:00:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.892+0000 7f8850a07740 1 -- 172.21.15.60:0/3685721312 wait complete. 2024-07-24T22:00:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.892+0000 7f8850a07740 20 client.27731 trim_cache size 0 max 16384 2024-07-24T22:00:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:00:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:00:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:00:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:00:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 msgr2=0x7f88340b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 0x7f88340b6300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f88380058c0 tx=0x7f8838009260 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 msgr2=0x7f884806e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 0x7f884806e010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8840009fd0 tx=0x7f884000ef10 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 shutdown_connections 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f88340b3e40 0x7f88340b6300 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f88380058c0 tx=0x7f8838009260 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f884806db60 0x7f884806e010 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8848108a20 0x7f884806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 --2- 172.21.15.60:0/3756744868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88481026a0 0x7f8848074400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.898+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 >> 172.21.15.60:0/3756744868 conn(0x7f88480fe0e0 msgr2=0x7f88481004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.900+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 shutdown_connections 2024-07-24T22:00:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:28.901+0000 7f884dcb0640 1 -- 172.21.15.60:0/3756744868 wait complete. 2024-07-24T22:00:28.905 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:29.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:29 smithi060 ceph-mon[33000]: pgmap v808: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 120 op/s 2024-07-24T22:00:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:29 smithi086 ceph-mon[37128]: pgmap v808: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 120 op/s 2024-07-24T22:00:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:29 smithi123 ceph-mon[37143]: pgmap v808: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 120 op/s 2024-07-24T22:00:31.720 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:31 smithi060 ceph-mon[33000]: pgmap v809: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 134 op/s 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.751+0000 7fc0f6c44640 1 -- 172.21.15.60:0/3635158926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 msgr2=0x7fc0f011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.751+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/3635158926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f011de80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009cd0 tx=0x7fc0e802a910 comp rx=0 tx=0).stop 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.751+0000 7fc0f6c44640 1 -- 172.21.15.60:0/3635158926 shutdown_connections 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/3635158926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f011de80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/3635158926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f0109f20 0x7fc0f010a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/3635158926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109b40 0x7fc0f0110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 -- 172.21.15.60:0/3635158926 >> 172.21.15.60:0/3635158926 conn(0x7fc0f006c500 msgr2=0x7fc0f006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 -- 172.21.15.60:0/3635158926 shutdown_connections 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.752+0000 7fc0f6c44640 1 -- 172.21.15.60:0/3635158926 wait complete. 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 Processor -- start 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 -- start start 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f0109b40 0x7fc0f01a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f20 0x7fc0f01a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0121430 con 0x7fc0f0109b40 2024-07-24T22:00:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01212b0 con 0x7fc0f0109f20 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f6c44640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01215b0 con 0x7fc0f011ba90 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.753+0000 7fc0f51e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42340/0 (socket says 172.21.15.60:42340) 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 -- 172.21.15.60:0/426753123 learned_addr learned my addr 172.21.15.60:0/426753123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 -- 172.21.15.60:0/426753123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f20 msgr2=0x7fc0f01a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f20 0x7fc0f01a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 -- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f0109b40 msgr2=0x7fc0f01a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f0109b40 0x7fc0f01a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 -- 172.21.15.60:0/426753123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f01ad970 con 0x7fc0f011ba90 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.754+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802f3b0 tx=0x7fc0e8032ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.755+0000 7fc0edffb640 1 -- 172.21.15.60:0/426753123 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8003c70 con 0x7fc0f011ba90 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.755+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f01adc00 con 0x7fc0f011ba90 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.755+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f007ada0 con 0x7fc0f011ba90 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.755+0000 7fc0edffb640 1 -- 172.21.15.60:0/426753123 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e8003e10 con 0x7fc0f011ba90 2024-07-24T22:00:31.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.756+0000 7fc0edffb640 1 -- 172.21.15.60:0/426753123 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8030bc0 con 0x7fc0f011ba90 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.759+0000 7fc0f8ff5740 1 Processor -- start 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 -- start start 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaeac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0edffb640 1 -- 172.21.15.60:0/426753123 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc0e803c410 con 0x7fc0f011ba90 2024-07-24T22:00:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a3bda619b0 con 0x55a3bdaea850 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3bda50eb0 con 0x55a3bdaeb170 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3bda54ef0 con 0x55a3bdaf7100 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f49e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f49e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38962/0 (socket says 172.21.15.60:38962) 2024-07-24T22:00:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.760+0000 7fc0f49e0640 1 -- 172.21.15.60:0/40076419 learned_addr learned my addr 172.21.15.60:0/40076419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 -- 172.21.15.60:0/40076419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 msgr2=0x55a3bdaf94f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 -- 172.21.15.60:0/40076419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 msgr2=0x55a3bdaeac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaeac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 -- 172.21.15.60:0/40076419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bd9da320 con 0x55a3bdaeb170 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.761+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000bb10 tx=0x7fc0e000d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=410910506ff8f06e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.762+0000 7fc0edffb640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 0x7fc0d80b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.762+0000 7fc0d6ffd640 1 -- 172.21.15.60:0/40076419 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0016070 con 0x55a3bdaeb170 2024-07-24T22:00:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.762+0000 7fc0d6ffd640 1 -- 172.21.15.60:0/40076419 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e000dce0 con 0x55a3bdaeb170 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.762+0000 7fc0d6ffd640 1 -- 172.21.15.60:0/40076419 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0010460 con 0x55a3bdaeb170 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.762+0000 7fc0edffb640 1 -- 172.21.15.60:0/426753123 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc0e8105070 con 0x7fc0f011ba90 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0effff640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 0x7fc0d80b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/40076419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 msgr2=0x55a3bdaf6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000bb10 tx=0x7fc0e000d2e0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/40076419 shutdown_connections 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf6bc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/40076419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaeac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/40076419 >> 172.21.15.60:0/40076419 conn(0x55a3bdae8680 msgr2=0x55a3bdae9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.763+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/40076419 shutdown_connections 2024-07-24T22:00:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.764+0000 7fc0effff640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 0x7fc0d80b6480 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc0f011e100 tx=0x7fc0e4009250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.767+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/40076419 wait complete. 2024-07-24T22:00:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.767+0000 7fc0f8ff5740 1 -- start start 2024-07-24T22:00:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.767+0000 7fc0f8ff5740 1 Processor -- start 2024-07-24T22:00:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaf4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a3bd8b4700 con 0x55a3bdaea850 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3bd9f35f0 con 0x55a3bdaeb170 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3bdaf9e20 con 0x55a3bdaf7100 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.768+0000 7fc0f8ff5740 10 client.0 _ll_register_callbacks cb 0x55a3bda13960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.769+0000 7fc0f51e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.769+0000 7fc0f51e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42362/0 (socket says 172.21.15.60:42362) 2024-07-24T22:00:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.769+0000 7fc0f51e1640 1 -- 172.21.15.60:0/1530922936 learned_addr learned my addr 172.21.15.60:0/1530922936 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.769+0000 7fc0effff640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.769+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaf4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.770+0000 7fc0f49e0640 1 -- 172.21.15.60:0/1530922936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 msgr2=0x55a3bdaf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.770+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.770+0000 7fc0f49e0640 1 -- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 msgr2=0x55a3bdaf4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.770+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.770+0000 7fc0f49e0640 1 -- 172.21.15.60:0/1530922936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bd9e6680 con 0x55a3bdaea850 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.771+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaf4030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005cd0 tx=0x7fc0e00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f240b369228d4e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.771+0000 7fc0cffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.772+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0016030 con 0x55a3bdaea850 2024-07-24T22:00:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.772+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e000dce0 con 0x55a3bdaea850 2024-07-24T22:00:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.772+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc0e0004250 con 0x55a3bdaea850 2024-07-24T22:00:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.773+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0004470 con 0x55a3bdaea850 2024-07-24T22:00:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.774+0000 7fc0f8ff5740 10 client.18366 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.774+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a3bd7b1650 con 0x55a3bdaea850 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.776+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e000dce0 con 0x55a3bdaea850 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.777+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc0e00037b0 con 0x55a3bdaea850 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 10 client.18366 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 10 client.18366 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 20 client.18366 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb18050 0x55a3bdb1a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 4 client.18366 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.778+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3bda63900 con 0x55a3bdb18050 2024-07-24T22:00:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.783+0000 7fc0effff640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb18050 0x55a3bdb1a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.786+0000 7fc0effff640 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb18050 0x55a3bdb1a4b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c907026eb26284a server_cookie=dc443b0695f324b in_seq=0 out_seq=0 2024-07-24T22:00:31.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.787+0000 7fc0cffff640 10 client.18366 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:00:31.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.789+0000 7fc0cffff640 1 -- 172.21.15.60:0/1530922936 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3bda63900 con 0x55a3bdb18050 2024-07-24T22:00:31.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.789+0000 7fc0cffff640 10 client.18366 handle_command_reply: tid=0 2024-07-24T22:00:31.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.789+0000 7fc0f8ff5740 1 client.18366 shutdown 2024-07-24T22:00:31.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.791+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 msgr2=0x55a3bdaf4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.791+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaf4030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005cd0 tx=0x7fc0e00049d0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 shutdown_connections 2024-07-24T22:00:31.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb18050 0x55a3bdb1a4b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdaf4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/1530922936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaf4030 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 >> 172.21.15.60:0/1530922936 conn(0x55a3bdae8680 msgr2=0x55a3bdaee220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:31 smithi086 ceph-mon[37128]: pgmap v809: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 134 op/s 2024-07-24T22:00:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.793+0000 7fc0cffff640 0 client.18366 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:00:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.794+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 shutdown_connections 2024-07-24T22:00:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.794+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/1530922936 wait complete. 2024-07-24T22:00:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.794+0000 7fc0f8ff5740 20 client.18366 trim_cache size 0 max 16384 2024-07-24T22:00:31.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.823+0000 7fc0f8ff5740 1 Processor -- start 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 -- start start 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 0x55a3bdaf5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdb1fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaf7100 0x55a3bdb20300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc0111b0 con 0x55a3bdaf7100 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc00e9c0 con 0x55a3bdaeb170 2024-07-24T22:00:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.824+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc00c8c0 con 0x55a3bdaea850 2024-07-24T22:00:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0f49e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 0x55a3bdaf5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0f49e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 0x55a3bdaf5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42364/0 (socket says 172.21.15.60:42364) 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0f49e0640 1 -- 172.21.15.60:0/832564 learned_addr learned my addr 172.21.15.60:0/832564 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdb1fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 -- 172.21.15.60:0/832564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 msgr2=0x55a3bdaf5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 0x55a3bdaf5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 -- 172.21.15.60:0/832564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaf7100 msgr2=0x55a3bdb20300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaf7100 0x55a3bdb20300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 -- 172.21.15.60:0/832564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bdaeafa0 con 0x55a3bdaeb170 2024-07-24T22:00:31.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.827+0000 7fc0effff640 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdb1fdc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a3bdb1ad00 tx=0x7fc0e400f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169a3ba27519bd4f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.830+0000 7fc0d67fc640 1 -- 172.21.15.60:0/832564 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400ce90 con 0x55a3bdaeb170 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.830+0000 7fc0d67fc640 1 -- 172.21.15.60:0/832564 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e402a070 con 0x55a3bdaeb170 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.830+0000 7fc0d67fc640 1 -- 172.21.15.60:0/832564 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400d820 con 0x55a3bdaeb170 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.831+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/832564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 msgr2=0x55a3bdb1fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.831+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdb1fdc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a3bdb1ad00 tx=0x7fc0e400f000 comp rx=0 tx=0).stop 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.833+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/832564 shutdown_connections 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.833+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaf7100 0x55a3bdb20300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.833+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaeb170 0x55a3bdb1fdc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.833+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/832564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaea850 0x55a3bdaf5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.833+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/832564 >> 172.21.15.60:0/832564 conn(0x55a3bdae8680 msgr2=0x55a3bdaf5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.834+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/832564 shutdown_connections 2024-07-24T22:00:31.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.834+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/832564 wait complete. 2024-07-24T22:00:31.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.834+0000 7fc0f8ff5740 1 -- start start 2024-07-24T22:00:31.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.834+0000 7fc0f8ff5740 1 Processor -- start 2024-07-24T22:00:31.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 0x55a3bdaff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc00ce80 con 0x55a3bdaea850 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc00e2f0 con 0x55a3bdaf7100 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc00d550 con 0x55a3bdaeb170 2024-07-24T22:00:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f8ff5740 10 client.0 _ll_register_callbacks cb 0x7fc0e000b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 0x55a3bdaff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 0x55a3bdaff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42368/0 (socket says 172.21.15.60:42368) 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0effff640 1 -- 172.21.15.60:0/2123904464 learned_addr learned my addr 172.21.15.60:0/2123904464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.838+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 -- 172.21.15.60:0/2123904464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 msgr2=0x55a3bdaff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 0x55a3bdaff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 -- 172.21.15.60:0/2123904464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 msgr2=0x55a3bdaff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:31.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 -- 172.21.15.60:0/2123904464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bdaf0ba0 con 0x55a3bdaf7100 2024-07-24T22:00:31.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0f51e1640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf0660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80c7d50 tx=0x7fc0e80cc3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494829df9c4acadf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:31.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0cffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:31.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.839+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80e6020 con 0x55a3bdaf7100 2024-07-24T22:00:31.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.840+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e402a070 con 0x55a3bdaf7100 2024-07-24T22:00:31.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.840+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fc0e80c82c0 con 0x55a3bdaf7100 2024-07-24T22:00:31.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.840+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0e8103e60 con 0x55a3bdaf7100 2024-07-24T22:00:31.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.840+0000 7fc0f8ff5740 10 client.27746 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:31.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.840+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a3bdb82640 con 0x55a3bdaf7100 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.841+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e402a070 con 0x55a3bdaf7100 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.841+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc0e804c030 con 0x55a3bdaf7100 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 10 client.27746 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 10 client.27746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 20 client.27746 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb82dd0 0x55a3bdb851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 4 client.27746 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3bda5f3a0 con 0x55a3bdb82dd0 2024-07-24T22:00:31.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.842+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb82dd0 0x55a3bdb851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:31.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.844+0000 7fc0f49e0640 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb82dd0 0x55a3bdb851f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5556f0c82893c7d server_cookie=56c640fb08909929 in_seq=0 out_seq=0 2024-07-24T22:00:31.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.844+0000 7fc0cffff640 10 client.27746 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:00:31.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.844+0000 7fc0cffff640 1 -- 172.21.15.60:0/2123904464 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3bda5f3a0 con 0x55a3bdb82dd0 2024-07-24T22:00:31.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.844+0000 7fc0cffff640 10 client.27746 handle_command_reply: tid=0 2024-07-24T22:00:31.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.844+0000 7fc0f8ff5740 1 client.27746 shutdown 2024-07-24T22:00:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 msgr2=0x55a3bdaf0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf0660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80c7d50 tx=0x7fc0e80cc3d0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 shutdown_connections 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55a3bdb82dd0 0x55a3bdb851f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3bdaf7100 0x55a3bdaf0660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3bdaeb170 0x55a3bdaff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 --2- 172.21.15.60:0/2123904464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a3bdaea850 0x55a3bdaff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 >> 172.21.15.60:0/2123904464 conn(0x55a3bdae8680 msgr2=0x55a3bdaf5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.846+0000 7fc0cffff640 0 client.27746 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.847+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 shutdown_connections 2024-07-24T22:00:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.847+0000 7fc0f8ff5740 1 -- 172.21.15.60:0/2123904464 wait complete. 2024-07-24T22:00:31.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.848+0000 7fc0f8ff5740 20 client.27746 trim_cache size 0 max 16384 2024-07-24T22:00:31.850 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:00:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 msgr2=0x7fc0d80b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 0x7fc0d80b6480 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc0f011e100 tx=0x7fc0e4009250 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 msgr2=0x7fc0f01ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802f3b0 tx=0x7fc0e8032ff0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 shutdown_connections 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0d80b3fc0 0x7fc0d80b6480 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.849+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f011ba90 0x7fc0f01ad1a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.850+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f20 0x7fc0f01a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.850+0000 7fc0f6c44640 1 --2- 172.21.15.60:0/426753123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f0109b40 0x7fc0f01a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.850+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 >> 172.21.15.60:0/426753123 conn(0x7fc0f006c500 msgr2=0x7fc0f011bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.850+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 shutdown_connections 2024-07-24T22:00:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:31.850+0000 7fc0f6c44640 1 -- 172.21.15.60:0/426753123 wait complete. 2024-07-24T22:00:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:31 smithi123 ceph-mon[37143]: pgmap v809: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 134 op/s 2024-07-24T22:00:32.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:00:33.349 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:33.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:33 smithi060 ceph-mon[33000]: pgmap v810: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 146 op/s 2024-07-24T22:00:33.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:33 smithi086 ceph-mon[37128]: pgmap v810: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 146 op/s 2024-07-24T22:00:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:33 smithi123 ceph-mon[37143]: pgmap v810: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 146 op/s 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 -- 172.21.15.60:0/3395049474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18501129c0 msgr2=0x7f1850112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 --2- 172.21.15.60:0/3395049474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18501129c0 0x7f1850112e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1844009900 tx=0x7f184402a440 comp rx=0 tx=0).stop 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 -- 172.21.15.60:0/3395049474 shutdown_connections 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 --2- 172.21.15.60:0/3395049474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f1850119b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 --2- 172.21.15.60:0/3395049474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18501129c0 0x7f1850112e00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 --2- 172.21.15.60:0/3395049474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f185010cfb0 0x7f185010d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.559+0000 7f18554e1640 1 -- 172.21.15.60:0/3395049474 >> 172.21.15.60:0/3395049474 conn(0x7f185006c2d0 msgr2=0x7f185006c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.565+0000 7f18554e1640 1 -- 172.21.15.60:0/3395049474 shutdown_connections 2024-07-24T22:00:35.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.571+0000 7f18554e1640 1 -- 172.21.15.60:0/3395049474 wait complete. 2024-07-24T22:00:35.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.571+0000 7f18554e1640 1 Processor -- start 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 -- start start 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 0x7f185006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f185006eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f185011d030 con 0x7f185010cfb0 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f185011ceb0 con 0x7f1850113340 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f18554e1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f185011d1b0 con 0x7f18501129c0 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42382/0 (socket says 172.21.15.60:42382) 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f184f7fe640 1 -- 172.21.15.60:0/3537822113 learned_addr learned my addr 172.21.15.60:0/3537822113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f185006eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.572+0000 7f184ffff640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 0x7f185006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.573+0000 7f184f7fe640 1 -- 172.21.15.60:0/3537822113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 msgr2=0x7f185006eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.573+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f185006eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.573+0000 7f184f7fe640 1 -- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 msgr2=0x7f185006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.573+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 0x7f185006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f184f7fe640 1 -- 172.21.15.60:0/3537822113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501a3230 con 0x7f18501129c0 2024-07-24T22:00:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f185006eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f184402a9b0 tx=0x7f1844003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f184ffff640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 0x7f185006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f184d7fa640 1 -- 172.21.15.60:0/3537822113 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184403f070 con 0x7f18501129c0 2024-07-24T22:00:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a34c0 con 0x7f18501129c0 2024-07-24T22:00:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.574+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a39f0 con 0x7f18501129c0 2024-07-24T22:00:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.575+0000 7f184d7fa640 1 -- 172.21.15.60:0/3537822113 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1844009520 con 0x7f18501129c0 2024-07-24T22:00:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.575+0000 7f184d7fa640 1 -- 172.21.15.60:0/3537822113 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844035030 con 0x7f18501129c0 2024-07-24T22:00:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.580+0000 7f1858af4740 1 Processor -- start 2024-07-24T22:00:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 -- start start 2024-07-24T22:00:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f1511e0 0x55dd6f15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dd6f0c79b0 con 0x55dd6f15d170 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dd6f0b6eb0 con 0x55dd6f1508c0 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.581+0000 7f1858af4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd6f0baef0 con 0x55dd6f1511e0 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:39022/0 (socket says 172.21.15.60:39022) 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 -- 172.21.15.60:0/3573738981 learned_addr learned my addr 172.21.15.60:0/3573738981 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f184ffff640 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f1511e0 0x55dd6f15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 -- 172.21.15.60:0/3573738981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f1511e0 msgr2=0x55dd6f15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f1511e0 0x55dd6f15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 -- 172.21.15.60:0/3573738981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 msgr2=0x55dd6f15f560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f15f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.582+0000 7f1854ce0640 1 -- 172.21.15.60:0/3573738981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6f040320 con 0x55dd6f1508c0 2024-07-24T22:00:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.583+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fd0 tx=0x7f18480108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1ef58a0e99eb8b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.584+0000 7f18367fc640 1 -- 172.21.15.60:0/3573738981 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848011760 con 0x55dd6f1508c0 2024-07-24T22:00:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.584+0000 7f18367fc640 1 -- 172.21.15.60:0/3573738981 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848011db0 con 0x55dd6f1508c0 2024-07-24T22:00:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.584+0000 7f18367fc640 1 -- 172.21.15.60:0/3573738981 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800e3c0 con 0x55dd6f1508c0 2024-07-24T22:00:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.585+0000 7f1858af4740 1 -- 172.21.15.60:0/3573738981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 msgr2=0x55dd6f150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.585+0000 7f1858af4740 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fd0 tx=0x7f18480108e0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 -- 172.21.15.60:0/3573738981 shutdown_connections 2024-07-24T22:00:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f1511e0 0x55dd6f15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 --2- 172.21.15.60:0/3573738981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1508c0 0x55dd6f150ca0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 -- 172.21.15.60:0/3573738981 >> 172.21.15.60:0/3573738981 conn(0x55dd6f14e6f0 msgr2=0x55dd6f14f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f184d7fa640 1 -- 172.21.15.60:0/3537822113 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1844042070 con 0x7f18501129c0 2024-07-24T22:00:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.587+0000 7f1858af4740 1 -- 172.21.15.60:0/3573738981 shutdown_connections 2024-07-24T22:00:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.588+0000 7f1858af4740 1 -- 172.21.15.60:0/3573738981 wait complete. 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.588+0000 7f1858af4740 1 -- start start 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.588+0000 7f1858af4740 1 Processor -- start 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.589+0000 7f184d7fa640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 0x7f18380b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.589+0000 7f184d7fa640 1 -- 172.21.15.60:0/3537822113 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f1844104770 con 0x7f18501129c0 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f15a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15d170 0x55dd6f15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55dd6ef1a700 con 0x55dd6f1508c0 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dd6f0595f0 con 0x55dd6f1511e0 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd6f15fdf0 con 0x55dd6f15d170 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.590+0000 7f1858af4740 10 client.0 _ll_register_callbacks cb 0x55dd6f079960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37334/0 (socket says 172.21.15.60:37334) 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 -- 172.21.15.60:0/3534377546 learned_addr learned my addr 172.21.15.60:0/3534377546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 -- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15d170 msgr2=0x55dd6f15ab40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15d170 0x55dd6f15ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 -- 172.21.15.60:0/3534377546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 msgr2=0x55dd6f15a600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f15a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.591+0000 7f184ffff640 1 -- 172.21.15.60:0/3534377546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6f04c680 con 0x55dd6f1508c0 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.592+0000 7f184ffff640 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1840009f50 tx=0x7f184000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e7b1d6ecfcc626 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.592+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 0x7f18380b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.595+0000 7f182f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.595+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840014660 con 0x55dd6f1508c0 2024-07-24T22:00:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.595+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f1840014800 con 0x55dd6f1508c0 2024-07-24T22:00:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.595+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1848011db0 con 0x55dd6f1508c0 2024-07-24T22:00:35.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.596+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1840014a00 con 0x55dd6f1508c0 2024-07-24T22:00:35.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.596+0000 7f1858af4740 10 client.18378 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:35.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.596+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dd6ee17650 con 0x55dd6f1508c0 2024-07-24T22:00:35.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.597+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848011db0 con 0x55dd6f1508c0 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1840013490 con 0x55dd6f1508c0 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 10 client.18378 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 20 client.18378 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f16d260 0x55dd6f16f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 4 client.18378 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.598+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd6f0c9900 con 0x55dd6f16d260 2024-07-24T22:00:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.600+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f16d260 0x55dd6f16f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.605+0000 7f1854ce0640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 0x7f18380b6300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1848009e10 tx=0x7f1848005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.605+0000 7f182f7fe640 10 client.18378 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.605+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f16d260 0x55dd6f16f6e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8e4db9d2eb745e server_cookie=72dc5e2477e1a21d in_seq=0 out_seq=0 2024-07-24T22:00:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.606+0000 7f182f7fe640 1 -- 172.21.15.60:0/3534377546 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd6f0c9900 con 0x55dd6f16d260 2024-07-24T22:00:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.606+0000 7f182f7fe640 10 client.18378 handle_command_reply: tid=0 2024-07-24T22:00:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.606+0000 7f1858af4740 1 client.18378 shutdown 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 msgr2=0x55dd6f15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1840009f50 tx=0x7f184000f670 comp rx=0 tx=0).stop 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 shutdown_connections 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f16d260 0x55dd6f16f6e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15d170 0x55dd6f15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 --2- 172.21.15.60:0/3534377546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1508c0 0x55dd6f15a0c0 secure :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1840009f50 tx=0x7f184000f670 comp rx=0 tx=0).stop 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 >> 172.21.15.60:0/3534377546 conn(0x55dd6f14e6f0 msgr2=0x55dd6f154290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f182f7fe640 0 client.18378 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.611+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 shutdown_connections 2024-07-24T22:00:35.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.612+0000 7f1858af4740 1 -- 172.21.15.60:0/3534377546 wait complete. 2024-07-24T22:00:35.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.612+0000 7f1858af4740 20 client.18378 trim_cache size 0 max 16384 2024-07-24T22:00:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.639+0000 7f1858af4740 1 Processor -- start 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.639+0000 7f1858af4740 1 -- start start 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 0x55dd6f14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f15d170 0x55dd6f14df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0112f0 con 0x55dd6f1511e0 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00eb00 con 0x55dd6f15d170 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.640+0000 7f1858af4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00c970 con 0x55dd6f15be00 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.641+0000 7f1854ce0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.641+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f15d170 0x55dd6f14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.641+0000 7f1854ce0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42398/0 (socket says 172.21.15.60:42398) 2024-07-24T22:00:35.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.641+0000 7f1854ce0640 1 -- 172.21.15.60:0/2388754681 learned_addr learned my addr 172.21.15.60:0/2388754681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f184ffff640 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 0x55dd6f14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f1854ce0640 1 -- 172.21.15.60:0/2388754681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f15d170 msgr2=0x55dd6f14df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f1854ce0640 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f15d170 0x55dd6f14df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f1854ce0640 1 -- 172.21.15.60:0/2388754681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 msgr2=0x55dd6f14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f1854ce0640 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 0x55dd6f14fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f1854ce0640 1 -- 172.21.15.60:0/2388754681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6f14ee90 con 0x55dd6f15be00 2024-07-24T22:00:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.646+0000 7f184ffff640 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 0x55dd6f14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.647+0000 7f1854ce0640 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fa0 tx=0x7f184801bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af03d8eb0f1fb5a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.648+0000 7f1835ffb640 1 -- 172.21.15.60:0/2388754681 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184801b980 con 0x55dd6f15be00 2024-07-24T22:00:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.648+0000 7f1835ffb640 1 -- 172.21.15.60:0/2388754681 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848011c10 con 0x55dd6f15be00 2024-07-24T22:00:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.648+0000 7f1835ffb640 1 -- 172.21.15.60:0/2388754681 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480058a0 con 0x55dd6f15be00 2024-07-24T22:00:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.650+0000 7f1858af4740 1 -- 172.21.15.60:0/2388754681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 msgr2=0x55dd6f174fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.650+0000 7f1858af4740 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fa0 tx=0x7f184801bb20 comp rx=0 tx=0).stop 2024-07-24T22:00:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.651+0000 7f1858af4740 1 -- 172.21.15.60:0/2388754681 shutdown_connections 2024-07-24T22:00:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.651+0000 7f1858af4740 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f174fb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.651+0000 7f1858af4740 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f15d170 0x55dd6f14df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.651+0000 7f1858af4740 1 --2- 172.21.15.60:0/2388754681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f1511e0 0x55dd6f14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.651+0000 7f1858af4740 1 -- 172.21.15.60:0/2388754681 >> 172.21.15.60:0/2388754681 conn(0x55dd6f14e6f0 msgr2=0x55dd6f15b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.654+0000 7f1858af4740 1 -- 172.21.15.60:0/2388754681 shutdown_connections 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.654+0000 7f1858af4740 1 -- 172.21.15.60:0/2388754681 wait complete. 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.654+0000 7f1858af4740 1 -- start start 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.654+0000 7f1858af4740 1 Processor -- start 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00cfc0 con 0x55dd6f15d170 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00e430 con 0x55dd6f1511e0 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00d690 con 0x55dd6f15be00 2024-07-24T22:00:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f1858af4740 10 client.0 _ll_register_callbacks cb 0x7f1840009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:35.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42408/0 (socket says 172.21.15.60:42408) 2024-07-24T22:00:35.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.655+0000 7f184f7fe640 1 -- 172.21.15.60:0/3920973559 learned_addr learned my addr 172.21.15.60:0/3920973559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184ffff640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184f7fe640 1 -- 172.21.15.60:0/3920973559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 msgr2=0x55dd6f161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184f7fe640 1 -- 172.21.15.60:0/3920973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 msgr2=0x55dd6f161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.656+0000 7f184f7fe640 1 -- 172.21.15.60:0/3920973559 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6f1d70b0 con 0x55dd6f15be00 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.658+0000 7f184f7fe640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1844031510 tx=0x7f1844031430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51b79cd094876ee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.658+0000 7f182f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:00:35.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.658+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440e4070 con 0x55dd6f15be00 2024-07-24T22:00:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.658+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f18440c78b0 con 0x55dd6f15be00 2024-07-24T22:00:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.659+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1848011c10 con 0x55dd6f15be00 2024-07-24T22:00:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.660+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1844102290 con 0x55dd6f15be00 2024-07-24T22:00:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.660+0000 7f1858af4740 10 client.27679 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.660+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dd6f1d8380 con 0x55dd6f15be00 2024-07-24T22:00:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.660+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848011c10 con 0x55dd6f15be00 2024-07-24T22:00:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.660+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f18440cb5b0 con 0x55dd6f15be00 2024-07-24T22:00:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 10 client.27679 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 20 client.27679 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f1d8b10 0x55dd6f1daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 4 client.27679 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd6f0c53a0 con 0x55dd6f1d8b10 2024-07-24T22:00:35.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:35 smithi060 ceph-mon[33000]: pgmap v811: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 141 op/s 2024-07-24T22:00:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f184ffff640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f1d8b10 0x55dd6f1daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.661+0000 7f182f7fe640 10 client.27679 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.662+0000 7f184ffff640 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f1d8b10 0x55dd6f1daf30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785fd5f25630b351 server_cookie=30b91bd4cec0bfd9 in_seq=0 out_seq=0 2024-07-24T22:00:35.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.666+0000 7f182f7fe640 1 -- 172.21.15.60:0/3920973559 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd6f0c53a0 con 0x55dd6f1d8b10 2024-07-24T22:00:35.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.666+0000 7f182f7fe640 10 client.27679 handle_command_reply: tid=0 2024-07-24T22:00:35.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 client.27679 shutdown 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 msgr2=0x55dd6f161590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1844031510 tx=0x7f1844031430 comp rx=0 tx=0).stop 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 shutdown_connections 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55dd6f1d8b10 0x55dd6f1daf30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dd6f15d170 0x55dd6f161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd6f15be00 0x55dd6f161590 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 --2- 172.21.15.60:0/3920973559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dd6f1511e0 0x55dd6f161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 >> 172.21.15.60:0/3920973559 conn(0x55dd6f14e6f0 msgr2=0x55dd6f153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.667+0000 7f182f7fe640 0 client.27679 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:35.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.668+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 shutdown_connections 2024-07-24T22:00:35.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.668+0000 7f1858af4740 1 -- 172.21.15.60:0/3920973559 wait complete. 2024-07-24T22:00:35.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.668+0000 7f1858af4740 20 client.27679 trim_cache size 0 max 16384 2024-07-24T22:00:35.671 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:00:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 msgr2=0x7f18380b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 0x7f18380b6300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1848009e10 tx=0x7f1848005fd0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 msgr2=0x7f185006e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f184402a9b0 tx=0x7f1844003b90 comp rx=0 tx=0).stop 2024-07-24T22:00:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 shutdown_connections 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18380b3e40 0x7f18380b6300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1850113340 0x7f185006eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18501129c0 0x7f185006e610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 --2- 172.21.15.60:0/3537822113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185010cfb0 0x7f185006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 >> 172.21.15.60:0/3537822113 conn(0x7f185006c2d0 msgr2=0x7f18501108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 shutdown_connections 2024-07-24T22:00:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:35.669+0000 7f18554e1640 1 -- 172.21.15.60:0/3537822113 wait complete. 2024-07-24T22:00:35.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:35 smithi086 ceph-mon[37128]: pgmap v811: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 141 op/s 2024-07-24T22:00:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:35 smithi123 ceph-mon[37143]: pgmap v811: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 141 op/s 2024-07-24T22:00:37.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T22:00:37.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T22:00:37.795 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:37 smithi123 ceph-mon[37143]: pgmap v812: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-07-24T22:00:37.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:37 smithi086 ceph-mon[37128]: pgmap v812: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-07-24T22:00:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:37 smithi060 ceph-mon[33000]: pgmap v812: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-07-24T22:00:37.926 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:39.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:39 smithi086 ceph-mon[37128]: pgmap v813: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 176 op/s 2024-07-24T22:00:39.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:39 smithi060 ceph-mon[33000]: pgmap v813: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 176 op/s 2024-07-24T22:00:39.858 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-24T22:00:39.858 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.6 2024-07-24T22:00:39.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:00:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:39 smithi123 ceph-mon[37143]: pgmap v813: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 176 op/s 2024-07-24T22:00:39.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.994+0000 7f209853d640 1 -- 172.21.15.60:0/651385138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108af0 msgr2=0x7f209010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:39.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.994+0000 7f209853d640 1 --2- 172.21.15.60:0/651385138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108af0 0x7f209010f2f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2088009ca0 tx=0x7f208802a8f0 comp rx=0 tx=0).stop 2024-07-24T22:00:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.995+0000 7f209853d640 1 -- 172.21.15.60:0/651385138 shutdown_connections 2024-07-24T22:00:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.995+0000 7f209853d640 1 --2- 172.21.15.60:0/651385138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108af0 0x7f209010f2f0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.995+0000 7f209853d640 1 --2- 172.21.15.60:0/651385138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108170 0x7f20901085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.995+0000 7f209853d640 1 --2- 172.21.15.60:0/651385138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 0x7f2090102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:39.995+0000 7f209853d640 1 -- 172.21.15.60:0/651385138 >> 172.21.15.60:0/651385138 conn(0x7f20900fe210 msgr2=0x7f2090100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.003+0000 7f209853d640 1 -- 172.21.15.60:0/651385138 shutdown_connections 2024-07-24T22:00:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.003+0000 7f209853d640 1 -- 172.21.15.60:0/651385138 wait complete. 2024-07-24T22:00:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 Processor -- start 2024-07-24T22:00:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 -- start start 2024-07-24T22:00:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 0x7f2090111e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 0x7f2090117720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2090110660 con 0x7f2090108170 2024-07-24T22:00:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20901104e0 con 0x7f2090108af0 2024-07-24T22:00:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.004+0000 7f209853d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20901107e0 con 0x7f2090102760 2024-07-24T22:00:40.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.007+0000 7f2095ad8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.007+0000 7f2095ad8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38732/0 (socket says 172.21.15.60:38732) 2024-07-24T22:00:40.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.007+0000 7f2095ad8640 1 -- 172.21.15.60:0/1155162059 learned_addr learned my addr 172.21.15.60:0/1155162059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.007+0000 7f2096ada640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 0x7f2090117720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f20962d9640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 0x7f2090111e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2095ad8640 1 -- 172.21.15.60:0/1155162059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 msgr2=0x7f2090111e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2095ad8640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 0x7f2090111e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2095ad8640 1 -- 172.21.15.60:0/1155162059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 msgr2=0x7f2090117720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2095ad8640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 0x7f2090117720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2095ad8640 1 -- 172.21.15.60:0/1155162059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2090117e70 con 0x7f2090108170 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.008+0000 7f2096ada640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 0x7f2090117720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:40.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.009+0000 7f2095ad8640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f208000b450 tx=0x7f208000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.009+0000 7f20877fe640 1 -- 172.21.15.60:0/1155162059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080015020 con 0x7f2090108170 2024-07-24T22:00:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.010+0000 7f20877fe640 1 -- 172.21.15.60:0/1155162059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2080011d40 con 0x7f2090108170 2024-07-24T22:00:40.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.010+0000 7f20877fe640 1 -- 172.21.15.60:0/1155162059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000f340 con 0x7f2090108170 2024-07-24T22:00:40.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.010+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2090118160 con 0x7f2090108170 2024-07-24T22:00:40.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.010+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20900726d0 con 0x7f2090108170 2024-07-24T22:00:40.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.015+0000 7f209a8ee740 1 Processor -- start 2024-07-24T22:00:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.018+0000 7f209a8ee740 1 -- start start 2024-07-24T22:00:40.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.019+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a72098310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.019+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.019+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.019+0000 7f209a8ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555a7200f9f0 con 0x555a72098850 2024-07-24T22:00:40.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.019+0000 7f209a8ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a71ffeef0 con 0x555a720a47e0 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f2096ada640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a72098310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f20962d9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f20962d9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38746/0 (socket says 172.21.15.60:38746) 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f20962d9640 1 -- 172.21.15.60:0/3513192458 learned_addr learned my addr 172.21.15.60:0/3513192458 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f2095ad8640 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.020+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555a72002f30 con 0x555a72097f30 2024-07-24T22:00:40.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.021+0000 7f20877fe640 1 -- 172.21.15.60:0/1155162059 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2080018020 con 0x7f2090108170 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20877fe640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 0x7f207c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20962d9640 1 -- 172.21.15.60:0/3513192458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 msgr2=0x555a72098310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20962d9640 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a72098310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20962d9640 1 -- 172.21.15.60:0/3513192458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 msgr2=0x555a720a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20962d9640 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20962d9640 1 -- 172.21.15.60:0/3513192458 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a71d5f930 con 0x555a72098850 2024-07-24T22:00:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.022+0000 7f20877fe640 1 -- 172.21.15.60:0/1155162059 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f20800e3ab0 con 0x7f2090108170 2024-07-24T22:00:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.023+0000 7f20962d9640 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f208c00a8d0 tx=0x7f208c00ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af86a624a4039d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.026+0000 7f2096ada640 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a72098310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.026+0000 7f2084ff9640 1 -- 172.21.15.60:0/3513192458 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c015070 con 0x555a72098850 2024-07-24T22:00:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.026+0000 7f2084ff9640 1 -- 172.21.15.60:0/3513192458 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c0113f0 con 0x555a72098850 2024-07-24T22:00:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.026+0000 7f2084ff9640 1 -- 172.21.15.60:0/3513192458 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0185c0 con 0x555a72098850 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.027+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 msgr2=0x555a720a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.027+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f208c00a8d0 tx=0x7f208c00ad90 comp rx=0 tx=0).stop 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f2096ada640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 0x7f207c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 shutdown_connections 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720a42a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3513192458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a72098310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 >> 172.21.15.60:0/3513192458 conn(0x555a72095d60 msgr2=0x555a72096f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 shutdown_connections 2024-07-24T22:00:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.028+0000 7f209a8ee740 1 -- 172.21.15.60:0/3513192458 wait complete. 2024-07-24T22:00:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.029+0000 7f209a8ee740 1 -- start start 2024-07-24T22:00:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.030+0000 7f209a8ee740 1 Processor -- start 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72097f30 0x555a720a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555a71e62890 con 0x555a72097f30 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a71fa1390 con 0x555a720a47e0 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f209a8ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555a720a7460 con 0x555a72098850 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f2096ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f2096ada640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46442/0 (socket says 172.21.15.60:46442) 2024-07-24T22:00:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.033+0000 7f2096ada640 1 -- 172.21.15.60:0/2299022091 learned_addr learned my addr 172.21.15.60:0/2299022091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f209a8ee740 10 client.0 _ll_register_callbacks cb 0x555a71fab500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 -- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 msgr2=0x555a720a1d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a1d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 -- 172.21.15.60:0/2299022091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72097f30 msgr2=0x555a720a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72097f30 0x555a720a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 -- 172.21.15.60:0/2299022091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a71f9bd60 con 0x555a720a47e0 2024-07-24T22:00:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f2096ada640 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2088031510 tx=0x7f20880319d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bf33f78cdcd0b5d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.034+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208c0113f0 con 0x555a720a47e0 2024-07-24T22:00:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.036+0000 7f2079ffb640 10 client.27767 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.036+0000 7f2079ffb640 1 -- 172.21.15.60:0/2299022091 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208804b020 con 0x555a720a47e0 2024-07-24T22:00:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.036+0000 7f2079ffb640 1 -- 172.21.15.60:0/2299022091 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f2088004510 con 0x555a720a47e0 2024-07-24T22:00:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.037+0000 7f2079ffb640 1 -- 172.21.15.60:0/2299022091 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2088004750 con 0x555a720a47e0 2024-07-24T22:00:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.037+0000 7f209a8ee740 10 client.27767 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.037+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=44}) -- 0x555a720b3320 con 0x555a720a47e0 2024-07-24T22:00:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.042+0000 7f2096ada640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 0x7f207c0b6410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f208802f570 tx=0x7f20880095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f2079ffb640 1 -- 172.21.15.60:0/2299022091 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f208802f640 con 0x555a720a47e0 2024-07-24T22:00:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 10 client.27767 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 20 client.27767 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a720b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 4 client.27767 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.043+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f208c0113f0 con 0x555a720b4c30 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.044+0000 7f20962d9640 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a720b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.045+0000 7f20962d9640 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a720b70f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2f42f100ec723d server_cookie=d6d16025bcbca9d3 in_seq=0 out_seq=0 2024-07-24T22:00:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.045+0000 7f2079ffb640 10 client.27767 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.046+0000 7f2079ffb640 1 -- 172.21.15.60:0/2299022091 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f208c0113f0 con 0x555a720b4c30 2024-07-24T22:00:40.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.046+0000 7f2079ffb640 10 client.27767 handle_command_reply: tid=0 2024-07-24T22:00:40.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.046+0000 7f209a8ee740 1 client.27767 shutdown 2024-07-24T22:00:40.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.051+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 msgr2=0x555a720a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.051+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2088031510 tx=0x7f20880319d0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 shutdown_connections 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a720b70f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720a2290 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f2079ffb640 0 client.27767 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 --2- 172.21.15.60:0/2299022091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72097f30 0x555a720a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 >> 172.21.15.60:0/2299022091 conn(0x555a72095d60 msgr2=0x555a7209b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 shutdown_connections 2024-07-24T22:00:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.053+0000 7f209a8ee740 1 -- 172.21.15.60:0/2299022091 wait complete. 2024-07-24T22:00:40.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.058+0000 7f209a8ee740 20 client.27767 trim_cache size 0 max 16384 2024-07-24T22:00:40.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.096+0000 7f209a8ee740 1 Processor -- start 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.096+0000 7f209a8ee740 1 -- start start 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.096+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a7209c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.096+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f209a8ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c83b0 con 0x555a72098850 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f20962d9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a7209c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f20962d9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a7209c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40800/0 (socket says 172.21.15.60:40800) 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f20962d9640 1 -- 172.21.15.60:0/3251970457 learned_addr learned my addr 172.21.15.60:0/3251970457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:40.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f209a8ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c5bc0 con 0x555a720a47e0 2024-07-24T22:00:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c3a30 con 0x555a72097f30 2024-07-24T22:00:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2095ad8640 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 -- 172.21.15.60:0/3251970457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 msgr2=0x555a7209c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a7209c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 -- 172.21.15.60:0/3251970457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 msgr2=0x555a720afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.097+0000 7f2096ada640 1 -- 172.21.15.60:0/3251970457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a72098680 con 0x555a720a47e0 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.098+0000 7f2096ada640 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720b01c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2088009bf0 tx=0x7f2088047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14219ca184de417c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.098+0000 7f207bfff640 1 -- 172.21.15.60:0/3251970457 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208804b070 con 0x555a720a47e0 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.098+0000 7f207bfff640 1 -- 172.21.15.60:0/3251970457 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088036070 con 0x555a720a47e0 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.098+0000 7f207bfff640 1 -- 172.21.15.60:0/3251970457 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088030950 con 0x555a720a47e0 2024-07-24T22:00:40.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.099+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 msgr2=0x555a720b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.099+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720b01c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2088009bf0 tx=0x7f2088047000 comp rx=0 tx=0).stop 2024-07-24T22:00:40.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.100+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 shutdown_connections 2024-07-24T22:00:40.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.100+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a720a47e0 0x555a720b01c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.100+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a72098850 0x555a720afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.100+0000 7f209a8ee740 1 --2- 172.21.15.60:0/3251970457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72097f30 0x555a7209c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.100+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 >> 172.21.15.60:0/3251970457 conn(0x555a72095d60 msgr2=0x555a720af570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.101+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 shutdown_connections 2024-07-24T22:00:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.101+0000 7f209a8ee740 1 -- 172.21.15.60:0/3251970457 wait complete. 2024-07-24T22:00:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.102+0000 7f209a8ee740 1 -- start start 2024-07-24T22:00:40.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.102+0000 7f209a8ee740 1 Processor -- start 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a720a47e0 0x555a720a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c4080 con 0x555a720a47e0 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c54f0 con 0x555a72097f30 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20901c4750 con 0x555a72098850 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f209a8ee740 10 client.0 _ll_register_callbacks cb 0x7f2088009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46466/0 (socket says 172.21.15.60:46466) 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 -- 172.21.15.60:0/4000655918 learned_addr learned my addr 172.21.15.60:0/4000655918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:40.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f2096ada640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a720a47e0 0x555a720a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f2095ad8640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 -- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 msgr2=0x555a720a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 -- 172.21.15.60:0/4000655918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a720a47e0 msgr2=0x555a720a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a720a47e0 0x555a720a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.103+0000 7f20962d9640 1 -- 172.21.15.60:0/4000655918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a7211eb90 con 0x555a72097f30 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.104+0000 7f20962d9640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f208c008010 tx=0x7f208c011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8b37a1fbb8bc15 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.104+0000 7f2079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.104+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c015040 con 0x555a72097f30 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.104+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088036070 con 0x555a72097f30 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.105+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f208c004450 con 0x555a72097f30 2024-07-24T22:00:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.105+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f208c00c940 con 0x555a72097f30 2024-07-24T22:00:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.107+0000 7f209a8ee740 10 client.27773 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.107+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555a7211feb0 con 0x555a72097f30 2024-07-24T22:00:40.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.107+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088036070 con 0x555a72097f30 2024-07-24T22:00:40.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.107+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f208c003690 con 0x555a72097f30 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 10 client.27773 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 20 client.27773 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a72122510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 4 client.27773 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a72011940 con 0x555a720b4c30 2024-07-24T22:00:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.108+0000 7f2095ad8640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a72122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.109+0000 7f2079ffb640 10 client.27773 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.109+0000 7f2095ad8640 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a72122510 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108e8d15bb68220 server_cookie=8525b1fd46fc3e40 in_seq=0 out_seq=0 2024-07-24T22:00:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.113+0000 7f2079ffb640 1 -- 172.21.15.60:0/4000655918 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a72011940 con 0x555a720b4c30 2024-07-24T22:00:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.113+0000 7f2079ffb640 10 client.27773 handle_command_reply: tid=0 2024-07-24T22:00:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.113+0000 7f209a8ee740 1 client.27773 shutdown 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 msgr2=0x555a720a86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f208c008010 tx=0x7f208c011d30 comp rx=0 tx=0).stop 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 shutdown_connections 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555a720b4c30 0x555a72122510 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555a720a47e0 0x555a720a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a72098850 0x555a720a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 --2- 172.21.15.60:0/4000655918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a72097f30 0x555a720a86e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 >> 172.21.15.60:0/4000655918 conn(0x555a72095d60 msgr2=0x555a720afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.121+0000 7f2079ffb640 0 client.27773 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:40.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.122+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 shutdown_connections 2024-07-24T22:00:40.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.122+0000 7f209a8ee740 1 -- 172.21.15.60:0/4000655918 wait complete. 2024-07-24T22:00:40.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.122+0000 7f209a8ee740 20 client.27773 trim_cache size 0 max 16384 2024-07-24T22:00:40.124 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:00:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T22:00:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "cb678673-1348-4328-af90-e352932ac616", 2024-07-24T22:00:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T22:00:40.125 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 msgr2=0x7f207c0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 0x7f207c0b6410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f208802f570 tx=0x7f20880095c0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 msgr2=0x7f2090112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f208000b450 tx=0x7f208000b910 comp rx=0 tx=0).stop 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 shutdown_connections 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f207c0b3f50 0x7f207c0b6410 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108af0 0x7f2090117720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2090108170 0x7f2090112380 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 --2- 172.21.15.60:0/1155162059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2090102760 0x7f2090111e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 >> 172.21.15.60:0/1155162059 conn(0x7f20900fe210 msgr2=0x7f2090100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 shutdown_connections 2024-07-24T22:00:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:40.124+0000 7f209853d640 1 -- 172.21.15.60:0/1155162059 wait complete. 2024-07-24T22:00:40.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:00:41.054 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:41 smithi086 ceph-mon[37128]: pgmap v814: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:41 smithi060 ceph-mon[33000]: pgmap v814: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-07-24T22:00:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T22:00:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:41.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:41 smithi123 ceph-mon[37143]: pgmap v814: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-07-24T22:00:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.083+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/3492148119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d81177f0 msgr2=0x7fa3d8119be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.084+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/3492148119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d81177f0 0x7fa3d8119be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009900 tx=0x7fa3c802a410 comp rx=0 tx=0).stop 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/3492148119 shutdown_connections 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/3492148119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d81177f0 0x7fa3d8119be0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/3492148119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d810a460 0x7fa3d81172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/3492148119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d8109b40 0x7fa3d8109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/3492148119 >> 172.21.15.60:0/3492148119 conn(0x7fa3d806d760 msgr2=0x7fa3d806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/3492148119 shutdown_connections 2024-07-24T22:00:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.085+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/3492148119 wait complete. 2024-07-24T22:00:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.086+0000 7fa3ddc3f640 1 Processor -- start 2024-07-24T22:00:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.087+0000 7fa3ddc3f640 1 -- start start 2024-07-24T22:00:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.087+0000 7fa3ddc3f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.087+0000 7fa3ddc3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 0x7fa3d81a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.087+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.087+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38778/0 (socket says 172.21.15.60:38778) 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3d77fe640 1 -- 172.21.15.60:0/1213181504 learned_addr learned my addr 172.21.15.60:0/1213181504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 0x7fa3d81a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d811d090 con 0x7fa3d8109b40 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d811cf10 con 0x7fa3d810a460 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d811d210 con 0x7fa3d81177f0 2024-07-24T22:00:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.088+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 0x7fa3d81a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.089+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 0x7fa3d81a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d6ffd640 1 -- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 msgr2=0x7fa3d81a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 0x7fa3d81a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d6ffd640 1 -- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 msgr2=0x7fa3d81a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d6ffd640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d81a9690 con 0x7fa3d810a460 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.090+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 0x7fa3d81a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:00:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.091+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 0x7fa3d81a3b30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000bd70 tx=0x7fa3d000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.092+0000 7fa3d4ff9640 1 -- 172.21.15.60:0/1213181504 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0019070 con 0x7fa3d810a460 2024-07-24T22:00:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.092+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d81a9980 con 0x7fa3d810a460 2024-07-24T22:00:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.092+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81b0040 con 0x7fa3d810a460 2024-07-24T22:00:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.092+0000 7fa3d4ff9640 1 -- 172.21.15.60:0/1213181504 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d00148f0 con 0x7fa3d810a460 2024-07-24T22:00:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.092+0000 7fa3d4ff9640 1 -- 172.21.15.60:0/1213181504 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0012860 con 0x7fa3d810a460 2024-07-24T22:00:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.100+0000 7fa3d4ff9640 1 -- 172.21.15.60:0/1213181504 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa3d001c070 con 0x7fa3d810a460 2024-07-24T22:00:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.103+0000 7fa3d4ff9640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 0x7fa3c00b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.104+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 0x7fa3c00b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.104+0000 7fa3d4ff9640 1 -- 172.21.15.60:0/1213181504 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fa3d00e3630 con 0x7fa3d810a460 2024-07-24T22:00:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.104+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 0x7fa3c00b6410 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009c80 tx=0x7fa3cc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.107+0000 7fa3e0990740 1 Processor -- start 2024-07-24T22:00:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.107+0000 7fa3e0990740 1 -- start start 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fd21e0 0x559141fddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fe0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46490/0 (socket says 172.21.15.60:46490) 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3d77fe640 1 -- 172.21.15.60:0/92867610 learned_addr learned my addr 172.21.15.60:0/92867610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559141f47280 con 0x559141fde170 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559141f36780 con 0x559141fd18c0 2024-07-24T22:00:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559141f3a7c0 con 0x559141fd21e0 2024-07-24T22:00:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.108+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fe0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 -- 172.21.15.60:0/92867610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fd21e0 msgr2=0x559141fddc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fd21e0 0x559141fddc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 -- 172.21.15.60:0/92867610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 msgr2=0x559141fe0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fe0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 -- 172.21.15.60:0/92867610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559141c9a790 con 0x559141fd18c0 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.109+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0114c0 tx=0x7fa3cc011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41f488e132e68ff server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.110+0000 7fa3bdffb640 1 -- 172.21.15.60:0/92867610 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc01e640 con 0x559141fd18c0 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.110+0000 7fa3bdffb640 1 -- 172.21.15.60:0/92867610 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc023070 con 0x559141fd18c0 2024-07-24T22:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.110+0000 7fa3bdffb640 1 -- 172.21.15.60:0/92867610 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc010b80 con 0x559141fd18c0 2024-07-24T22:00:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 msgr2=0x559141fd1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0114c0 tx=0x7fa3cc011000 comp rx=0 tx=0).stop 2024-07-24T22:00:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 shutdown_connections 2024-07-24T22:00:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fe0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fd21e0 0x559141fddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 --2- 172.21.15.60:0/92867610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd18c0 0x559141fd1ca0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0114c0 tx=0x7fa3cc011000 comp rx=0 tx=0).stop 2024-07-24T22:00:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.111+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 >> 172.21.15.60:0/92867610 conn(0x559141fcf6f0 msgr2=0x559141fd0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.112+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 shutdown_connections 2024-07-24T22:00:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.112+0000 7fa3e0990740 1 -- 172.21.15.60:0/92867610 wait complete. 2024-07-24T22:00:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 -- start start 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 Processor -- start 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd21e0 0x559141fddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fdaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559141e70ab0 con 0x559141fde170 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559141ebe9e0 con 0x559141fd21e0 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559141fe0df0 con 0x559141fdb5d0 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.113+0000 7fa3e0990740 10 client.0 _ll_register_callbacks cb 0x559141c46a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.114+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.114+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40818/0 (socket says 172.21.15.60:40818) 2024-07-24T22:00:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.114+0000 7fa3d7fff640 1 -- 172.21.15.60:0/4245827174 learned_addr learned my addr 172.21.15.60:0/4245827174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.115+0000 7fa3d7fff640 1 -- 172.21.15.60:0/4245827174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd21e0 msgr2=0x559141fddb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.115+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd21e0 0x559141fddb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.115+0000 7fa3d7fff640 1 -- 172.21.15.60:0/4245827174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 msgr2=0x559141fdaf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.115+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fdaf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.115+0000 7fa3d7fff640 1 -- 172.21.15.60:0/4245827174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559141d13620 con 0x559141fdb5d0 2024-07-24T22:00:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.116+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009850 tx=0x7fa3c8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79670b62d2a000b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.117+0000 7fa3b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:00:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.117+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8035040 con 0x559141fdb5d0 2024-07-24T22:00:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.117+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559141ee5ce0 con 0x559141fdb5d0 2024-07-24T22:00:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.119+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fa3c80046c0 con 0x559141fdb5d0 2024-07-24T22:00:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.120+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3c80048c0 con 0x559141fdb5d0 2024-07-24T22:00:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.121+0000 7fa3e0990740 10 client.27691 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.121+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55914200ece0 con 0x559141fdb5d0 2024-07-24T22:00:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559141ee5ce0 con 0x559141fdb5d0 2024-07-24T22:00:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa3c80314a0 con 0x559141fdb5d0 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 10 client.27691 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 10 client.27691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 20 client.27691 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142010680 0x559142012ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 4 client.27691 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.126+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559141f491d0 con 0x559142010680 2024-07-24T22:00:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.127+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142010680 0x559142012ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.133+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142010680 0x559142012ae0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbe8a2505c53bce server_cookie=3830d9e13da0eed in_seq=0 out_seq=0 2024-07-24T22:00:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.133+0000 7fa3b6ffd640 10 client.27691 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.136+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/4245827174 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559141f491d0 con 0x559142010680 2024-07-24T22:00:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.136+0000 7fa3b6ffd640 10 client.27691 handle_command_reply: tid=0 2024-07-24T22:00:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.137+0000 7fa3e0990740 1 client.27691 shutdown 2024-07-24T22:00:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.140+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 msgr2=0x559141fdba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.140+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009850 tx=0x7fa3c8005270 comp rx=0 tx=0).stop 2024-07-24T22:00:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 shutdown_connections 2024-07-24T22:00:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142010680 0x559142012ae0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x559141fdba50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fde170 0x559141fdaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 --2- 172.21.15.60:0/4245827174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fd21e0 0x559141fddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 >> 172.21.15.60:0/4245827174 conn(0x559141fcf6f0 msgr2=0x559141fd5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.141+0000 7fa3b6ffd640 0 client.27691 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.144+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 shutdown_connections 2024-07-24T22:00:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.145+0000 7fa3e0990740 1 -- 172.21.15.60:0/4245827174 wait complete. 2024-07-24T22:00:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.145+0000 7fa3e0990740 20 client.27691 trim_cache size 0 max 16384 2024-07-24T22:00:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.181+0000 7fa3e0990740 1 Processor -- start 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.181+0000 7fa3e0990740 1 -- start start 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.181+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.181+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x5591420183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.181+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 0x559142018930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.182+0000 7fa3e0990740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4011320 con 0x559141fd21e0 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.182+0000 7fa3e0990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c400eb30 con 0x559141fde170 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.182+0000 7fa3e0990740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c400c9a0 con 0x559141fdb5d0 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.182+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x5591420183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.183+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 0x559142018930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.183+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x5591420183f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40822/0 (socket says 172.21.15.60:40822) 2024-07-24T22:00:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.183+0000 7fa3d6ffd640 1 -- 172.21.15.60:0/2211884446 learned_addr learned my addr 172.21.15.60:0/2211884446 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.183+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.184+0000 7fa3d7fff640 1 -- 172.21.15.60:0/2211884446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 msgr2=0x5591420183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.184+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x5591420183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.184+0000 7fa3d7fff640 1 -- 172.21.15.60:0/2211884446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 msgr2=0x559141fd0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.184+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fd0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.184+0000 7fa3d7fff640 1 -- 172.21.15.60:0/2211884446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559141fe0560 con 0x559141fde170 2024-07-24T22:00:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.187+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 0x559142018930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8031440 tx=0x7fa3c802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a16707766262a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.187+0000 7fa3bd7fa640 1 -- 172.21.15.60:0/2211884446 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8035030 con 0x559141fde170 2024-07-24T22:00:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.187+0000 7fa3bd7fa640 1 -- 172.21.15.60:0/2211884446 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c80043a0 con 0x559141fde170 2024-07-24T22:00:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.188+0000 7fa3bd7fa640 1 -- 172.21.15.60:0/2211884446 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8050070 con 0x559141fde170 2024-07-24T22:00:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.189+0000 7fa3e0990740 1 -- 172.21.15.60:0/2211884446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 msgr2=0x559142018930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.189+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 0x559142018930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8031440 tx=0x7fa3c802a410 comp rx=0 tx=0).stop 2024-07-24T22:00:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.190+0000 7fa3e0990740 1 -- 172.21.15.60:0/2211884446 shutdown_connections 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.190+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fde170 0x559142018930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.190+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fdb5d0 0x5591420183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.190+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2211884446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fd0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.190+0000 7fa3e0990740 1 -- 172.21.15.60:0/2211884446 >> 172.21.15.60:0/2211884446 conn(0x559141fcf6f0 msgr2=0x559141fd0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.191+0000 7fa3e0990740 1 -- 172.21.15.60:0/2211884446 shutdown_connections 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.191+0000 7fa3e0990740 1 -- 172.21.15.60:0/2211884446 wait complete. 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.191+0000 7fa3e0990740 1 -- start start 2024-07-24T22:00:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 Processor -- start 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fe18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 0x559141fe1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fde170 0x55914207bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c400cff0 con 0x559141fd21e0 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c400e460 con 0x559141fdb5d0 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c400d6c0 con 0x559141fde170 2024-07-24T22:00:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.192+0000 7fa3e0990740 10 client.0 _ll_register_callbacks cb 0x7fa3c8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 0x559141fe1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 0x559141fe1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46522/0 (socket says 172.21.15.60:46522) 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d6ffd640 1 -- 172.21.15.60:0/2504726212 learned_addr learned my addr 172.21.15.60:0/2504726212 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fe18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d7fff640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fde170 0x55914207bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 -- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fde170 msgr2=0x55914207bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fde170 0x55914207bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 -- 172.21.15.60:0/2504726212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 msgr2=0x559141fe1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 0x559141fe1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.193+0000 7fa3d77fe640 1 -- 172.21.15.60:0/2504726212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914207c2d0 con 0x559141fd21e0 2024-07-24T22:00:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.194+0000 7fa3d77fe640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fe18d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc017000 tx=0x7fa3cc010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28520f3ba5b26d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.194+0000 7fa3b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.195+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004020 con 0x559141fd21e0 2024-07-24T22:00:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.195+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fa3cc0041c0 con 0x559141fd21e0 2024-07-24T22:00:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.197+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c80043a0 con 0x559141fd21e0 2024-07-24T22:00:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.201+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc0043c0 con 0x559141fd21e0 2024-07-24T22:00:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.202+0000 7fa3e0990740 10 client.18402 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.202+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559142083670 con 0x559141fd21e0 2024-07-24T22:00:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.205+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c80043a0 con 0x559141fd21e0 2024-07-24T22:00:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.205+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa3cc00f040 con 0x559141fd21e0 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.207+0000 7fa3e0990740 10 client.18402 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.207+0000 7fa3e0990740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.207+0000 7fa3e0990740 20 client.18402 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.208+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142083a10 0x559142085e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.208+0000 7fa3e0990740 4 client.18402 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:00:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.208+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559141f44c70 con 0x559142083a10 2024-07-24T22:00:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.208+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142083a10 0x559142085e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.209+0000 7fa3d6ffd640 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142083a10 0x559142085e00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ee4abf1d220164 server_cookie=3631ddab2ed990f7 in_seq=0 out_seq=0 2024-07-24T22:00:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.209+0000 7fa3b6ffd640 10 client.18402 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.420+0000 7fa3b6ffd640 1 -- 172.21.15.60:0/2504726212 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559141f44c70 con 0x559142083a10 2024-07-24T22:00:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.420+0000 7fa3b6ffd640 10 client.18402 handle_command_reply: tid=0 2024-07-24T22:00:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 client.18402 shutdown 2024-07-24T22:00:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 msgr2=0x559141fe18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fe18d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc017000 tx=0x7fa3cc010760 comp rx=0 tx=0).stop 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 shutdown_connections 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559142083a10 0x559142085e00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559141fde170 0x55914207bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559141fdb5d0 0x559141fe1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 --2- 172.21.15.60:0/2504726212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559141fd21e0 0x559141fe18d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 >> 172.21.15.60:0/2504726212 conn(0x559141fcf6f0 msgr2=0x559141fdcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3b6ffd640 0 client.18402 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 shutdown_connections 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.421+0000 7fa3e0990740 1 -- 172.21.15.60:0/2504726212 wait complete. 2024-07-24T22:00:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.422+0000 7fa3e0990740 20 client.18402 trim_cache size 0 max 16384 2024-07-24T22:00:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:00:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:00:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:00:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.422+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 msgr2=0x7fa3c00b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.422+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 0x7fa3c00b6410 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009c80 tx=0x7fa3cc005f90 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.422+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 msgr2=0x7fa3d81a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.422+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 0x7fa3d81a3b30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000bd70 tx=0x7fa3d000be60 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 shutdown_connections 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa3c00b3f50 0x7fa3c00b6410 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3d81177f0 0x7fa3d81a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3d810a460 0x7fa3d81a3b30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 --2- 172.21.15.60:0/1213181504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3d8109b40 0x7fa3d81a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 >> 172.21.15.60:0/1213181504 conn(0x7fa3d806d760 msgr2=0x7fa3d8117cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 shutdown_connections 2024-07-24T22:00:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:42.423+0000 7fa3ddc3f640 1 -- 172.21.15.60:0/1213181504 wait complete. 2024-07-24T22:00:42.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:42 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:00:42.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:42 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:42 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:00:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:42 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:42.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:42 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:00:42.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:42 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:43 smithi086 ceph-mon[37128]: pgmap v815: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-07-24T22:00:43.820 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:43 smithi060 ceph-mon[33000]: pgmap v815: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-07-24T22:00:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:43 smithi123 ceph-mon[37143]: pgmap v815: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-07-24T22:00:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.492+0000 7fa787fff640 1 -- 172.21.15.60:0/2064596439 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c031810 con 0x7fa790109f20 2024-07-24T22:00:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.492+0000 7fa796495640 1 -- 172.21.15.60:0/2064596439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 msgr2=0x7fa79010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.492+0000 7fa796495640 1 --2- 172.21.15.60:0/2064596439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 0x7fa79010a360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009930 tx=0x7fa78c02a460 comp rx=0 tx=0).stop 2024-07-24T22:00:45.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 -- 172.21.15.60:0/2064596439 shutdown_connections 2024-07-24T22:00:45.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 --2- 172.21.15.60:0/2064596439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa79011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 --2- 172.21.15.60:0/2064596439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 0x7fa79010a360 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 --2- 172.21.15.60:0/2064596439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 -- 172.21.15.60:0/2064596439 >> 172.21.15.60:0/2064596439 conn(0x7fa79006c4f0 msgr2=0x7fa79006c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 -- 172.21.15.60:0/2064596439 shutdown_connections 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.494+0000 7fa796495640 1 -- 172.21.15.60:0/2064596439 wait complete. 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 Processor -- start 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 -- start start 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790118ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 0x7fa790119410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa790112280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa790121430 con 0x7fa79011ba90 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901212b0 con 0x7fa790109f20 2024-07-24T22:00:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.495+0000 7fa796495640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901215b0 con 0x7fa790109b40 2024-07-24T22:00:45.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.496+0000 7fa795c94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa790112280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.496+0000 7fa795c94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa790112280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38828/0 (socket says 172.21.15.60:38828) 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.496+0000 7fa795c94640 1 -- 172.21.15.60:0/2545892518 learned_addr learned my addr 172.21.15.60:0/2545892518 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.496+0000 7fa795493640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790118ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 -- 172.21.15.60:0/2545892518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 msgr2=0x7fa790119410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 0x7fa790119410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 -- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 msgr2=0x7fa790112280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa790112280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 -- 172.21.15.60:0/2545892518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa790112b40 con 0x7fa790109b40 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.497+0000 7fa795493640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790118ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa78000d6d0 tx=0x7fa78000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.498+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780017070 con 0x7fa790109b40 2024-07-24T22:00:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.498+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa790112e30 con 0x7fa790109b40 2024-07-24T22:00:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.498+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa790117e20 con 0x7fa790109b40 2024-07-24T22:00:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.498+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa780009820 con 0x7fa790109b40 2024-07-24T22:00:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.499+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780017070 con 0x7fa790109b40 2024-07-24T22:00:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.503+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7740052c0 con 0x7fa790109b40 2024-07-24T22:00:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.503+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa780009c90 con 0x7fa790109b40 2024-07-24T22:00:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.505+0000 7fa7867fc640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 0x7fa77c0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.505+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7fa7800e3470 con 0x7fa790109b40 2024-07-24T22:00:45.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.506+0000 7fa794c92640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 0x7fa77c0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.508+0000 7fa794c92640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 0x7fa77c0b6500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009230 tx=0x7fa78c005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.514+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fa7800aa720 con 0x7fa790109b40 2024-07-24T22:00:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.755+0000 7fec0a4d6640 1 -- 172.21.15.60:0/1411577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04100700 msgr2=0x7fec04100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.755+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/1411577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04100700 0x7fec04100ae0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7febf4009900 tx=0x7febf402a420 comp rx=0 tx=0).stop 2024-07-24T22:00:45.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:45 smithi060 ceph-mon[33000]: pgmap v816: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 390 op/s 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 -- 172.21.15.60:0/1411577 shutdown_connections 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/1411577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec041068c0 0x7fec041070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/1411577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec04105f40 0x7fec04106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/1411577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04100700 0x7fec04100ae0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 -- 172.21.15.60:0/1411577 >> 172.21.15.60:0/1411577 conn(0x7fec040fc3f0 msgr2=0x7fec040fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 -- 172.21.15.60:0/1411577 shutdown_connections 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.756+0000 7fec0a4d6640 1 -- 172.21.15.60:0/1411577 wait complete. 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 Processor -- start 2024-07-24T22:00:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 -- start start 2024-07-24T22:00:45.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec04105f40 0x7fec04074730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041068c0 0x7fec0406d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec04118d90 con 0x7fec04105f40 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec04118c10 con 0x7fec041068c0 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.757+0000 7fec0a4d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec04118f10 con 0x7fec04100700 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.758+0000 7fec03fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.758+0000 7fec03fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40878/0 (socket says 172.21.15.60:40878) 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.758+0000 7fec03fff640 1 -- 172.21.15.60:0/2825791033 learned_addr learned my addr 172.21.15.60:0/2825791033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec08a73640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041068c0 0x7fec0406d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec03fff640 1 -- 172.21.15.60:0/2825791033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041068c0 msgr2=0x7fec0406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec03fff640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041068c0 0x7fec0406d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec03fff640 1 -- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec04105f40 msgr2=0x7fec04074730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec03fff640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec04105f40 0x7fec04074730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.759+0000 7fec03fff640 1 -- 172.21.15.60:0/2825791033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0406ded0 con 0x7fec04100700 2024-07-24T22:00:45.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.763+0000 7fec03fff640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf4003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.764+0000 7fec017fa640 1 -- 172.21.15.60:0/2825791033 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4040030 con 0x7fec04100700 2024-07-24T22:00:45.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.764+0000 7fec017fa640 1 -- 172.21.15.60:0/2825791033 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf403c8f0 con 0x7fec04100700 2024-07-24T22:00:45.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.764+0000 7fec017fa640 1 -- 172.21.15.60:0/2825791033 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf40319a0 con 0x7fec04100700 2024-07-24T22:00:45.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.766+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0406e160 con 0x7fec04100700 2024-07-24T22:00:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.766+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec04073030 con 0x7fec04100700 2024-07-24T22:00:45.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 Processor -- start 2024-07-24T22:00:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 -- start start 2024-07-24T22:00:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b47f30 0x555f13b4a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555f13ab2960 con 0x555f13b3bfa0 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f13aa1e60 con 0x555f13b47f30 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.770+0000 7fec0c887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f13aa5ea0 con 0x555f13b3b680 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.771+0000 7fec08a73640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.771+0000 7fec08a73640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:40888/0 (socket says 172.21.15.60:40888) 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.771+0000 7fec08a73640 1 -- 172.21.15.60:0/2213271270 learned_addr learned my addr 172.21.15.60:0/2213271270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 -- 172.21.15.60:0/2213271270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b47f30 msgr2=0x555f13b4a320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b47f30 0x555f13b4a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 -- 172.21.15.60:0/2213271270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 msgr2=0x555f13b479f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b479f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 -- 172.21.15.60:0/2213271270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13a2b2d0 con 0x555f13b3b680 2024-07-24T22:00:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.772+0000 7fec08a73640 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febfc010370 tx=0x7febfc010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2372e36a09cf96e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.774+0000 7fec017fa640 1 -- 172.21.15.60:0/2825791033 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7febf4043030 con 0x7fec04100700 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.775+0000 7fec017fa640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 0x7febec0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.775+0000 7fec017fa640 1 -- 172.21.15.60:0/2825791033 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7febf41047a0 con 0x7fec04100700 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.776+0000 7febea7fc640 1 -- 172.21.15.60:0/2213271270 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc016070 con 0x555f13b3b680 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.776+0000 7febea7fc640 1 -- 172.21.15.60:0/2213271270 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc004d20 con 0x555f13b3b680 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.776+0000 7febea7fc640 1 -- 172.21.15.60:0/2213271270 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc005380 con 0x555f13b3b680 2024-07-24T22:00:45.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.776+0000 7fec03fff640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 0x7febec0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.777+0000 7fec0c887740 1 -- 172.21.15.60:0/2213271270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 msgr2=0x555f13b3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.777+0000 7fec0c887740 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febfc010370 tx=0x7febfc010830 comp rx=0 tx=0).stop 2024-07-24T22:00:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.778+0000 7fec03fff640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 0x7febec0b6350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x555f13b3a390 tx=0x7febf402a420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 -- 172.21.15.60:0/2213271270 shutdown_connections 2024-07-24T22:00:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b47f30 0x555f13b4a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 --2- 172.21.15.60:0/2213271270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3b680 0x555f13b3ba60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 -- 172.21.15.60:0/2213271270 >> 172.21.15.60:0/2213271270 conn(0x555f13b394b0 msgr2=0x555f13b3a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.780+0000 7fec0c887740 1 -- 172.21.15.60:0/2213271270 shutdown_connections 2024-07-24T22:00:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.782+0000 7fec0c887740 1 -- 172.21.15.60:0/2213271270 wait complete. 2024-07-24T22:00:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.782+0000 7fec0c887740 1 -- start start 2024-07-24T22:00:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 Processor -- start 2024-07-24T22:00:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3bfa0 0x555f13b454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b47f30 0x555f13b45a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec08a73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec08a73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46584/0 (socket says 172.21.15.60:46584) 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec08a73640 1 -- 172.21.15.60:0/3735076540 learned_addr learned my addr 172.21.15.60:0/3735076540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555f13905700 con 0x555f13b47f30 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f13a445a0 con 0x555f13b3b680 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f13b4abb0 con 0x555f13b3bfa0 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.783+0000 7fec0c887740 10 client.0 _ll_register_callbacks cb 0x555f13a64910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 -- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3bfa0 msgr2=0x555f13b454c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:00:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3bfa0 0x555f13b454c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 -- 172.21.15.60:0/3735076540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b47f30 msgr2=0x555f13b45a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b47f30 0x555f13b45a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13a37630 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec08a73640 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d80 tx=0x7febfc004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409dd6f8f2a29635 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7febe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc040030 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.784+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc004d20 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.785+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7febfc012a00 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.785+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7febfc043040 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.785+0000 7fec0c887740 10 client.27800 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.785+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f13802650 con 0x555f13b3b680 2024-07-24T22:00:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.786+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc004d20 con 0x555f13b3b680 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7febfc046070 con 0x555f13b3b680 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 10 client.27800 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 10 client.27800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 20 client.27800 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13b57e40 0x555f13b5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 4 client.27800 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.787+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f13ab48b0 con 0x555f13b57e40 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.789+0000 7fec037fe640 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13b57e40 0x555f13b5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.790+0000 7fec037fe640 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13b57e40 0x555f13b5a300 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da4508486b570a2 server_cookie=c07777edc4ea8910 in_seq=0 out_seq=0 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.790+0000 7febe37fe640 10 client.27800 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.791+0000 7febe37fe640 1 -- 172.21.15.60:0/3735076540 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f13ab48b0 con 0x555f13b57e40 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.791+0000 7febe37fe640 10 client.27800 handle_command_reply: tid=0 2024-07-24T22:00:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.792+0000 7fec0c887740 1 client.27800 shutdown 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 msgr2=0x555f13b44f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d80 tx=0x7febfc004b80 comp rx=0 tx=0).stop 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 shutdown_connections 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13b57e40 0x555f13b5a300 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b47f30 0x555f13b45a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b3bfa0 0x555f13b454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 --2- 172.21.15.60:0/3735076540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b44f80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 >> 172.21.15.60:0/3735076540 conn(0x555f13b394b0 msgr2=0x555f13b3f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7febe37fe640 0 client.27800 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 shutdown_connections 2024-07-24T22:00:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 1 -- 172.21.15.60:0/3735076540 wait complete. 2024-07-24T22:00:45.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.793+0000 7fec0c887740 20 client.27800 trim_cache size 0 max 16384 2024-07-24T22:00:45.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.834+0000 7fec0c887740 1 Processor -- start 2024-07-24T22:00:45.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.835+0000 7fec0c887740 1 -- start start 2024-07-24T22:00:45.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.835+0000 7fec0c887740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 0x555f13b3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.836+0000 7fec0c887740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 0x555f13b5fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.836+0000 7fec0c887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13b60150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.836+0000 7fec037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 0x555f13b5fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.836+0000 7fec037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 0x555f13b5fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35452/0 (socket says 172.21.15.60:35452) 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.837+0000 7fec037fe640 1 -- 172.21.15.60:0/3948234511 learned_addr learned my addr 172.21.15.60:0/3948234511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.837+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a8990 con 0x555f13b3b680 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.837+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a61a0 con 0x555f13b3bfa0 2024-07-24T22:00:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.837+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a40a0 con 0x555f13b47f30 2024-07-24T22:00:45.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.837+0000 7fec08a73640 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 0x555f13b3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 -- 172.21.15.60:0/3948234511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 msgr2=0x555f13b60150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:45.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13b60150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 -- 172.21.15.60:0/3948234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 msgr2=0x555f13b5fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 0x555f13b5fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 -- 172.21.15.60:0/3948234511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13b47620 con 0x555f13b3b680 2024-07-24T22:00:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.838+0000 7fec08a73640 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 0x555f13b3eed0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7febfc009f40 tx=0x7febfc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1c46e2e476066c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.839+0000 7febe9ffb640 1 -- 172.21.15.60:0/3948234511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc040070 con 0x555f13b3b680 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.839+0000 7febe9ffb640 1 -- 172.21.15.60:0/3948234511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc003e20 con 0x555f13b3b680 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.839+0000 7febe9ffb640 1 -- 172.21.15.60:0/3948234511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc016040 con 0x555f13b3b680 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 msgr2=0x555f13b3eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 0x555f13b3eed0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7febfc009f40 tx=0x7febfc015000 comp rx=0 tx=0).stop 2024-07-24T22:00:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 shutdown_connections 2024-07-24T22:00:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13b60150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3bfa0 0x555f13b5fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 --2- 172.21.15.60:0/3948234511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3b680 0x555f13b3eed0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 >> 172.21.15.60:0/3948234511 conn(0x555f13b394b0 msgr2=0x555f13b3e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 shutdown_connections 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- 172.21.15.60:0/3948234511 wait complete. 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.840+0000 7fec0c887740 1 -- start start 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 Processor -- start 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13bc0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a4660 con 0x555f13b3bfa0 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a5ad0 con 0x555f13b3b680 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a4d30 con 0x555f13b47f30 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.841+0000 7fec0c887740 10 client.0 _ll_register_callbacks cb 0x7febfc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec08a73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec08a73640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b4c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35462/0 (socket says 172.21.15.60:35462) 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec08a73640 1 -- 172.21.15.60:0/3117340160 learned_addr learned my addr 172.21.15.60:0/3117340160 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec08a73640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13bc0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 -- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 msgr2=0x555f13bc0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13bc0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 -- 172.21.15.60:0/3117340160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 msgr2=0x555f13b4c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b4c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 -- 172.21.15.60:0/3117340160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13bc1110 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7fec037fe640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b4ca40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7febf8006e50 tx=0x7febf80167f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa0aaf5c9ef6534 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.842+0000 7febe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.843+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf802a040 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.843+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7febf8009d40 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.843+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc003e20 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.843+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7febf803c050 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 10 client.18414 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f13bc22a0 con 0x555f13b3bfa0 2024-07-24T22:00:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc003e20 con 0x555f13b3bfa0 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7febf8015540 con 0x555f13b3bfa0 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 10 client.18414 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 20 client.18414 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13bc2a60 0x555f13bc4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 4 client.18414 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.844+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f13ab0350 con 0x555f13bc2a60 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.845+0000 7fec08a73640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13bc2a60 0x555f13bc4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.846+0000 7fec08a73640 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13bc2a60 0x555f13bc4e80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ead776a6241ce78 server_cookie=29a2e338635c45d7 in_seq=0 out_seq=0 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.846+0000 7febe37fe640 10 client.18414 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.846+0000 7febe37fe640 1 -- 172.21.15.60:0/3117340160 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x555f13ab0350 con 0x555f13bc2a60 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.846+0000 7febe37fe640 10 client.18414 handle_command_reply: tid=0 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.846+0000 7fec0c887740 1 client.18414 shutdown 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 msgr2=0x555f13b4ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b4ca40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7febf8006e50 tx=0x7febf80167f0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 shutdown_connections 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555f13bc2a60 0x555f13bc4e80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7febe37fe640 0 client.18414 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f13b47f30 0x555f13bc0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f13b3bfa0 0x555f13b4ca40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 --2- 172.21.15.60:0/3117340160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f13b3b680 0x555f13b4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.848+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 >> 172.21.15.60:0/3117340160 conn(0x555f13b394b0 msgr2=0x555f13b46dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.849+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 shutdown_connections 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.849+0000 7fec0c887740 1 -- 172.21.15.60:0/3117340160 wait complete. 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.849+0000 7fec0c887740 20 client.18414 trim_cache size 0 max 16384 2024-07-24T22:00:45.854 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (14837 inodes in the stack)", 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "cb678673-1348-4328-af90-e352932ac616": { 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "cb678673-1348-4328-af90-e352932ac616", 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:45.855 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:00:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.853+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 msgr2=0x7febec0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.853+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 0x7febec0b6350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x555f13b3a390 tx=0x7febf402a420 comp rx=0 tx=0).stop 2024-07-24T22:00:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.853+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 msgr2=0x7fec040741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.853+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf4003f10 comp rx=0 tx=0).stop 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.854+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 shutdown_connections 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.854+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7febec0b3e90 0x7febec0b6350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.854+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041068c0 0x7fec0406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.855+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec04105f40 0x7fec04074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.855+0000 7fec0a4d6640 1 --2- 172.21.15.60:0/2825791033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100700 0x7fec040741f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.855+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 >> 172.21.15.60:0/2825791033 conn(0x7fec040fc3f0 msgr2=0x7fec040fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.855+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 shutdown_connections 2024-07-24T22:00:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.855+0000 7fec0a4d6640 1 -- 172.21.15.60:0/2825791033 wait complete. 2024-07-24T22:00:45.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.965+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7740055b0 con 0x7fa790109b40 2024-07-24T22:00:45.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.970+0000 7fa7867fc640 1 -- 172.21.15.60:0/2545892518 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fa7800aa720 con 0x7fa790109b40 2024-07-24T22:00:45.972 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:00:45.973 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:00:45.973 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:00:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.974+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 msgr2=0x7fa77c0b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.974+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 0x7fa77c0b6500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009230 tx=0x7fa78c005fd0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.974+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 msgr2=0x7fa790118ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.974+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790118ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa78000d6d0 tx=0x7fa78000db90 comp rx=0 tx=0).stop 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 shutdown_connections 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa77c0b4040 0x7fa77c0b6500 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa79011ba90 0x7fa790112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa790109f20 0x7fa790119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 --2- 172.21.15.60:0/2545892518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa790109b40 0x7fa790118ed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.975+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 >> 172.21.15.60:0/2545892518 conn(0x7fa79006c4f0 msgr2=0x7fa79011bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.976+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 shutdown_connections 2024-07-24T22:00:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:45.976+0000 7fa796495640 1 -- 172.21.15.60:0/2545892518 wait complete. 2024-07-24T22:00:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:45 smithi123 ceph-mon[37143]: pgmap v816: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 390 op/s 2024-07-24T22:00:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:45 smithi086 ceph-mon[37128]: pgmap v816: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 390 op/s 2024-07-24T22:00:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:46 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:46.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:46 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2545892518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:46 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:46.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:46 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2545892518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:47.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:46 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:47.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:46 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2545892518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:00:47.226 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 140.6 --await-for 45.0' 2024-07-24T22:00:47.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 140.6 --await-for 45.0 2024-07-24T22:00:47.686 INFO:tasks.cephfs.filesystem:scrub status for tag:cb678673-1348-4328-af90-e352932ac616 - {'path': '/', 'tag': 'cb678673-1348-4328-af90-e352932ac616', 'options': 'recursive,force'} 2024-07-24T22:00:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:00:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:47 smithi060 ceph-mon[33000]: pgmap v817: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 771 op/s 2024-07-24T22:00:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:47 smithi123 ceph-mon[37143]: pgmap v817: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 771 op/s 2024-07-24T22:00:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:47 smithi086 ceph-mon[37128]: pgmap v817: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 771 op/s 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 -- 172.21.15.60:0/1312939158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108ae0 msgr2=0x7f9c0c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 --2- 172.21.15.60:0/1312939158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c10f2e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ca0 tx=0x7f9c0402a8e0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 -- 172.21.15.60:0/1312939158 shutdown_connections 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 --2- 172.21.15.60:0/1312939158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c10f2e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 --2- 172.21.15.60:0/1312939158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108160 0x7f9c0c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 --2- 172.21.15.60:0/1312939158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c102750 0x7f9c0c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 -- 172.21.15.60:0/1312939158 >> 172.21.15.60:0/1312939158 conn(0x7f9c0c0fe200 msgr2=0x7f9c0c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 -- 172.21.15.60:0/1312939158 shutdown_connections 2024-07-24T22:00:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.761+0000 7f9c10e78640 1 -- 172.21.15.60:0/1312939158 wait complete. 2024-07-24T22:00:49.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.762+0000 7f9c10e78640 1 Processor -- start 2024-07-24T22:00:49.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.762+0000 7f9c10e78640 1 -- start start 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c102750 0x7f9c0c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108160 0x7f9c0c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1149f0 con 0x7f9c0c108160 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c114870 con 0x7f9c0c102750 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c10e78640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c114b70 con 0x7f9c0c108ae0 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c0ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c0ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42484/0 (socket says 172.21.15.60:42484) 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c0ad9d640 1 -- 172.21.15.60:0/2510377900 learned_addr learned my addr 172.21.15.60:0/2510377900 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.763+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108160 0x7f9c0c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 -- 172.21.15.60:0/2510377900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c102750 msgr2=0x7f9c0c19aec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:00:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c102750 0x7f9c0c19aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 -- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108160 msgr2=0x7f9c0c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:49.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108160 0x7f9c0c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 -- 172.21.15.60:0/2510377900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1a0f70 con 0x7f9c0c108ae0 2024-07-24T22:00:49.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c0ad9d640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c04031e70 tx=0x7f9c04031ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9bfb7fe640 1 -- 172.21.15.60:0/2510377900 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04003d20 con 0x7f9c0c108ae0 2024-07-24T22:00:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1a1200 con 0x7f9c0c108ae0 2024-07-24T22:00:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.764+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c06c5e0 con 0x7f9c0c108ae0 2024-07-24T22:00:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.765+0000 7f9bfb7fe640 1 -- 172.21.15.60:0/2510377900 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c04036030 con 0x7f9c0c108ae0 2024-07-24T22:00:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.765+0000 7f9bfb7fe640 1 -- 172.21.15.60:0/2510377900 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040308d0 con 0x7f9c0c108ae0 2024-07-24T22:00:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.767+0000 7f9c13229740 1 Processor -- start 2024-07-24T22:00:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9bfb7fe640 1 -- 172.21.15.60:0/2510377900 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9c04030a70 con 0x7f9c0c108ae0 2024-07-24T22:00:49.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 -- start start 2024-07-24T22:00:49.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70c4cd0 0x561be70c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561be7039670 con 0x561be70c4cd0 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561be7028b70 con 0x561be70c3770 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c13229740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561be702cbb0 con 0x561be70cf5a0 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.768+0000 7f9c0a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35504/0 (socket says 172.21.15.60:35504) 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 -- 172.21.15.60:0/4048237595 learned_addr learned my addr 172.21.15.60:0/4048237595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70c4cd0 0x561be70c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9bfb7fe640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 0x7f9bf40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 -- 172.21.15.60:0/4048237595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 msgr2=0x561be70d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 -- 172.21.15.60:0/4048237595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70c4cd0 msgr2=0x561be70c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70c4cd0 0x561be70c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 -- 172.21.15.60:0/4048237595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be6fc0d30 con 0x561be70c3770 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9bfb7fe640 1 -- 172.21.15.60:0/2510377900 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f9c04105110 con 0x7f9c0c108ae0 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0ad9d640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 0x7f9bf40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.769+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009bd0 tx=0x7f9bfc00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84eb80e8ef0cd23d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.770+0000 7f9bf8ff9640 1 -- 172.21.15.60:0/4048237595 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc005020 con 0x561be70c3770 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.770+0000 7f9c0ad9d640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 0x7f9bf40b6280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cbd00 tx=0x7f9c04005ef0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.770+0000 7f9bf8ff9640 1 -- 172.21.15.60:0/4048237595 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0047a0 con 0x561be70c3770 2024-07-24T22:00:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.770+0000 7f9bf8ff9640 1 -- 172.21.15.60:0/4048237595 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc010690 con 0x561be70c3770 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 -- 172.21.15.60:0/4048237595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 msgr2=0x561be70c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009bd0 tx=0x7f9bfc00e720 comp rx=0 tx=0).stop 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 -- 172.21.15.60:0/4048237595 shutdown_connections 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c3770 0x561be70c3b90 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009bd0 tx=0x7f9bfc00e720 comp rx=0 tx=0).stop 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 --2- 172.21.15.60:0/4048237595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70c4cd0 0x561be70c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 -- 172.21.15.60:0/4048237595 >> 172.21.15.60:0/4048237595 conn(0x561be708d720 msgr2=0x561be70c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 -- 172.21.15.60:0/4048237595 shutdown_connections 2024-07-24T22:00:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.771+0000 7f9c13229740 1 -- 172.21.15.60:0/4048237595 wait complete. 2024-07-24T22:00:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.772+0000 7f9c13229740 1 -- start start 2024-07-24T22:00:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.772+0000 7f9c13229740 1 Processor -- start 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561be6e8c7a0 con 0x561be70cd0f0 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561be6fcaff0 con 0x561be70c4cd0 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561be70d2500 con 0x561be70cf5a0 2024-07-24T22:00:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c13229740 10 client.0 _ll_register_callbacks cb 0x561be6d36090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c09d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c09d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35512/0 (socket says 172.21.15.60:35512) 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c09d9b640 1 -- 172.21.15.60:0/2630146856 learned_addr learned my addr 172.21.15.60:0/2630146856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.773+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c09d9b640 1 -- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 msgr2=0x561be70ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c09d9b640 1 -- 172.21.15.60:0/2630146856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 msgr2=0x561be70e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.774+0000 7f9c09d9b640 1 -- 172.21.15.60:0/2630146856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be6fcf0a0 con 0x561be70c4cd0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001aa10 tx=0x7f9c000046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9960699dabecca8b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00020060 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f9c00017ba0 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bfc0047a0 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9c00017da0 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9c13229740 10 client.27815 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.775+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561be6d98b90 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0047a0 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9c00012070 con 0x561be70c4cd0 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 10 client.27815 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 10 client.27815 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:00:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 20 client.27815 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:49.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be70e89b0 0x561be70eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 4 client.27815 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:49.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.776+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be703b5c0 con 0x561be70e89b0 2024-07-24T22:00:49.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.780+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be70e89b0 0x561be70eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.781+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be70e89b0 0x561be70eae30 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3be0b5594bebc5d server_cookie=d94da73c8d09bc60 in_seq=0 out_seq=0 2024-07-24T22:00:49.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.781+0000 7f9bf1ffb640 10 client.27815 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:49.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.782+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2630146856 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561be703b5c0 con 0x561be70e89b0 2024-07-24T22:00:49.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.782+0000 7f9bf1ffb640 10 client.27815 handle_command_reply: tid=0 2024-07-24T22:00:49.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.782+0000 7f9c13229740 1 client.27815 shutdown 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.782+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 msgr2=0x561be70cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001aa10 tx=0x7f9c000046e0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 shutdown_connections 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be70e89b0 0x561be70eae30 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70cf5a0 0x561be70ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 --2- 172.21.15.60:0/2630146856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70c4cd0 0x561be70cc670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 >> 172.21.15.60:0/2630146856 conn(0x561be708d720 msgr2=0x561be70c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9bf1ffb640 0 client.27815 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 shutdown_connections 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 1 -- 172.21.15.60:0/2630146856 wait complete. 2024-07-24T22:00:49.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.783+0000 7f9c13229740 20 client.27815 trim_cache size 0 max 16384 2024-07-24T22:00:49.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.813+0000 7f9c13229740 1 Processor -- start 2024-07-24T22:00:49.816 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:49 smithi060 ceph-mon[33000]: pgmap v818: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-07-24T22:00:49.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.813+0000 7f9c13229740 1 -- start start 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec011290 con 0x561be70cd0f0 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec00eaa0 con 0x561be70cf5a0 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c13229740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec00c910 con 0x561be70c4cd0 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c09d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c09d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42504/0 (socket says 172.21.15.60:42504) 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c09d9b640 1 -- 172.21.15.60:0/507826457 learned_addr learned my addr 172.21.15.60:0/507826457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.814+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.815+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.816+0000 7f9c09d9b640 1 -- 172.21.15.60:0/507826457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 msgr2=0x561be70ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.816+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.816+0000 7f9c09d9b640 1 -- 172.21.15.60:0/507826457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 msgr2=0x561be70e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.816+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.816+0000 7f9c09d9b640 1 -- 172.21.15.60:0/507826457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be70bb5c0 con 0x561be70c4cd0 2024-07-24T22:00:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.817+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001a730 tx=0x7f9c00000e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a645fd0de0bbc625 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.817+0000 7f9bf3fff640 1 -- 172.21.15.60:0/507826457 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000236e0 con 0x561be70c4cd0 2024-07-24T22:00:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.817+0000 7f9bf3fff640 1 -- 172.21.15.60:0/507826457 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00032070 con 0x561be70c4cd0 2024-07-24T22:00:49.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.818+0000 7f9bf3fff640 1 -- 172.21.15.60:0/507826457 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004280 con 0x561be70c4cd0 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.818+0000 7f9c13229740 1 -- 172.21.15.60:0/507826457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 msgr2=0x561be70caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.818+0000 7f9c13229740 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001a730 tx=0x7f9c00000e40 comp rx=0 tx=0).stop 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 -- 172.21.15.60:0/507826457 shutdown_connections 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 --2- 172.21.15.60:0/507826457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70caef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 -- 172.21.15.60:0/507826457 >> 172.21.15.60:0/507826457 conn(0x561be708d720 msgr2=0x561be70cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 -- 172.21.15.60:0/507826457 shutdown_connections 2024-07-24T22:00:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.819+0000 7f9c13229740 1 -- 172.21.15.60:0/507826457 wait complete. 2024-07-24T22:00:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.820+0000 7f9c13229740 1 -- start start 2024-07-24T22:00:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.820+0000 7f9c13229740 1 Processor -- start 2024-07-24T22:00:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.821+0000 7f9c13229740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.821+0000 7f9c13229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.821+0000 7f9c13229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.821+0000 7f9c13229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec00cf60 con 0x561be70cd0f0 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c13229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec00e3d0 con 0x561be70cf5a0 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.821+0000 7f9c0a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c0a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57174/0 (socket says 172.21.15.60:57174) 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c0a59c640 1 -- 172.21.15.60:0/2372097487 learned_addr learned my addr 172.21.15.60:0/2372097487 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec00d630 con 0x561be70c4cd0 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c13229740 10 client.0 _ll_register_callbacks cb 0x7f9c0001a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.822+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.823+0000 7f9c0a59c640 1 -- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 msgr2=0x561be70e0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.823+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70e0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.823+0000 7f9c0a59c640 1 -- 172.21.15.60:0/2372097487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 msgr2=0x561be70f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:49.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.823+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:49.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.823+0000 7f9c0a59c640 1 -- 172.21.15.60:0/2372097487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be70c6040 con 0x561be70cd0f0 2024-07-24T22:00:49.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.824+0000 7f9c0a59c640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00efa0 tx=0x7f9bfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501adcc4e6be7c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:49.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.824+0000 7f9bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.825+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc01d040 con 0x561be70cd0f0 2024-07-24T22:00:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.825+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f9bfc005040 con 0x561be70cd0f0 2024-07-24T22:00:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.825+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c00032070 con 0x561be70cd0f0 2024-07-24T22:00:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.825+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9bfc005240 con 0x561be70cd0f0 2024-07-24T22:00:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.826+0000 7f9c13229740 10 client.18432 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.826+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561be7153710 con 0x561be70cd0f0 2024-07-24T22:00:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.827+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0053e0 con 0x561be70cd0f0 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9bfc025440 con 0x561be70cd0f0 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 10 client.18432 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 10 client.18432 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 20 client.18432 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be7153ab0 0x561be7155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 4 client.18432 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.6, "await_for": 45.0}] 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.6, "await_for": 45.0}) -- 0x7f9c00032070 con 0x561be7153ab0 2024-07-24T22:00:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.828+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be7153ab0 0x561be7155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:49.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.829+0000 7f9c09d9b640 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be7153ab0 0x561be7155ed0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcffd37d8b2f29d5 server_cookie=e3375866803fbc55 in_seq=0 out_seq=0 2024-07-24T22:00:49.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:49.829+0000 7f9bf1ffb640 10 client.18432 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:49 smithi123 ceph-mon[37143]: pgmap v818: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-07-24T22:00:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:49 smithi086 ceph-mon[37128]: pgmap v818: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-07-24T22:00:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:51 smithi060 ceph-mon[33000]: pgmap v819: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 1.35k op/s 2024-07-24T22:00:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:51 smithi123 ceph-mon[37143]: pgmap v819: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 1.35k op/s 2024-07-24T22:00:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:51 smithi086 ceph-mon[37128]: pgmap v819: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 1.35k op/s 2024-07-24T22:00:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:53 smithi060 ceph-mon[33000]: pgmap v820: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 1.61k op/s 2024-07-24T22:00:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:53 smithi123 ceph-mon[37143]: pgmap v820: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 1.61k op/s 2024-07-24T22:00:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:53 smithi086 ceph-mon[37128]: pgmap v820: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 1.61k op/s 2024-07-24T22:00:55.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.304+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/2372097487 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f9c00032070 con 0x561be7153ab0 2024-07-24T22:00:55.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.304+0000 7f9bf1ffb640 10 client.18432 handle_command_reply: tid=0 2024-07-24T22:00:55.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.304+0000 7f9c13229740 1 client.18432 shutdown 2024-07-24T22:00:55.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 msgr2=0x561be70f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00efa0 tx=0x7f9bfc00ecb0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 shutdown_connections 2024-07-24T22:00:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561be7153ab0 0x561be7155ed0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be70cf5a0 0x561be70f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561be70cd0f0 0x561be70f0160 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9bf1ffb640 0 client.18432 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 --2- 172.21.15.60:0/2372097487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561be70c4cd0 0x561be70e0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 >> 172.21.15.60:0/2372097487 conn(0x561be708d720 msgr2=0x561be70c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 shutdown_connections 2024-07-24T22:00:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 1 -- 172.21.15.60:0/2372097487 wait complete. 2024-07-24T22:00:55.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.305+0000 7f9c13229740 20 client.18432 trim_cache size 0 max 16384 2024-07-24T22:00:55.311 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:00:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T22:00:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:00:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 42, 2024-07-24T22:00:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:00:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "8686a43e": { 2024-07-24T22:00:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 42, 2024-07-24T22:00:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:00:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:00:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:00:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:00:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:00:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:00:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.6, 2024-07-24T22:00:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:00:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:00:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:00:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:00:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:00:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:00:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:00:55.316 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:00:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 msgr2=0x7f9bf40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 0x7f9bf40b6280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cbd00 tx=0x7f9c04005ef0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 msgr2=0x7f9c0c1a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c04031e70 tx=0x7f9c04031ea0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 shutdown_connections 2024-07-24T22:00:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9bf40b3dc0 0x7f9bf40b6280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c0c108ae0 0x7f9c0c1a07a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c0c108160 0x7f9c0c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 --2- 172.21.15.60:0/2510377900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c102750 0x7f9c0c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 >> 172.21.15.60:0/2510377900 conn(0x7f9c0c0fe200 msgr2=0x7f9c0c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 shutdown_connections 2024-07-24T22:00:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:00:55.306+0000 7f9c10e78640 1 -- 172.21.15.60:0/2510377900 wait complete. 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.538+0000 7f5380961640 1 -- 172.21.15.60:0/3189266776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c108140 msgr2=0x7f537c108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.538+0000 7f5380961640 1 --2- 172.21.15.60:0/3189266776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c108140 0x7f537c108580 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402a410 comp rx=0 tx=0).stop 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.538+0000 7f5380961640 1 -- 172.21.15.60:0/3189266776 shutdown_connections 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 --2- 172.21.15.60:0/3189266776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108ac0 0x7f537c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 --2- 172.21.15.60:0/3189266776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c108140 0x7f537c108580 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 --2- 172.21.15.60:0/3189266776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c102740 0x7f537c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 -- 172.21.15.60:0/3189266776 >> 172.21.15.60:0/3189266776 conn(0x7f537c0fe180 msgr2=0x7f537c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 -- 172.21.15.60:0/3189266776 shutdown_connections 2024-07-24T22:00:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 -- 172.21.15.60:0/3189266776 wait complete. 2024-07-24T22:00:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 Processor -- start 2024-07-24T22:00:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.539+0000 7f5380961640 1 -- start start 2024-07-24T22:00:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108140 0x7f537c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c108ac0 0x7f537c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112750 con 0x7f537c102740 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1125d0 con 0x7f537c108140 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1128d0 con 0x7f537c108ac0 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57180/0 (socket says 172.21.15.60:57180) 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 -- 172.21.15.60:0/3245524325 learned_addr learned my addr 172.21.15.60:0/3245524325 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537ad9d640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c108ac0 0x7f537c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5373fff640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108140 0x7f537c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 -- 172.21.15.60:0/3245524325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c108ac0 msgr2=0x7f537c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c108ac0 0x7f537c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 -- 172.21.15.60:0/3245524325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108140 msgr2=0x7f537c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108140 0x7f537c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 -- 172.21.15.60:0/3245524325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1a0ee0 con 0x7f537c102740 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f537a59c640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f536c00e9f0 tx=0x7f536c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f53737fe640 1 -- 172.21.15.60:0/3245524325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c00cd80 con 0x7f537c102740 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f53737fe640 1 -- 172.21.15.60:0/3245524325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0054d0 con 0x7f537c102740 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f53737fe640 1 -- 172.21.15.60:0/3245524325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c010690 con 0x7f537c102740 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.540+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c1a11d0 con 0x7f537c102740 2024-07-24T22:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.541+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a7890 con 0x7f537c102740 2024-07-24T22:00:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f53737fe640 1 -- 172.21.15.60:0/3245524325 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f536c004e90 con 0x7f537c102740 2024-07-24T22:00:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 Processor -- start 2024-07-24T22:00:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 -- start start 2024-07-24T22:00:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 0x5632f0b16ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b22c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5632f0a8c280 con 0x5632f0b168c0 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632f0a7b780 con 0x5632f0b171e0 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f538378a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5632f0a7f7c0 con 0x5632f0b23170 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.544+0000 7f537a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b22c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f537ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 0x5632f0b16ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f537ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 0x5632f0b16ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57182/0 (socket says 172.21.15.60:57182) 2024-07-24T22:00:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f537ad9d640 1 -- 172.21.15.60:0/3067840961 learned_addr learned my addr 172.21.15.60:0/3067840961 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:42548/0 (socket says 172.21.15.60:42548) 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 -- 172.21.15.60:0/3067840961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 msgr2=0x5632f0b22c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b22c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 -- 172.21.15.60:0/3067840961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 msgr2=0x5632f0b16ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 0x5632f0b16ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 -- 172.21.15.60:0/3067840961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f07df790 con 0x5632f0b23170 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f53737fe640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 0x7f53680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5373fff640 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f536402aa00 tx=0x7f5364005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4239cf8cbb9ed9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f537ad9d640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 0x7f53680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f53737fe640 1 -- 172.21.15.60:0/3245524325 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f536c0e4440 con 0x7f537c102740 2024-07-24T22:00:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.545+0000 7f5370ff9640 1 -- 172.21.15.60:0/3067840961 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364035040 con 0x5632f0b23170 2024-07-24T22:00:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.546+0000 7f5370ff9640 1 -- 172.21.15.60:0/3067840961 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53640054f0 con 0x5632f0b23170 2024-07-24T22:00:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.546+0000 7f5370ff9640 1 -- 172.21.15.60:0/3067840961 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640035f0 con 0x5632f0b23170 2024-07-24T22:00:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.546+0000 7f538378a740 1 -- 172.21.15.60:0/3067840961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 msgr2=0x5632f0b25560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.546+0000 7f538378a740 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f536402aa00 tx=0x7f5364005c00 comp rx=0 tx=0).stop 2024-07-24T22:00:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.546+0000 7f537ad9d640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 0x7f53680b6200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53740116b0 tx=0x7f5374009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 -- 172.21.15.60:0/3067840961 shutdown_connections 2024-07-24T22:00:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b25560 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b22c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 --2- 172.21.15.60:0/3067840961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b168c0 0x5632f0b16ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 -- 172.21.15.60:0/3067840961 >> 172.21.15.60:0/3067840961 conn(0x5632f0b146f0 msgr2=0x5632f0b15890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 -- 172.21.15.60:0/3067840961 shutdown_connections 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 -- 172.21.15.60:0/3067840961 wait complete. 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 -- start start 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.547+0000 7f538378a740 1 Processor -- start 2024-07-24T22:00:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b20290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5632f09b5ab0 con 0x5632f0b23170 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632f0a039e0 con 0x5632f0b171e0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5632f0b25df0 con 0x5632f0b168c0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f538378a740 10 client.0 _ll_register_callbacks cb 0x5632f078ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f5373fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f5373fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b1fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35584/0 (socket says 172.21.15.60:35584) 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f5373fff640 1 -- 172.21.15.60:0/2723685129 learned_addr learned my addr 172.21.15.60:0/2723685129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b20290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537a59c640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 -- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 msgr2=0x5632f0b22870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b22870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 -- 172.21.15.60:0/2723685129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 msgr2=0x5632f0b1fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b1fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 -- 172.21.15.60:0/2723685129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f0858620 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f537ad9d640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b20290 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5374011f40 tx=0x7f5374004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67562d91783c52cd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.548+0000 7f5361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374003c40 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632f0a2ace0 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f5374003de0 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5374004050 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f538378a740 10 client.18444 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5632f0b31590 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.549+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5632f0a2ace0 con 0x5632f0b23170 2024-07-24T22:00:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5374026070 con 0x5632f0b23170 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 10 client.18444 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 10 client.18444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 20 client.18444 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b32ce0 0x5632f0b35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 4 client.18444 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632f0a8e1d0 con 0x5632f0b32ce0 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.550+0000 7f537a59c640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b32ce0 0x5632f0b35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.551+0000 7f5361ffb640 10 client.18444 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.551+0000 7f537a59c640 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b32ce0 0x5632f0b35180 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e79016e1adcbd4 server_cookie=809c2c1d17b9a54 in_seq=0 out_seq=0 2024-07-24T22:00:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f5361ffb640 1 -- 172.21.15.60:0/2723685129 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632f0a8e1d0 con 0x5632f0b32ce0 2024-07-24T22:00:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f5361ffb640 10 client.18444 handle_command_reply: tid=0 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 client.18444 shutdown 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 msgr2=0x5632f0b20290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b20290 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5374011f40 tx=0x7f5374004ab0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 shutdown_connections 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b32ce0 0x5632f0b35180 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f5361ffb640 0 client.18444 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b20290 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b1fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 --2- 172.21.15.60:0/2723685129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b22870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 >> 172.21.15.60:0/2723685129 conn(0x5632f0b146f0 msgr2=0x5632f0b1a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 shutdown_connections 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 1 -- 172.21.15.60:0/2723685129 wait complete. 2024-07-24T22:00:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.552+0000 7f538378a740 20 client.18444 trim_cache size 0 max 16384 2024-07-24T22:00:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.570+0000 7f538378a740 1 Processor -- start 2024-07-24T22:00:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.570+0000 7f538378a740 1 -- start start 2024-07-24T22:00:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.570+0000 7f538378a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b1f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f538378a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f538378a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b3aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f538378a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ae910 con 0x5632f0b23170 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f538378a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ac120 con 0x5632f0b171e0 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f538378a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1a9f90 con 0x5632f0b168c0 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f537a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b1f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35598/0 (socket says 172.21.15.60:35598) 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f537ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b3aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 -- 172.21.15.60:0/357831733 learned_addr learned my addr 172.21.15.60:0/357831733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 -- 172.21.15.60:0/357831733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 msgr2=0x5632f0b1f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b1f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 -- 172.21.15.60:0/357831733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 msgr2=0x5632f0b3aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b3aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 -- 172.21.15.60:0/357831733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f0b1e1f0 con 0x5632f0b171e0 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f537ad9d640 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b3aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.571+0000 7f5373fff640 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f53640096b0 tx=0x7f5364000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8f9d9d3f5e90 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f5363fff640 1 -- 172.21.15.60:0/357831733 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364035040 con 0x5632f0b171e0 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f5363fff640 1 -- 172.21.15.60:0/357831733 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536404ed80 con 0x5632f0b171e0 2024-07-24T22:00:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f5363fff640 1 -- 172.21.15.60:0/357831733 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640046c0 con 0x5632f0b171e0 2024-07-24T22:00:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 -- 172.21.15.60:0/357831733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 msgr2=0x5632f0b136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f53640096b0 tx=0x7f5364000bc0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 -- 172.21.15.60:0/357831733 shutdown_connections 2024-07-24T22:00:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b23170 0x5632f0b3aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b171e0 0x5632f0b136a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 --2- 172.21.15.60:0/357831733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b168c0 0x5632f0b1f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 -- 172.21.15.60:0/357831733 >> 172.21.15.60:0/357831733 conn(0x5632f0b146f0 msgr2=0x5632f0b12f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 -- 172.21.15.60:0/357831733 shutdown_connections 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.572+0000 7f538378a740 1 -- 172.21.15.60:0/357831733 wait complete. 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 -- start start 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 Processor -- start 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 0x5632f0b266f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1aa5e0 con 0x5632f0b171e0 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1aba50 con 0x5632f0b168c0 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1aacb0 con 0x5632f0b23170 2024-07-24T22:00:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f538378a740 10 client.0 _ll_register_callbacks cb 0x7f5374009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f537a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f537a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35610/0 (socket says 172.21.15.60:35610) 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.573+0000 7f537a59c640 1 -- 172.21.15.60:0/2420311767 learned_addr learned my addr 172.21.15.60:0/2420311767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537ad9d640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5373fff640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 0x5632f0b266f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 -- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 msgr2=0x5632f0b9ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b9ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 -- 172.21.15.60:0/2420311767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 msgr2=0x5632f0b266f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 0x5632f0b266f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 -- 172.21.15.60:0/2420311767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f0b26fc0 con 0x5632f0b168c0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5373fff640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 0x5632f0b266f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f537a59c640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f536c0e2130 tx=0x7f536c0e4830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a5f65a310b9592 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:00:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0aa940 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f536c0a7280 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536404ed80 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f536c005180 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f538378a740 10 client.27839 fetch_fsmap learned FSMap version 44 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.574+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5632f0b9d0c0 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536404ed80 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f536c0e4040 con 0x5632f0b168c0 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 10 client.27839 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 10 client.27839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 20 client.27839 populate_metadata read hostname 'smithi060' 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b9d8a0 0x5632f0b9fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:00:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 4 client.27839 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:00:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632f0a89c70 con 0x5632f0b9d8a0 2024-07-24T22:00:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.575+0000 7f5373fff640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b9d8a0 0x5632f0b9fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:00:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.576+0000 7f5373fff640 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b9d8a0 0x5632f0b9fcc0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505e623087888c8e server_cookie=e297e336b9304bd1 in_seq=0 out_seq=0 2024-07-24T22:00:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.576+0000 7f5361ffb640 10 client.27839 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f5361ffb640 1 -- 172.21.15.60:0/2420311767 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632f0a89c70 con 0x5632f0b9d8a0 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f5361ffb640 10 client.27839 handle_command_reply: tid=0 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 client.27839 shutdown 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 msgr2=0x5632f0b261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f536c0e2130 tx=0x7f536c0e4830 comp rx=0 tx=0).stop 2024-07-24T22:00:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 shutdown_connections 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5632f0b9d8a0 0x5632f0b9fcc0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5632f0b23170 0x5632f0b9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.591+0000 7f5361ffb640 0 client.27839 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5632f0b171e0 0x5632f0b266f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 1 --2- 172.21.15.60:0/2420311767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f0b168c0 0x5632f0b261b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 >> 172.21.15.60:0/2420311767 conn(0x5632f0b146f0 msgr2=0x5632f0b137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 shutdown_connections 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 1 -- 172.21.15.60:0/2420311767 wait complete. 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.592+0000 7f538378a740 20 client.27839 trim_cache size 0 max 16384 2024-07-24T22:00:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:00:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:00:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:00:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 msgr2=0x7f53680b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 0x7f53680b6200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53740116b0 tx=0x7f5374009500 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 msgr2=0x7f537c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f536c00e9f0 tx=0x7f536c00edf0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 shutdown_connections 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f53680b3d40 0x7f53680b6200 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537c108ac0 0x7f537c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.593+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f537c108140 0x7f537c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.594+0000 7f5380961640 1 --2- 172.21.15.60:0/3245524325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c102740 0x7f537c19ae40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.594+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 >> 172.21.15.60:0/3245524325 conn(0x7f537c0fe180 msgr2=0x7f537c105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:00:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.594+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 shutdown_connections 2024-07-24T22:00:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:00:55.594+0000 7f5380961640 1 -- 172.21.15.60:0/3245524325 wait complete. 2024-07-24T22:00:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:55 smithi060 ceph-mon[33000]: pgmap v821: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-07-24T22:00:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:55 smithi123 ceph-mon[37143]: pgmap v821: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-07-24T22:00:56.028 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8686a43e', quiesce took 16.2 seconds. Will release after: 4.400000000000002 2024-07-24T22:00:56.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:55 smithi086 ceph-mon[37128]: pgmap v821: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-07-24T22:00:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:57 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:57 smithi060 ceph-mon[33000]: pgmap v822: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.56k op/s 2024-07-24T22:00:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:57 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:57 smithi123 ceph-mon[37143]: pgmap v822: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.56k op/s 2024-07-24T22:00:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:57 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:00:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:57 smithi086 ceph-mon[37128]: pgmap v822: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 1.56k op/s 2024-07-24T22:00:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:00:59 smithi123 ceph-mon[37143]: pgmap v823: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 988 KiB/s wr, 1.48k op/s 2024-07-24T22:01:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:00:59 smithi086 ceph-mon[37128]: pgmap v823: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 988 KiB/s wr, 1.48k op/s 2024-07-24T22:01:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:00:59 smithi060 ceph-mon[33000]: pgmap v823: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 988 KiB/s wr, 1.48k op/s 2024-07-24T22:01:00.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:01:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:00.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:00 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:01:00.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:00 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:01:01.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:00 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:01:01.734 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:01:01.734 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:01 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:01:01.735 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:01 smithi086 ceph-mon[37128]: pgmap v824: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 593 KiB/s wr, 1.22k op/s 2024-07-24T22:01:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:01:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:01 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:01:01.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:01 smithi123 ceph-mon[37143]: pgmap v824: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 593 KiB/s wr, 1.22k op/s 2024-07-24T22:01:01.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:01:01.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:01 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:01:01.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:01 smithi060 ceph-mon[33000]: pgmap v824: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 593 KiB/s wr, 1.22k op/s 2024-07-24T22:01:02.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 -- 172.21.15.60:0/1142899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a40101690 msgr2=0x7f5a40101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 --2- 172.21.15.60:0/1142899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a40101690 0x7f5a40101a70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009900 tx=0x7f5a3002a410 comp rx=0 tx=0).stop 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 -- 172.21.15.60:0/1142899429 shutdown_connections 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 --2- 172.21.15.60:0/1142899429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 0x7f5a4010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 --2- 172.21.15.60:0/1142899429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a401070a0 0x7f5a401074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 --2- 172.21.15.60:0/1142899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a40101690 0x7f5a40101a70 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 -- 172.21.15.60:0/1142899429 >> 172.21.15.60:0/1142899429 conn(0x7f5a400fd100 msgr2=0x7f5a400ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 -- 172.21.15.60:0/1142899429 shutdown_connections 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.507+0000 7f5a44999640 1 -- 172.21.15.60:0/1142899429 wait complete. 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 Processor -- start 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 -- start start 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 0x7f5a401977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 0x7f5a40197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:02.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 0x7f5a40198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111630 con 0x7f5a401070a0 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401114b0 con 0x7f5a40107a20 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a44999640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401117b0 con 0x7f5a40101690 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a3dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 0x7f5a40197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a3dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 0x7f5a40197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40242/0 (socket says 172.21.15.60:40242) 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a3dd9b640 1 -- 172.21.15.60:0/3763320287 learned_addr learned my addr 172.21.15.60:0/3763320287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.508+0000 7f5a3e59c640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 0x7f5a401977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3ed9d640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 0x7f5a40198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 -- 172.21.15.60:0/3763320287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 msgr2=0x7f5a40198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 0x7f5a40198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 -- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 msgr2=0x7f5a40197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 0x7f5a40197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 -- 172.21.15.60:0/3763320287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a401a6f90 con 0x7f5a40101690 2024-07-24T22:01:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a3e59c640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 0x7f5a401977b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5a3002a930 tx=0x7f5a30004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3003f030 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a401a7220 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30003aa0 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.509+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30035070 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.510+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a7750 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.510+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a40101a70 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.512+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5a30042030 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.513+0000 7f5a2f7fe640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 0x7f5a280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.513+0000 7f5a3dd9b640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 0x7f5a280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.513+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8609+0+0 (secure 0 0 0) 0x7f5a301047f0 con 0x7f5a40101690 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.514+0000 7f5a3dd9b640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 0x7f5a280b6200 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a40106880 tx=0x7f5a3400a380 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:02.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.516+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5a300cb450 con 0x7f5a40101690 2024-07-24T22:01:02.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.749+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a401a7b40 con 0x7f5a40101690 2024-07-24T22:01:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.750+0000 7f5a2f7fe640 1 -- 172.21.15.60:0/3763320287 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f5a300cf9a0 con 0x7f5a40101690 2024-07-24T22:01:02.753 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:01:02.753 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:01:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:01:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 msgr2=0x7f5a280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 0x7f5a280b6200 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a40106880 tx=0x7f5a3400a380 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 msgr2=0x7f5a401977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 0x7f5a401977b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5a3002a930 tx=0x7f5a30004270 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 shutdown_connections 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5a280b3d40 0x7f5a280b6200 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a40107a20 0x7f5a40198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a401070a0 0x7f5a40197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 --2- 172.21.15.60:0/3763320287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a40101690 0x7f5a401977b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 >> 172.21.15.60:0/3763320287 conn(0x7f5a400fd100 msgr2=0x7f5a400fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 shutdown_connections 2024-07-24T22:01:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:02.753+0000 7f5a44999640 1 -- 172.21.15.60:0/3763320287 wait complete. 2024-07-24T22:01:03.375 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 8686a43e --release --await' 2024-07-24T22:01:03.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 8686a43e --release --await 2024-07-24T22:01:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:03.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:03 smithi060 ceph-mon[33000]: pgmap v825: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 269 KiB/s wr, 1.06k op/s 2024-07-24T22:01:03.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:03 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3763320287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:03.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:03 smithi060 ceph-mon[33000]: osdmap e105: 12 total, 12 up, 12 in 2024-07-24T22:01:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:03 smithi123 ceph-mon[37143]: pgmap v825: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 269 KiB/s wr, 1.06k op/s 2024-07-24T22:01:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:03 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3763320287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:03 smithi123 ceph-mon[37143]: osdmap e105: 12 total, 12 up, 12 in 2024-07-24T22:01:04.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:03 smithi086 ceph-mon[37128]: pgmap v825: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 269 KiB/s wr, 1.06k op/s 2024-07-24T22:01:04.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:03 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3763320287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:04.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:03 smithi086 ceph-mon[37128]: osdmap e105: 12 total, 12 up, 12 in 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 -- 172.21.15.60:0/738157082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 msgr2=0x7fc5e010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 --2- 172.21.15.60:0/738157082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e010e1e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009ca0 tx=0x7fc5d802a8c0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 -- 172.21.15.60:0/738157082 shutdown_connections 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 --2- 172.21.15.60:0/738157082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e010e1e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 --2- 172.21.15.60:0/738157082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0107060 0x7fc5e01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 --2- 172.21.15.60:0/738157082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0101650 0x7fc5e0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 -- 172.21.15.60:0/738157082 >> 172.21.15.60:0/738157082 conn(0x7fc5e00fd0c0 msgr2=0x7fc5e00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 -- 172.21.15.60:0/738157082 shutdown_connections 2024-07-24T22:01:05.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.169+0000 7fc5e857d640 1 -- 172.21.15.60:0/738157082 wait complete. 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 Processor -- start 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 -- start start 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 0x7fc5e0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 0x7fc5e01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01116c0 con 0x7fc5e01079e0 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e0111540 con 0x7fc5e0101650 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e857d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e0111840 con 0x7fc5e0107060 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40260/0 (socket says 172.21.15.60:40260) 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/2045019424 learned_addr learned my addr 172.21.15.60:0/2045019424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6319640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 0x7fc5e0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 0x7fc5e01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/2045019424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 msgr2=0x7fc5e01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 0x7fc5e01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/2045019424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 msgr2=0x7fc5e0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 0x7fc5e0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/2045019424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e019ed10 con 0x7fc5e01079e0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e6319640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 0x7fc5e0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.170+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 0x7fc5e01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009430 tx=0x7fc5d80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5d77fe640 1 -- 172.21.15.60:0/2045019424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003c30 con 0x7fc5e01079e0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5d77fe640 1 -- 172.21.15.60:0/2045019424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d8003dd0 con 0x7fc5e01079e0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e019efa0 con 0x7fc5e01079e0 2024-07-24T22:01:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5d77fe640 1 -- 172.21.15.60:0/2045019424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d802faa0 con 0x7fc5e01079e0 2024-07-24T22:01:05.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.171+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e01a5740 con 0x7fc5e01079e0 2024-07-24T22:01:05.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 Processor -- start 2024-07-24T22:01:05.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5d77fe640 1 -- 172.21.15.60:0/2045019424 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc5d802fc40 con 0x7fc5e01079e0 2024-07-24T22:01:05.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 -- start start 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78fede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ffc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560e78f64dd0 con 0x560e78fee350 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e78f542d0 con 0x560e78ffa2e0 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e78f58310 con 0x560e78feda30 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.174+0000 7fc5e6319640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e5b18640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ffc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40276/0 (socket says 172.21.15.60:40276) 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 -- 172.21.15.60:0/572336611 learned_addr learned my addr 172.21.15.60:0/572336611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78fede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 -- 172.21.15.60:0/572336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 msgr2=0x560e78fede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78fede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 -- 172.21.15.60:0/572336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 msgr2=0x560e78ffc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ffc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 -- 172.21.15.60:0/572336611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e78cb4b00 con 0x560e78fee350 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 0x7fc5cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e6319640 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00aa10 tx=0x7fc5dc00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d631cd9ee6a13929 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 0x7fc5cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5d77fe640 1 -- 172.21.15.60:0/2045019424 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc5d8105740 con 0x7fc5e01079e0 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.175+0000 7fc5d4ff9640 1 -- 172.21.15.60:0/572336611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc015070 con 0x560e78fee350 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5d4ff9640 1 -- 172.21.15.60:0/572336611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5dc00cbe0 con 0x560e78fee350 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5d4ff9640 1 -- 172.21.15.60:0/572336611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0105b0 con 0x560e78fee350 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 -- 172.21.15.60:0/572336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 msgr2=0x560e78ff9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 0x7fc5cc0b6280 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00058c0 tx=0x7fc5d000b3a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00aa10 tx=0x7fc5dc00aed0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 -- 172.21.15.60:0/572336611 shutdown_connections 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ffc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e78ff9da0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/572336611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78fede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 -- 172.21.15.60:0/572336611 >> 172.21.15.60:0/572336611 conn(0x560e78feb860 msgr2=0x560e78feca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 -- 172.21.15.60:0/572336611 shutdown_connections 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.176+0000 7fc5ea92e740 1 -- 172.21.15.60:0/572336611 wait complete. 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 -- start start 2024-07-24T22:01:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 Processor -- start 2024-07-24T22:01:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78ff9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 0x560e78ff71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560e78db7bb0 con 0x560e78ffa2e0 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e78ef6900 con 0x560e78fee350 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e78ffcf60 con 0x560e78feda30 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5ea92e740 10 client.0 _ll_register_callbacks cb 0x560e78f00c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40292/0 (socket says 172.21.15.60:40292) 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/3919235972 learned_addr learned my addr 172.21.15.60:0/3919235972 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5e6319640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78ff9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.177+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 0x560e78ff71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 msgr2=0x560e78ff9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78ff9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/3919235972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 msgr2=0x560e78ff71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 0x560e78ff71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/3919235972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e78d2b960 con 0x560e78ffa2e0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6319640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78ff9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 0x560e78ff71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80c84b0 tx=0x7fc5d80cbcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b0b55347e13d3c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80e6060 con 0x560e78ffa2e0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc5d80d5050 con 0x560e78ffa2e0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5dc00cbe0 con 0x560e78ffa2e0 2024-07-24T22:01:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc5d80cc430 con 0x560e78ffa2e0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5ea92e740 10 client.18468 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560e79008a20 con 0x560e78ffa2e0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.178+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5dc00cbe0 con 0x560e78ffa2e0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc5d8004bd0 con 0x560e78ffa2e0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 10 client.18468 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 10 client.18468 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 20 client.18468 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e79009e60 0x560e7900c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 4 client.18468 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e78f66d20 con 0x560e79009e60 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5e6319640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e79009e60 0x560e7900c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5e6319640 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e79009e60 0x560e7900c300 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87636ab30290f03 server_cookie=3a86ff6bffbad88d in_seq=0 out_seq=0 2024-07-24T22:01:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.179+0000 7fc5c9ffb640 10 client.18468 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/3919235972 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e78f66d20 con 0x560e79009e60 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5c9ffb640 10 client.18468 handle_command_reply: tid=0 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 client.18468 shutdown 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 msgr2=0x560e78ff7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80c84b0 tx=0x7fc5d80cbcd0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 shutdown_connections 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e79009e60 0x560e7900c300 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ff7930 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78fee350 0x560e78ff71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5c9ffb640 0 client.18468 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/3919235972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e78ff9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 >> 172.21.15.60:0/3919235972 conn(0x560e78feb860 msgr2=0x560e78ff1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 shutdown_connections 2024-07-24T22:01:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 1 -- 172.21.15.60:0/3919235972 wait complete. 2024-07-24T22:01:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.180+0000 7fc5ea92e740 20 client.18468 trim_cache size 0 max 16384 2024-07-24T22:01:05.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.198+0000 7fc5ea92e740 1 Processor -- start 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 -- start start 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e79011c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ff43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01b0c00 con 0x560e78fee350 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01ae410 con 0x560e78ffa2e0 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01ac280 con 0x560e78feda30 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40306/0 (socket says 172.21.15.60:40306) 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 -- 172.21.15.60:0/722803594 learned_addr learned my addr 172.21.15.60:0/722803594 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ff43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 -- 172.21.15.60:0/722803594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 msgr2=0x560e79011c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e6319640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e79011c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e79011c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 -- 172.21.15.60:0/722803594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 msgr2=0x560e78ff43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ff43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e5b18640 1 -- 172.21.15.60:0/722803594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e78fe6f00 con 0x560e78fee350 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.199+0000 7fc5e6319640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e79011c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.200+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00133d0 tx=0x7fc5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e442006dc6fdf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.200+0000 7fc5cbfff640 1 -- 172.21.15.60:0/722803594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0012650 con 0x560e78fee350 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.200+0000 7fc5cbfff640 1 -- 172.21.15.60:0/722803594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0012ca0 con 0x560e78fee350 2024-07-24T22:01:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.200+0000 7fc5cbfff640 1 -- 172.21.15.60:0/722803594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0022c90 con 0x560e78fee350 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- 172.21.15.60:0/722803594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 msgr2=0x560e79012150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00133d0 tx=0x7fc5d00046c0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- 172.21.15.60:0/722803594 shutdown_connections 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78ffa2e0 0x560e78ff43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78fee350 0x560e79012150 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/722803594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78feda30 0x560e79011c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- 172.21.15.60:0/722803594 >> 172.21.15.60:0/722803594 conn(0x560e78feb860 msgr2=0x560e78ff23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- 172.21.15.60:0/722803594 shutdown_connections 2024-07-24T22:01:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- 172.21.15.60:0/722803594 wait complete. 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 -- start start 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.201+0000 7fc5ea92e740 1 Processor -- start 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78feda30 0x560e79002540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78fee350 0x560e79002ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01ac8d0 con 0x560e78ffa2e0 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01add40 con 0x560e78feda30 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e01acfa0 con 0x560e78fee350 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5ea92e740 10 client.0 _ll_register_callbacks cb 0x7fc5d80cbe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40316/0 (socket says 172.21.15.60:40316) 2024-07-24T22:01:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/1514467316 learned_addr learned my addr 172.21.15.60:0/1514467316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e5b18640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78fee350 0x560e79002ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6319640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78feda30 0x560e79002540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78fee350 msgr2=0x560e79002ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78fee350 0x560e79002ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/1514467316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78feda30 msgr2=0x560e79002540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78feda30 0x560e79002540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.202+0000 7fc5e6b1a640 1 -- 172.21.15.60:0/1514467316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e78ff7510 con 0x560e78ffa2e0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5e6b1a640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc5d81033e0 tx=0x7fc5d8103360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3af07f161ac5a58 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80e6070 con 0x560e78ffa2e0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc5d80c6b50 con 0x560e78ffa2e0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0012ca0 con 0x560e78ffa2e0 2024-07-24T22:01:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc5d80d5260 con 0x560e78ffa2e0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 10 client.18474 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560e79079ae0 con 0x560e78ffa2e0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0012ca0 con 0x560e78ffa2e0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc5d80b20d0 con 0x560e78ffa2e0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 10 client.18474 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 10 client.18474 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 20 client.18474 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e7907a020 0x560e7907c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 4 client.18474 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8686a43e", "release": true, "await": true}] 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.203+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8686a43e", "release": true, "await": true}) -- 0x560e78f627c0 con 0x560e7907a020 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.204+0000 7fc5e6319640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e7907a020 0x560e7907c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.204+0000 7fc5e6319640 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e7907a020 0x560e7907c440 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b242fb304de93190 server_cookie=332ffaff59500429 in_seq=0 out_seq=0 2024-07-24T22:01:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.204+0000 7fc5c9ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5c9ffb640 1 -- 172.21.15.60:0/1514467316 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e78f627c0 con 0x560e7907a020 2024-07-24T22:01:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5c9ffb640 10 client.18474 handle_command_reply: tid=0 2024-07-24T22:01:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 client.18474 shutdown 2024-07-24T22:01:05.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:05 smithi060 ceph-mon[33000]: pgmap v827: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 73 KiB/s wr, 822 op/s 2024-07-24T22:01:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 msgr2=0x560e78ffd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc5d81033e0 tx=0x7fc5d8103360 comp rx=0 tx=0).stop 2024-07-24T22:01:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 shutdown_connections 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560e7907a020 0x560e7907c440 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e78ffa2e0 0x560e78ffd400 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e78fee350 0x560e79002ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 --2- 172.21.15.60:0/1514467316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e78feda30 0x560e79002540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5c9ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 >> 172.21.15.60:0/1514467316 conn(0x560e78feb860 msgr2=0x560e78ff87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 shutdown_connections 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.977+0000 7fc5ea92e740 1 -- 172.21.15.60:0/1514467316 wait complete. 2024-07-24T22:01:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.978+0000 7fc5ea92e740 20 client.18474 trim_cache size 0 max 16384 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 44, 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "8686a43e": { 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 44, 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.6, 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:05.984 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.978+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 msgr2=0x7fc5cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.978+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 0x7fc5cc0b6280 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00058c0 tx=0x7fc5d000b3a0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.978+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 msgr2=0x7fc5e019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.978+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009430 tx=0x7fc5d80319f0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 shutdown_connections 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc5cc0b3dc0 0x7fc5cc0b6280 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e01079e0 0x7fc5e019e540 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5e0107060 0x7fc5e01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 --2- 172.21.15.60:0/2045019424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e0101650 0x7fc5e0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 >> 172.21.15.60:0/2045019424 conn(0x7fc5e00fd0c0 msgr2=0x7fc5e006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 shutdown_connections 2024-07-24T22:01:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:05.979+0000 7fc5e857d640 1 -- 172.21.15.60:0/2045019424 wait complete. 2024-07-24T22:01:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:05 smithi123 ceph-mon[37143]: pgmap v827: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 73 KiB/s wr, 822 op/s 2024-07-24T22:01:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:05 smithi086 ceph-mon[37128]: pgmap v827: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 73 KiB/s wr, 822 op/s 2024-07-24T22:01:06.806 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8686a43e', total seconds elapsed: 26.9 2024-07-24T22:01:06.806 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.6 2024-07-24T22:01:06.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:01:07.168 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:07.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:07 smithi060 ceph-mon[33000]: pgmap v828: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 72 KiB/s wr, 706 op/s 2024-07-24T22:01:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:07 smithi123 ceph-mon[37143]: pgmap v828: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 72 KiB/s wr, 706 op/s 2024-07-24T22:01:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:07 smithi086 ceph-mon[37128]: pgmap v828: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 72 KiB/s wr, 706 op/s 2024-07-24T22:01:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- 172.21.15.60:0/3660409042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81015d0 msgr2=0x7fc6a81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/3660409042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81015d0 0x7fc6a81019b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009900 tx=0x7fc6a402a410 comp rx=0 tx=0).stop 2024-07-24T22:01:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- 172.21.15.60:0/3660409042 shutdown_connections 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/3660409042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a8107950 0x7fc6a810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/3660409042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8106fd0 0x7fc6a8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/3660409042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a81015d0 0x7fc6a81019b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- 172.21.15.60:0/3660409042 >> 172.21.15.60:0/3660409042 conn(0x7fc6a80fd030 msgr2=0x7fc6a80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- 172.21.15.60:0/3660409042 shutdown_connections 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- 172.21.15.60:0/3660409042 wait complete. 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 Processor -- start 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.690+0000 7fc6b06a9640 1 -- start start 2024-07-24T22:01:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81015d0 0x7fc6a8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 0x7fc6a819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8111580 con 0x7fc6a8106fd0 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8111400 con 0x7fc6a8107950 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8111700 con 0x7fc6a81015d0 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6ae445640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81015d0 0x7fc6a8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49772/0 (socket says 172.21.15.60:49772) 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 -- 172.21.15.60:0/503293653 learned_addr learned my addr 172.21.15.60:0/503293653 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6aec46640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 0x7fc6a819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 -- 172.21.15.60:0/503293653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81015d0 msgr2=0x7fc6a8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81015d0 0x7fc6a8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 -- 172.21.15.60:0/503293653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 msgr2=0x7fc6a819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 0x7fc6a819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 -- 172.21.15.60:0/503293653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a819ed00 con 0x7fc6a8106fd0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6aec46640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 0x7fc6a819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6adc44640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc69800d950 tx=0x7fc69800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc69f7fe640 1 -- 172.21.15.60:0/503293653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698017070 con 0x7fc6a8106fd0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a819eff0 con 0x7fc6a8106fd0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc69f7fe640 1 -- 172.21.15.60:0/503293653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc698009c30 con 0x7fc6a8106fd0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc69f7fe640 1 -- 172.21.15.60:0/503293653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69800f620 con 0x7fc6a8106fd0 2024-07-24T22:01:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.691+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81a5890 con 0x7fc6a8106fd0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 Processor -- start 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc69f7fe640 1 -- 172.21.15.60:0/503293653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc698010460 con 0x7fc6a8106fd0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 -- start start 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559a119e0280 con 0x559a11a7b580 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a119cf780 con 0x559a11a6a8c0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.694+0000 7fc6b3400740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a119d37c0 con 0x559a11a6b1e0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46172/0 (socket says 172.21.15.60:46172) 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 -- 172.21.15.60:0/3097798923 learned_addr learned my addr 172.21.15.60:0/3097798923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6adc44640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49776/0 (socket says 172.21.15.60:49776) 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6aec46640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 -- 172.21.15.60:0/3097798923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 msgr2=0x559a11a6aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 -- 172.21.15.60:0/3097798923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 msgr2=0x559a11a7d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 -- 172.21.15.60:0/3097798923 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a11733790 con 0x559a11a6b1e0 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6adc44640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc69f7fe640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 0x7fc6940b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6ae445640 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009900 tx=0x7fc6a4032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d070ef678b2a90b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc69f7fe640 1 -- 172.21.15.60:0/503293653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc6980e3c70 con 0x7fc6a8106fd0 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc6aec46640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 0x7fc6940b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.695+0000 7fc69cff9640 1 -- 172.21.15.60:0/3097798923 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4040070 con 0x559a11a6b1e0 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc69cff9640 1 -- 172.21.15.60:0/3097798923 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a4004030 con 0x559a11a6b1e0 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc69cff9640 1 -- 172.21.15.60:0/3097798923 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4035040 con 0x559a11a6b1e0 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 -- 172.21.15.60:0/3097798923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 msgr2=0x559a11a7b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009900 tx=0x7fc6a4032bd0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6aec46640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 0x7fc6940b6200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0005fd0 tx=0x7fc6a0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 -- 172.21.15.60:0/3097798923 shutdown_connections 2024-07-24T22:01:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a7b040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 --2- 172.21.15.60:0/3097798923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 -- 172.21.15.60:0/3097798923 >> 172.21.15.60:0/3097798923 conn(0x559a11a686f0 msgr2=0x559a11a69890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 -- 172.21.15.60:0/3097798923 shutdown_connections 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.696+0000 7fc6b3400740 1 -- 172.21.15.60:0/3097798923 wait complete. 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 -- start start 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 Processor -- start 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a7acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559a11909ab0 con 0x559a11a7b580 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a119579e0 con 0x559a11a6a8c0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a11a7e200 con 0x559a11a6b1e0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6b3400740 10 client.0 _ll_register_callbacks cb 0x559a116dfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49790/0 (socket says 172.21.15.60:49790) 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6aec46640 1 -- 172.21.15.60:0/2153851417 learned_addr learned my addr 172.21.15.60:0/2153851417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6ae445640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a7acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6adc44640 1 -- 172.21.15.60:0/2153851417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 msgr2=0x559a11a7acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6adc44640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a7acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6adc44640 1 -- 172.21.15.60:0/2153851417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 msgr2=0x559a11a786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.697+0000 7fc6adc44640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6adc44640 1 -- 172.21.15.60:0/2153851417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a117ac620 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6adc44640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a781a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc6980e1390 tx=0x7fc698005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ed4050d27e02226 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698020070 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a1197ece0 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc6980a6e40 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc6980aa620 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 10 client.27772 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a11a899d0 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559a1197ece0 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc6980e1ae0 con 0x559a11a6b1e0 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 10 client.27772 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 10 client.27772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 20 client.27772 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a8d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 4 client.27772 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.698+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a119e21d0 con 0x559a11a8ae90 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.699+0000 7fc6ae445640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a8d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.699+0000 7fc6ae445640 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a8d280 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4c4faddad8d6d1 server_cookie=e459095e118b8b89 in_seq=0 out_seq=0 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.699+0000 7fc691ffb640 10 client.27772 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc691ffb640 1 -- 172.21.15.60:0/2153851417 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a119e21d0 con 0x559a11a8ae90 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc691ffb640 10 client.27772 handle_command_reply: tid=0 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 client.27772 shutdown 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 msgr2=0x559a11a781a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a781a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc6980e1390 tx=0x7fc698005f70 comp rx=0 tx=0).stop 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 shutdown_connections 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a8d280 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc691ffb640 0 client.27772 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a781a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2153851417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a7acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 >> 172.21.15.60:0/2153851417 conn(0x559a11a686f0 msgr2=0x559a11a6e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 shutdown_connections 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 1 -- 172.21.15.60:0/2153851417 wait complete. 2024-07-24T22:01:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.700+0000 7fc6b3400740 20 client.27772 trim_cache size 0 max 16384 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 Processor -- start 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 -- start start 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6a8c0 0x559a11a76a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6b1e0 0x559a11a76f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81ae950 con 0x559a11a7b580 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81ac160 con 0x559a11a6b1e0 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6b3400740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81a9fd0 con 0x559a11a6a8c0 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6ae445640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6a8c0 0x559a11a76a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49796/0 (socket says 172.21.15.60:49796) 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6b1e0 0x559a11a76f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.716+0000 7fc6aec46640 1 -- 172.21.15.60:0/2891502223 learned_addr learned my addr 172.21.15.60:0/2891502223 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 -- 172.21.15.60:0/2891502223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6a8c0 msgr2=0x559a11a76a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6a8c0 0x559a11a76a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 -- 172.21.15.60:0/2891502223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6b1e0 msgr2=0x559a11a76f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6b1e0 0x559a11a76f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 -- 172.21.15.60:0/2891502223 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a11a8d5f0 con 0x559a11a7b580 2024-07-24T22:01:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc6aec46640 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0010840 tx=0x7fc6a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce7cdf6596908d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc693fff640 1 -- 172.21.15.60:0/2891502223 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001c430 con 0x559a11a7b580 2024-07-24T22:01:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc693fff640 1 -- 172.21.15.60:0/2891502223 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a001ca80 con 0x559a11a7b580 2024-07-24T22:01:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.717+0000 7fc693fff640 1 -- 172.21.15.60:0/2891502223 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001fac0 con 0x559a11a7b580 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 -- 172.21.15.60:0/2891502223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 msgr2=0x559a11a7d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0010840 tx=0x7fc6a0015000 comp rx=0 tx=0).stop 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 -- 172.21.15.60:0/2891502223 shutdown_connections 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a7d260 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6b1e0 0x559a11a76f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 --2- 172.21.15.60:0/2891502223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6a8c0 0x559a11a76a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 -- 172.21.15.60:0/2891502223 >> 172.21.15.60:0/2891502223 conn(0x559a11a686f0 msgr2=0x559a11a78170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 -- 172.21.15.60:0/2891502223 shutdown_connections 2024-07-24T22:01:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.718+0000 7fc6b3400740 1 -- 172.21.15.60:0/2891502223 wait complete. 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.719+0000 7fc6b3400740 1 -- start start 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.719+0000 7fc6b3400740 1 Processor -- start 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.719+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.719+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a92af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.719+0000 7fc6b3400740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6b3400740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81aa620 con 0x559a11a7b580 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6b3400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81aba90 con 0x559a11a6a8c0 2024-07-24T22:01:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6b3400740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81aacf0 con 0x559a11a6b1e0 2024-07-24T22:01:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6b3400740 10 client.0 _ll_register_callbacks cb 0x7fc69800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6adc44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a92af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6ae445640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49808/0 (socket says 172.21.15.60:49808) 2024-07-24T22:01:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 -- 172.21.15.60:0/1177535213 learned_addr learned my addr 172.21.15.60:0/1177535213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 -- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 msgr2=0x559a11a92af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a92af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 -- 172.21.15.60:0/1177535213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 msgr2=0x559a11a6ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 -- 172.21.15.60:0/1177535213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a11a93720 con 0x559a11a7b580 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6ae445640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6aec46640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc6a001ce40 tx=0x7fc6a001ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8536100f2bdaa585 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004c50 con 0x559a11a7b580 2024-07-24T22:01:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7fc6a002e070 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.720+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a001ca80 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc6a0004df0 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 10 client.18492 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a11a6bb70 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a001ca80 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc6a001f890 con 0x559a11a7b580 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 10 client.18492 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 20 client.18492 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a6c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 4 client.18492 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a119ddc70 con 0x559a11a8ae90 2024-07-24T22:01:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.721+0000 7fc6ae445640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a6c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.722+0000 7fc691ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.722+0000 7fc6ae445640 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a6c2f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2dd3e338af266b server_cookie=5472c2072277083e in_seq=0 out_seq=0 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc691ffb640 1 -- 172.21.15.60:0/1177535213 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a119ddc70 con 0x559a11a8ae90 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc691ffb640 10 client.18492 handle_command_reply: tid=0 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 client.18492 shutdown 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 msgr2=0x559a11a93030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc6a001ce40 tx=0x7fc6a001ce70 comp rx=0 tx=0).stop 2024-07-24T22:01:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 shutdown_connections 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559a11a8ae90 0x559a11a6c2f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a11a7b580 0x559a11a93030 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a11a6b1e0 0x559a11a92af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 --2- 172.21.15.60:0/1177535213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a11a6a8c0 0x559a11a6ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc691ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 >> 172.21.15.60:0/1177535213 conn(0x559a11a686f0 msgr2=0x559a11a78170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 shutdown_connections 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 1 -- 172.21.15.60:0/1177535213 wait complete. 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.736+0000 7fc6b3400740 20 client.18492 trim_cache size 0 max 16384 2024-07-24T22:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:01:08.740 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:01:08.740 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:01:08.740 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 msgr2=0x7fc6940b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 0x7fc6940b6200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0005fd0 tx=0x7fc6a0009500 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 msgr2=0x7fc6a81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc69800d950 tx=0x7fc69800de10 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 shutdown_connections 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc6940b3d40 0x7fc6940b6200 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6a8107950 0x7fc6a819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a8106fd0 0x7fc6a81991a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 --2- 172.21.15.60:0/503293653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81015d0 0x7fc6a8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 >> 172.21.15.60:0/503293653 conn(0x7fc6a80fd030 msgr2=0x7fc6a8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:08.741 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 shutdown_connections 2024-07-24T22:01:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:08.737+0000 7fc6b06a9640 1 -- 172.21.15.60:0/503293653 wait complete. 2024-07-24T22:01:09.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.285+0000 7f5ae542b640 1 -- 172.21.15.60:0/57760963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0101650 msgr2=0x7f5ae0101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:09.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.285+0000 7f5ae542b640 1 --2- 172.21.15.60:0/57760963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0101650 0x7f5ae0101a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009900 tx=0x7f5ad402a440 comp rx=0 tx=0).stop 2024-07-24T22:01:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.288+0000 7f5ae542b640 1 -- 172.21.15.60:0/57760963 shutdown_connections 2024-07-24T22:01:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.288+0000 7f5ae542b640 1 --2- 172.21.15.60:0/57760963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae01079e0 0x7f5ae010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.288+0000 7f5ae542b640 1 --2- 172.21.15.60:0/57760963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae0107060 0x7f5ae01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.288+0000 7f5ae542b640 1 --2- 172.21.15.60:0/57760963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0101650 0x7f5ae0101a30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.288+0000 7f5ae542b640 1 -- 172.21.15.60:0/57760963 >> 172.21.15.60:0/57760963 conn(0x7f5ae00fd0c0 msgr2=0x7f5ae00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.289+0000 7f5ae542b640 1 -- 172.21.15.60:0/57760963 shutdown_connections 2024-07-24T22:01:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.289+0000 7f5ae542b640 1 -- 172.21.15.60:0/57760963 wait complete. 2024-07-24T22:01:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.291+0000 7f5ae542b640 1 Processor -- start 2024-07-24T22:01:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.291+0000 7f5ae542b640 1 -- start start 2024-07-24T22:01:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.291+0000 7f5ae542b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 0x7f5ae0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.291+0000 7f5ae542b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ae542b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae01079e0 0x7f5ae019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ae542b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01115f0 con 0x7f5ae01079e0 2024-07-24T22:01:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ae542b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0111470 con 0x7f5ae0101650 2024-07-24T22:01:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ae542b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0111770 con 0x7f5ae0107060 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ade7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 0x7f5ae0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 0x7f5ae0197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60714/0 (socket says 172.21.15.60:60714) 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5adeffd640 1 -- 172.21.15.60:0/2639333004 learned_addr learned my addr 172.21.15.60:0/2639333004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.292+0000 7f5ade7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46206/0 (socket says 172.21.15.60:46206) 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5adf7fe640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae01079e0 0x7f5ae019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 -- 172.21.15.60:0/2639333004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 msgr2=0x7f5ae0197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 0x7f5ae0197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 -- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae01079e0 msgr2=0x7f5ae019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae01079e0 0x7f5ae019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 -- 172.21.15.60:0/2639333004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae019dc10 con 0x7f5ae0107060 2024-07-24T22:01:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.293+0000 7f5ade7fc640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000d6d0 tx=0x7f5ad000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.294+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0017070 con 0x7f5ae0107060 2024-07-24T22:01:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.294+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad0009820 con 0x7f5ae0107060 2024-07-24T22:01:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.295+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0004b90 con 0x7f5ae0107060 2024-07-24T22:01:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.295+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae019df00 con 0x7f5ae0107060 2024-07-24T22:01:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.295+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae006a470 con 0x7f5ae0107060 2024-07-24T22:01:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.296+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae0102900 con 0x7f5ae0107060 2024-07-24T22:01:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.299+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5ad0004d30 con 0x7f5ae0107060 2024-07-24T22:01:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.300+0000 7f5ac7fff640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 0x7f5ac80b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.300+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8565+0+0 (secure 0 0 0) 0x7f5ad00e4c10 con 0x7f5ae0107060 2024-07-24T22:01:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.300+0000 7f5adeffd640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 0x7f5ac80b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.301+0000 7f5adeffd640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 0x7f5ac80b6120 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40096c0 tx=0x7f5ad4005d10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:09.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.305+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5ad0020020 con 0x7f5ae0107060 2024-07-24T22:01:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.560+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ae0198c90 con 0x7f5ae0107060 2024-07-24T22:01:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.561+0000 7f5ac7fff640 1 -- 172.21.15.60:0/2639333004 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7f5ad00abff0 con 0x7f5ae0107060 2024-07-24T22:01:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:01:09.563 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:01:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 msgr2=0x7f5ac80b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 0x7f5ac80b6120 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40096c0 tx=0x7f5ad4005d10 comp rx=0 tx=0).stop 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 msgr2=0x7f5ae01980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000d6d0 tx=0x7f5ad000db90 comp rx=0 tx=0).stop 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 shutdown_connections 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5ac80b3c60 0x7f5ac80b6120 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae01079e0 0x7f5ae019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ae0107060 0x7f5ae01980a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 --2- 172.21.15.60:0/2639333004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae0101650 0x7f5ae0197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 >> 172.21.15.60:0/2639333004 conn(0x7f5ae00fd0c0 msgr2=0x7f5ae00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 shutdown_connections 2024-07-24T22:01:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:09.563+0000 7f5ae542b640 1 -- 172.21.15.60:0/2639333004 wait complete. 2024-07-24T22:01:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:09 smithi060 ceph-mon[33000]: pgmap v829: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 248 KiB/s wr, 319 op/s 2024-07-24T22:01:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:09 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2639333004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:09.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:09 smithi086 ceph-mon[37128]: pgmap v829: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 248 KiB/s wr, 319 op/s 2024-07-24T22:01:09.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:09 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2639333004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:09 smithi123 ceph-mon[37143]: pgmap v829: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 248 KiB/s wr, 319 op/s 2024-07-24T22:01:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:09 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2639333004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:10.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:10 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:01:10.855 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 140.6 --await-for 45.0' 2024-07-24T22:01:10.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 140.6 --await-for 45.0 2024-07-24T22:01:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:10 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:01:11.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:10 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:01:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:11 smithi060 ceph-mon[33000]: pgmap v830: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 660 KiB/s wr, 150 op/s 2024-07-24T22:01:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:11 smithi060 ceph-mon[33000]: osdmap e106: 12 total, 12 up, 12 in 2024-07-24T22:01:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:01:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:01:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:01:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:11 smithi123 ceph-mon[37143]: pgmap v830: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 660 KiB/s wr, 150 op/s 2024-07-24T22:01:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:11 smithi123 ceph-mon[37143]: osdmap e106: 12 total, 12 up, 12 in 2024-07-24T22:01:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:01:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:01:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:01:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:11 smithi086 ceph-mon[37128]: pgmap v830: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 660 KiB/s wr, 150 op/s 2024-07-24T22:01:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:11 smithi086 ceph-mon[37128]: osdmap e106: 12 total, 12 up, 12 in 2024-07-24T22:01:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:01:12.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:01:12.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:01:13.904 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:13 smithi060 ceph-mon[33000]: pgmap v832: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 131 op/s 2024-07-24T22:01:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:13 smithi123 ceph-mon[37143]: pgmap v832: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 131 op/s 2024-07-24T22:01:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:13 smithi086 ceph-mon[37128]: pgmap v832: 97 pgs: 15 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 74 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 131 op/s 2024-07-24T22:01:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.075+0000 7f72820f6640 1 -- 172.21.15.60:0/2399253761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c1014c0 msgr2=0x7f727c1018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.075+0000 7f72820f6640 1 --2- 172.21.15.60:0/2399253761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c1014c0 0x7f727c1018a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7278009900 tx=0x7f727802a410 comp rx=0 tx=0).stop 2024-07-24T22:01:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.076+0000 7f72820f6640 1 -- 172.21.15.60:0/2399253761 shutdown_connections 2024-07-24T22:01:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.076+0000 7f72820f6640 1 --2- 172.21.15.60:0/2399253761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c107850 0x7f727c10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.076+0000 7f72820f6640 1 --2- 172.21.15.60:0/2399253761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c106ed0 0x7f727c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.076+0000 7f72820f6640 1 --2- 172.21.15.60:0/2399253761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c1014c0 0x7f727c1018a0 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7278009900 tx=0x7f727802a410 comp rx=0 tx=0).stop 2024-07-24T22:01:14.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.076+0000 7f72820f6640 1 -- 172.21.15.60:0/2399253761 >> 172.21.15.60:0/2399253761 conn(0x7f727c0fcf90 msgr2=0x7f727c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.081+0000 7f72820f6640 1 -- 172.21.15.60:0/2399253761 shutdown_connections 2024-07-24T22:01:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.081+0000 7f72820f6640 1 -- 172.21.15.60:0/2399253761 wait complete. 2024-07-24T22:01:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.082+0000 7f72820f6640 1 Processor -- start 2024-07-24T22:01:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 -- start start 2024-07-24T22:01:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c106ed0 0x7f727c10ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c107850 0x7f727c10ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112660 con 0x7f727c107850 2024-07-24T22:01:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1124e0 con 0x7f727c106ed0 2024-07-24T22:01:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.083+0000 7f72820f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1127e0 con 0x7f727c10f4c0 2024-07-24T22:01:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.084+0000 7f72818f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.084+0000 7f72810f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c106ed0 0x7f727c10ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.084+0000 7f72818f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46224/0 (socket says 172.21.15.60:46224) 2024-07-24T22:01:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.084+0000 7f72818f5640 1 -- 172.21.15.60:0/1662088819 learned_addr learned my addr 172.21.15.60:0/1662088819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.086+0000 7f72808f3640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c107850 0x7f727c10ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.087+0000 7f72818f5640 1 -- 172.21.15.60:0/1662088819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c106ed0 msgr2=0x7f727c10ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.087+0000 7f72818f5640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c106ed0 0x7f727c10ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.087+0000 7f72818f5640 1 -- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c107850 msgr2=0x7f727c10ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.087+0000 7f72818f5640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c107850 0x7f727c10ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.087+0000 7f72818f5640 1 -- 172.21.15.60:0/1662088819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c073c40 con 0x7f727c10f4c0 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.088+0000 7f72818f5640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f727000e970 tx=0x7f727000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.088+0000 7f726e7fc640 1 -- 172.21.15.60:0/1662088819 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000ccc0 con 0x7f727c10f4c0 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.088+0000 7f726e7fc640 1 -- 172.21.15.60:0/1662088819 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727000ce60 con 0x7f727c10f4c0 2024-07-24T22:01:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.089+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c073f30 con 0x7f727c10f4c0 2024-07-24T22:01:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.089+0000 7f726e7fc640 1 -- 172.21.15.60:0/1662088819 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270010430 con 0x7f727c10f4c0 2024-07-24T22:01:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.089+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c074420 con 0x7f727c10f4c0 2024-07-24T22:01:14.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.099+0000 7f7285709740 1 Processor -- start 2024-07-24T22:01:14.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.101+0000 7f7285709740 1 -- start start 2024-07-24T22:01:14.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.101+0000 7f726e7fc640 1 -- 172.21.15.60:0/1662088819 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7270004f50 con 0x7f727c10f4c0 2024-07-24T22:01:14.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccb970 0x5571beccbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5571bec416a0 con 0x5571beccced0 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571bec30ba0 con 0x5571becd77a0 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.102+0000 7f7285709740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5571bec34be0 con 0x5571beccb970 2024-07-24T22:01:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.103+0000 7f726e7fc640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 0x7f72680b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.103+0000 7f726e7fc640 1 -- 172.21.15.60:0/1662088819 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f72700e45e0 con 0x7f727c10f4c0 2024-07-24T22:01:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72818f5640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 0x7f72680b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49858/0 (socket says 172.21.15.60:49858) 2024-07-24T22:01:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 -- 172.21.15.60:0/4005188636 learned_addr learned my addr 172.21.15.60:0/4005188636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72818f5640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 0x7f72680b6370 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f72700ab0c0 tx=0x7f72700048b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 -- 172.21.15.60:0/4005188636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccb970 msgr2=0x5571beccbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccb970 0x5571beccbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 -- 172.21.15.60:0/4005188636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 msgr2=0x5571becd9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.104+0000 7f72808f3640 1 -- 172.21.15.60:0/4005188636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571bebc8d60 con 0x5571beccced0 2024-07-24T22:01:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.108+0000 7f72808f3640 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7274012840 tx=0x7f7274009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b4ac6c1a776fe9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.110+0000 7f72677fe640 1 -- 172.21.15.60:0/4005188636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274010a70 con 0x5571beccced0 2024-07-24T22:01:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.110+0000 7f72677fe640 1 -- 172.21.15.60:0/4005188636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274010c10 con 0x5571beccced0 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.111+0000 7f7285709740 1 -- 172.21.15.60:0/4005188636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 msgr2=0x5571beccb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.111+0000 7f7285709740 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7274012840 tx=0x7f7274009960 comp rx=0 tx=0).stop 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.112+0000 7f7285709740 1 -- 172.21.15.60:0/4005188636 shutdown_connections 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.112+0000 7f7285709740 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.112+0000 7f7285709740 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccb970 0x5571beccbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.112+0000 7f7285709740 1 --2- 172.21.15.60:0/4005188636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571beccb430 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.112+0000 7f7285709740 1 -- 172.21.15.60:0/4005188636 >> 172.21.15.60:0/4005188636 conn(0x5571bec95750 msgr2=0x5571becc9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:14.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.113+0000 7f7285709740 1 -- 172.21.15.60:0/4005188636 shutdown_connections 2024-07-24T22:01:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.115+0000 7f7285709740 1 -- 172.21.15.60:0/4005188636 wait complete. 2024-07-24T22:01:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.116+0000 7f7285709740 1 -- start start 2024-07-24T22:01:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.116+0000 7f7285709740 1 Processor -- start 2024-07-24T22:01:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becd4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f72808f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f72808f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49874/0 (socket says 172.21.15.60:49874) 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5571bea947a0 con 0x5571beccb970 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571bebd3020 con 0x5571becd77a0 2024-07-24T22:01:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5571becda6b0 con 0x5571beccced0 2024-07-24T22:01:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f7285709740 10 client.0 _ll_register_callbacks cb 0x5571be93e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.117+0000 7f72808f3640 1 -- 172.21.15.60:0/3552359538 learned_addr learned my addr 172.21.15.60:0/3552359538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.118+0000 7f72808f3640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72810f4640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becd4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72808f3640 1 -- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 msgr2=0x5571becd4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72808f3640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becd4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72808f3640 1 -- 172.21.15.60:0/3552359538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 msgr2=0x5571becd4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72808f3640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.119+0000 7f72808f3640 1 -- 172.21.15.60:0/3552359538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571bebd70d0 con 0x5571beccb970 2024-07-24T22:01:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.120+0000 7f72808f3640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f727403fb70 tx=0x7f727403fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46045aa777a73956 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.120+0000 7f7264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.124+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274044030 con 0x5571beccb970 2024-07-24T22:01:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.124+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7274010c10 con 0x5571beccb970 2024-07-24T22:01:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.124+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f727403f3b0 con 0x5571beccb970 2024-07-24T22:01:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.128+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7274056050 con 0x5571beccb970 2024-07-24T22:01:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.128+0000 7f7285709740 10 client.18510 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.128+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5571be9a0b90 con 0x5571beccb970 2024-07-24T22:01:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274010c10 con 0x5571beccb970 2024-07-24T22:01:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f727400c070 con 0x5571beccb970 2024-07-24T22:01:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 10 client.18510 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 10 client.18510 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 20 client.18510 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed09690 0x5571bed0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 4 client.18510 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571bec435f0 con 0x5571bed09690 2024-07-24T22:01:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.129+0000 7f72810f4640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed09690 0x5571bed0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.132+0000 7f72810f4640 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed09690 0x5571bed0ba80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c1c274a8851d39 server_cookie=43e2c540aa7e5260 in_seq=0 out_seq=0 2024-07-24T22:01:14.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.132+0000 7f7264ff9640 10 client.18510 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:14.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.133+0000 7f7264ff9640 1 -- 172.21.15.60:0/3552359538 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571bec435f0 con 0x5571bed09690 2024-07-24T22:01:14.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.133+0000 7f7264ff9640 10 client.18510 handle_command_reply: tid=0 2024-07-24T22:01:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.133+0000 7f7285709740 1 client.18510 shutdown 2024-07-24T22:01:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.134+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 msgr2=0x5571becd71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.134+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f727403fb70 tx=0x7f727403fbd0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 shutdown_connections 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed09690 0x5571bed0ba80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becd4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becd4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 --2- 172.21.15.60:0/3552359538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becd71d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 >> 172.21.15.60:0/3552359538 conn(0x5571bec95750 msgr2=0x5571beccedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.135+0000 7f7264ff9640 0 client.18510 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.136+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 shutdown_connections 2024-07-24T22:01:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.136+0000 7f7285709740 1 -- 172.21.15.60:0/3552359538 wait complete. 2024-07-24T22:01:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.138+0000 7f7285709740 20 client.18510 trim_cache size 0 max 16384 2024-07-24T22:01:14.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.178+0000 7f7285709740 1 Processor -- start 2024-07-24T22:01:14.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.178+0000 7f7285709740 1 -- start start 2024-07-24T22:01:14.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 0x5571beccb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571becc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571becd77a0 0x5571becc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f727c115120 con 0x5571beccced0 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1b0e60 con 0x5571beccb970 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f72810f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571becc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f72810f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571becc4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49890/0 (socket says 172.21.15.60:49890) 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f72810f4640 1 -- 172.21.15.60:0/2043182169 learned_addr learned my addr 172.21.15.60:0/2043182169 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f72808f3640 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 0x5571beccb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.179+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1aecd0 con 0x5571becd77a0 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571becd77a0 0x5571becc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 -- 172.21.15.60:0/2043182169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571becd77a0 msgr2=0x5571becc8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571becd77a0 0x5571becc8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 -- 172.21.15.60:0/2043182169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 msgr2=0x5571becc4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571becc4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 -- 172.21.15.60:0/2043182169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571becd50d0 con 0x5571beccb970 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.180+0000 7f72808f3640 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 0x5571beccb180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7274009880 tx=0x7f7274000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3d4be0458e2ede server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.181+0000 7f7266ffd640 1 -- 172.21.15.60:0/2043182169 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274044070 con 0x5571beccb970 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.181+0000 7f7266ffd640 1 -- 172.21.15.60:0/2043182169 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274045040 con 0x5571beccb970 2024-07-24T22:01:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.181+0000 7f7266ffd640 1 -- 172.21.15.60:0/2043182169 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727403f3b0 con 0x5571beccb970 2024-07-24T22:01:14.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.182+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 msgr2=0x5571beccb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:14.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.182+0000 7f7285709740 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 0x5571beccb180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7274009880 tx=0x7f7274000f40 comp rx=0 tx=0).stop 2024-07-24T22:01:14.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.185+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 shutdown_connections 2024-07-24T22:01:14.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.185+0000 7f7285709740 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571becd77a0 0x5571becc8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.185+0000 7f7285709740 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccced0 0x5571becc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.185+0000 7f7285709740 1 --2- 172.21.15.60:0/2043182169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571beccb970 0x5571beccb180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.185+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 >> 172.21.15.60:0/2043182169 conn(0x5571bec95750 msgr2=0x5571becd2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:14.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.187+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 shutdown_connections 2024-07-24T22:01:14.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.189+0000 7f7285709740 1 -- 172.21.15.60:0/2043182169 wait complete. 2024-07-24T22:01:14.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.189+0000 7f7285709740 1 -- start start 2024-07-24T22:01:14.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.189+0000 7f7285709740 1 Processor -- start 2024-07-24T22:01:14.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.190+0000 7f7285709740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becdfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becdb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1af320 con 0x5571beccb970 2024-07-24T22:01:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1b0790 con 0x5571becd77a0 2024-07-24T22:01:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1af9f0 con 0x5571beccced0 2024-07-24T22:01:14.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f72808f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f72808f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:60780/0 (socket says 172.21.15.60:60780) 2024-07-24T22:01:14.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f72808f3640 1 -- 172.21.15.60:0/1238582457 learned_addr learned my addr 172.21.15.60:0/1238582457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:14.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.191+0000 7f7285709740 10 client.0 _ll_register_callbacks cb 0x7f72740126a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.194+0000 7f72810f4640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becdb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becdfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 -- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 msgr2=0x5571becdb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becdb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 -- 172.21.15.60:0/1238582457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 msgr2=0x5571becdfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becdfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.195+0000 7f72808f3640 1 -- 172.21.15.60:0/1238582457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571becdbe70 con 0x5571becd77a0 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.196+0000 7f72808f3640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f7274009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b672d3a37fc38f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:14.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.196+0000 7f7264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.196+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274044070 con 0x5571becd77a0 2024-07-24T22:01:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.196+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7274045040 con 0x5571becd77a0 2024-07-24T22:01:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.196+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f7274004e40 con 0x5571becd77a0 2024-07-24T22:01:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.197+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f727400c530 con 0x5571becd77a0 2024-07-24T22:01:14.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.200+0000 7f7285709740 10 client.27896 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:14.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.200+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5571bed73f80 con 0x5571becd77a0 2024-07-24T22:01:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.200+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274045040 con 0x5571becd77a0 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f727400c040 con 0x5571becd77a0 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 10 client.27896 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 10 client.27896 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 20 client.27896 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed74690 0x5571bed76ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 4 client.27896 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.6, "await_for": 45.0}] 2024-07-24T22:01:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.202+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.6, "await_for": 45.0}) -- 0x5571bec3f090 con 0x5571bed74690 2024-07-24T22:01:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.205+0000 7f72810f4640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed74690 0x5571bed76ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.210+0000 7f72810f4640 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed74690 0x5571bed76ab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82537322e00c40b2 server_cookie=8d96e12e22d9fe13 in_seq=0 out_seq=0 2024-07-24T22:01:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:14.210+0000 7f7264ff9640 10 client.27896 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:15 smithi123 ceph-mon[37143]: pgmap v833: 97 pgs: 23 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-07-24T22:01:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:15 smithi086 ceph-mon[37128]: pgmap v833: 97 pgs: 23 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-07-24T22:01:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:15 smithi060 ceph-mon[33000]: pgmap v833: 97 pgs: 23 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-07-24T22:01:17.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:01:17.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:17 smithi060 ceph-mon[33000]: pgmap v834: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-07-24T22:01:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:17 smithi123 ceph-mon[37143]: pgmap v834: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-07-24T22:01:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:17 smithi086 ceph-mon[37128]: pgmap v834: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-07-24T22:01:18.194 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:19.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:19 smithi060 ceph-mon[33000]: pgmap v835: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-07-24T22:01:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:19 smithi123 ceph-mon[37143]: pgmap v835: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-07-24T22:01:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:19 smithi086 ceph-mon[37128]: pgmap v835: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-07-24T22:01:20.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.489+0000 7f39145b4640 1 -- 172.21.15.60:0/1747572818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c10f130 msgr2=0x7f390c115930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:20.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.489+0000 7f39145b4640 1 --2- 172.21.15.60:0/1747572818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c10f130 0x7f390c115930 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3904009ca0 tx=0x7f390402a8b0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 -- 172.21.15.60:0/1747572818 shutdown_connections 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 --2- 172.21.15.60:0/1747572818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c10f130 0x7f390c115930 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 --2- 172.21.15.60:0/1747572818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 --2- 172.21.15.60:0/1747572818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c108da0 0x7f390c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 -- 172.21.15.60:0/1747572818 >> 172.21.15.60:0/1747572818 conn(0x7f390c078420 msgr2=0x7f390c07a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:20.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 -- 172.21.15.60:0/1747572818 shutdown_connections 2024-07-24T22:01:20.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.491+0000 7f39145b4640 1 -- 172.21.15.60:0/1747572818 wait complete. 2024-07-24T22:01:20.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.492+0000 7f39145b4640 1 Processor -- start 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.492+0000 7f39145b4640 1 -- start start 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.492+0000 7f39145b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c108da0 0x7f390c0741c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f39145b4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f39145b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c10f130 0x7f390c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f39145b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f390c118e30 con 0x7f390c108da0 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f39145b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f390c118cb0 con 0x7f390c10f130 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f39145b4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f390c118fb0 con 0x7f390c10e7b0 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46938/0 (socket says 172.21.15.60:46938) 2024-07-24T22:01:20.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 -- 172.21.15.60:0/3491159125 learned_addr learned my addr 172.21.15.60:0/3491159125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:20.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3912350640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c108da0 0x7f390c0741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 -- 172.21.15.60:0/3491159125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c10f130 msgr2=0x7f390c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:01:20.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c10f130 0x7f390c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 -- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c108da0 msgr2=0x7f390c0741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:20.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c108da0 0x7f390c0741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.493+0000 7f3911b4f640 1 -- 172.21.15.60:0/3491159125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c06de30 con 0x7f390c10e7b0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.494+0000 7f3911b4f640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b4a0 tx=0x7f38fc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.494+0000 7f39037fe640 1 -- 172.21.15.60:0/3491159125 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc014070 con 0x7f390c10e7b0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.495+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c06e120 con 0x7f390c10e7b0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.495+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c073190 con 0x7f390c10e7b0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.495+0000 7f39037fe640 1 -- 172.21.15.60:0/3491159125 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38fc010ac0 con 0x7f390c10e7b0 2024-07-24T22:01:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.495+0000 7f39037fe640 1 -- 172.21.15.60:0/3491159125 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc017bb0 con 0x7f390c10e7b0 2024-07-24T22:01:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.501+0000 7f39037fe640 1 -- 172.21.15.60:0/3491159125 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f38fc017070 con 0x7f390c10e7b0 2024-07-24T22:01:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.502+0000 7f3916965740 1 Processor -- start 2024-07-24T22:01:20.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.502+0000 7f3916965740 1 -- start start 2024-07-24T22:01:20.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.502+0000 7f3916965740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8b680 0x557db2b8ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3916965740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3916965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3916965740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557db2b02960 con 0x557db2b8bfa0 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3916965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557db2af1e60 con 0x557db2b97f30 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3916965740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557db2af5ea0 con 0x557db2b8b680 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3912350640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3912350640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43146/0 (socket says 172.21.15.60:43146) 2024-07-24T22:01:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f3912350640 1 -- 172.21.15.60:0/3624240524 learned_addr learned my addr 172.21.15.60:0/3624240524 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:20.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.503+0000 7f39037fe640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 0x7f38f80b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912b51640 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8b680 0x557db2b8ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 -- 172.21.15.60:0/3624240524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8b680 msgr2=0x557db2b8ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8b680 0x557db2b8ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 -- 172.21.15.60:0/3624240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 msgr2=0x557db2b9a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3911b4f640 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b9a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 -- 172.21.15.60:0/3624240524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db2a7b2d0 con 0x557db2b8bfa0 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912b51640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 0x7f38f80b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f39037fe640 1 -- 172.21.15.60:0/3491159125 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f38fc0e34d0 con 0x7f390c10e7b0 2024-07-24T22:01:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3912350640 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f390800b820 tx=0x7f390800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600909bde29a7407 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.504+0000 7f3900ff9640 1 -- 172.21.15.60:0/3624240524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800d950 con 0x557db2b8bfa0 2024-07-24T22:01:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.505+0000 7f3900ff9640 1 -- 172.21.15.60:0/3624240524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3908019070 con 0x557db2b8bfa0 2024-07-24T22:01:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.505+0000 7f3912b51640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 0x7f38f80b6330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3904031040 tx=0x7f390402a8b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.505+0000 7f3900ff9640 1 -- 172.21.15.60:0/3624240524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39080123e0 con 0x557db2b8bfa0 2024-07-24T22:01:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 -- 172.21.15.60:0/3624240524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 msgr2=0x557db2b979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f390800b820 tx=0x7f390800bce0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 -- 172.21.15.60:0/3624240524 shutdown_connections 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b9a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8bfa0 0x557db2b979f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 --2- 172.21.15.60:0/3624240524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8b680 0x557db2b8ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 -- 172.21.15.60:0/3624240524 >> 172.21.15.60:0/3624240524 conn(0x557db2b894b0 msgr2=0x557db2b8a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 -- 172.21.15.60:0/3624240524 shutdown_connections 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.506+0000 7f3916965740 1 -- 172.21.15.60:0/3624240524 wait complete. 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.507+0000 7f3916965740 1 -- start start 2024-07-24T22:01:20.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.507+0000 7f3916965740 1 Processor -- start 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.508+0000 7f3916965740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.508+0000 7f3916965740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b94e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.508+0000 7f3916965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.508+0000 7f3916965740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557db2955700 con 0x557db2b8b680 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.508+0000 7f3916965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557db2a945a0 con 0x557db2b97f30 2024-07-24T22:01:20.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.509+0000 7f3916965740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557db2b9ac00 con 0x557db2b8bfa0 2024-07-24T22:01:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.509+0000 7f3916965740 10 client.0 _ll_register_callbacks cb 0x557db2ab4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43152/0 (socket says 172.21.15.60:43152) 2024-07-24T22:01:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 -- 172.21.15.60:0/1775742988 learned_addr learned my addr 172.21.15.60:0/1775742988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 -- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 msgr2=0x557db2b94e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b94e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 -- 172.21.15.60:0/1775742988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 msgr2=0x557db2b955d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b955d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.510+0000 7f3912350640 1 -- 172.21.15.60:0/1775742988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db2a87630 con 0x557db2b8b680 2024-07-24T22:01:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.511+0000 7f3912350640 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3908014d60 tx=0x7f39080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e43cb35d09b129 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:20.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908023070 con 0x557db2b8b680 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f390800dd60 con 0x557db2b8b680 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3908019070 con 0x557db2b8b680 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f390802d050 con 0x557db2b8b680 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.513+0000 7f3916965740 10 client.18528 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.514+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557db2852650 con 0x557db2b8b680 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3908019070 con 0x557db2b8b680 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3908013070 con 0x557db2b8b680 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 10 client.18528 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 20 client.18528 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2ba78f0 0x557db2ba9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 4 client.18528 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.515+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557db2b048b0 con 0x557db2ba78f0 2024-07-24T22:01:20.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.520+0000 7f3911b4f640 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2ba78f0 0x557db2ba9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:20.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.530+0000 7f3911b4f640 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2ba78f0 0x557db2ba9ce0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6cd529888d9f25 server_cookie=ec9cc5c31e87b835 in_seq=0 out_seq=0 2024-07-24T22:01:20.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:20.531+0000 7f38f5ffb640 10 client.18528 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.553+0000 7f38f5ffb640 1 -- 172.21.15.60:0/1775742988 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557db2b048b0 con 0x557db2ba78f0 2024-07-24T22:01:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.553+0000 7f38f5ffb640 10 client.18528 handle_command_reply: tid=0 2024-07-24T22:01:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.553+0000 7f3916965740 1 client.18528 shutdown 2024-07-24T22:01:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 msgr2=0x557db2b979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3908014d60 tx=0x7f39080046c0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 shutdown_connections 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2ba78f0 0x557db2ba9ce0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b97f30 0x557db2b955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f38f5ffb640 0 client.18528 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.554+0000 7f3916965740 1 --2- 172.21.15.60:0/1775742988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.555+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 >> 172.21.15.60:0/1775742988 conn(0x557db2b894b0 msgr2=0x557db2b8f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.555+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 shutdown_connections 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.555+0000 7f3916965740 1 -- 172.21.15.60:0/1775742988 wait complete. 2024-07-24T22:01:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.555+0000 7f3916965740 20 client.18528 trim_cache size 0 max 16384 2024-07-24T22:01:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.590+0000 7f3916965740 1 Processor -- start 2024-07-24T22:01:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 -- start start 2024-07-24T22:01:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b97f30 0x557db2b93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00113e0 con 0x557db2b97f30 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000ebf0 con 0x557db2b8b680 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.591+0000 7f3916965740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000ca60 con 0x557db2b8bfa0 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3911b4f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34572/0 (socket says 172.21.15.60:34572) 2024-07-24T22:01:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 -- 172.21.15.60:0/679667958 learned_addr learned my addr 172.21.15.60:0/679667958 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912b51640 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b97f30 0x557db2b93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 -- 172.21.15.60:0/679667958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 msgr2=0x557db2b936a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b936a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 -- 172.21.15.60:0/679667958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b97f30 msgr2=0x557db2b93be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b97f30 0x557db2b93be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.592+0000 7f3912350640 1 -- 172.21.15.60:0/679667958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db2b8bdd0 con 0x557db2b8b680 2024-07-24T22:01:21.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.593+0000 7f3912350640 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3908004810 tx=0x7f3908014640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2b1bc753afdbf6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.593+0000 7f38f7fff640 1 -- 172.21.15.60:0/679667958 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908023070 con 0x557db2b8b680 2024-07-24T22:01:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.593+0000 7f38f7fff640 1 -- 172.21.15.60:0/679667958 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3908019070 con 0x557db2b8b680 2024-07-24T22:01:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.593+0000 7f38f7fff640 1 -- 172.21.15.60:0/679667958 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800ddc0 con 0x557db2b8b680 2024-07-24T22:01:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 -- 172.21.15.60:0/679667958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 msgr2=0x557db2b979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3908004810 tx=0x7f3908014640 comp rx=0 tx=0).stop 2024-07-24T22:01:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 -- 172.21.15.60:0/679667958 shutdown_connections 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2b97f30 0x557db2b93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b8bfa0 0x557db2b936a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 --2- 172.21.15.60:0/679667958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b979d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 -- 172.21.15.60:0/679667958 >> 172.21.15.60:0/679667958 conn(0x557db2b894b0 msgr2=0x557db2b8a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 -- 172.21.15.60:0/679667958 shutdown_connections 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.594+0000 7f3916965740 1 -- 172.21.15.60:0/679667958 wait complete. 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 -- start start 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 Processor -- start 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b97f30 0x557db2b9bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000d0b0 con 0x557db2ba78f0 2024-07-24T22:01:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000e520 con 0x557db2b8b680 2024-07-24T22:01:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000d780 con 0x557db2b97f30 2024-07-24T22:01:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.595+0000 7f3916965740 10 client.0 _ll_register_callbacks cb 0x7f390800b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.596+0000 7f3912b51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.596+0000 7f3912b51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43168/0 (socket says 172.21.15.60:43168) 2024-07-24T22:01:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.596+0000 7f3912b51640 1 -- 172.21.15.60:0/2412513945 learned_addr learned my addr 172.21.15.60:0/2412513945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.597+0000 7f3912b51640 1 -- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b97f30 msgr2=0x557db2b9bc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.597+0000 7f3912b51640 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b97f30 0x557db2b9bc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.597+0000 7f3912b51640 1 -- 172.21.15.60:0/2412513945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 msgr2=0x557db2b9b6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.597+0000 7f3912b51640 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b9b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.597+0000 7f3912b51640 1 -- 172.21.15.60:0/2412513945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db2c10ad0 con 0x557db2ba78f0 2024-07-24T22:01:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.599+0000 7f3912b51640 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3904030c20 tx=0x7f39040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f83ff96a807d18 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.599+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.600+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904004450 con 0x557db2ba78f0 2024-07-24T22:01:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.600+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f39040045f0 con 0x557db2ba78f0 2024-07-24T22:01:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.600+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3908019070 con 0x557db2ba78f0 2024-07-24T22:01:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.601+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f39040047f0 con 0x557db2ba78f0 2024-07-24T22:01:21.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.605+0000 7f3916965740 10 client.18534 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:21.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.605+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557db2c11e10 con 0x557db2ba78f0 2024-07-24T22:01:21.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3908019070 con 0x557db2ba78f0 2024-07-24T22:01:21.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3904047420 con 0x557db2ba78f0 2024-07-24T22:01:21.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f3916965740 10 client.18534 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:21.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f3916965740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:21.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f3916965740 20 client.18534 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:21.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.606+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2c12650 0x557db2c14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.607+0000 7f3916965740 4 client.18534 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:01:21.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.607+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557db2b00350 con 0x557db2c12650 2024-07-24T22:01:21.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.607+0000 7f3912350640 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2c12650 0x557db2c14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.620+0000 7f3912350640 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2c12650 0x557db2c14a70 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c72c73cb549cc3 server_cookie=2ae2f8f18b003999 in_seq=0 out_seq=0 2024-07-24T22:01:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.620+0000 7f38f5ffb640 10 client.18534 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.714+0000 7f38f5ffb640 1 -- 172.21.15.60:0/2412513945 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557db2b00350 con 0x557db2c12650 2024-07-24T22:01:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.714+0000 7f38f5ffb640 10 client.18534 handle_command_reply: tid=0 2024-07-24T22:01:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.714+0000 7f3916965740 1 client.18534 shutdown 2024-07-24T22:01:21.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:21 smithi060 ceph-mon[33000]: pgmap v836: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.714+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 msgr2=0x557db2c10390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.714+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3904030c20 tx=0x7f39040095c0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 shutdown_connections 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557db2c12650 0x557db2c14a70 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557db2ba78f0 0x557db2c10390 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557db2b97f30 0x557db2b9bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 --2- 172.21.15.60:0/2412513945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557db2b8b680 0x557db2b9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 >> 172.21.15.60:0/2412513945 conn(0x557db2b894b0 msgr2=0x557db2b935d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f38f5ffb640 0 client.18534 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 shutdown_connections 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 1 -- 172.21.15.60:0/2412513945 wait complete. 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.715+0000 7f3916965740 20 client.18534 trim_cache size 0 max 16384 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:01:21.718 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 msgr2=0x7f38f80b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 0x7f38f80b6330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3904031040 tx=0x7f390402a8b0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 msgr2=0x7f390c074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b4a0 tx=0x7f38fc00b960 comp rx=0 tx=0).stop 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 shutdown_connections 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f38f80b3e70 0x7f38f80b6330 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f390c10f130 0x7f390c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.716+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c10e7b0 0x7f390c074700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.717+0000 7f39145b4640 1 --2- 172.21.15.60:0/3491159125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f390c108da0 0x7f390c0741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.717+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 >> 172.21.15.60:0/3491159125 conn(0x7f390c078420 msgr2=0x7f390c07a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.717+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 shutdown_connections 2024-07-24T22:01:21.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:21.717+0000 7f39145b4640 1 -- 172.21.15.60:0/3491159125 wait complete. 2024-07-24T22:01:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 -- 172.21.15.60:0/1530559202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be011ba90 msgr2=0x7f1be011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/1530559202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be011ba90 0x7f1be011de80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009be0 tx=0x7f1bd802a6f0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 -- 172.21.15.60:0/1530559202 shutdown_connections 2024-07-24T22:01:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/1530559202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be011ba90 0x7f1be011de80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/1530559202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be0109f20 0x7f1be010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/1530559202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be0110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.904+0000 7f1be4ab8640 1 -- 172.21.15.60:0/1530559202 >> 172.21.15.60:0/1530559202 conn(0x7f1be006d760 msgr2=0x7f1be006db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.908+0000 7f1be4ab8640 1 -- 172.21.15.60:0/1530559202 shutdown_connections 2024-07-24T22:01:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.910+0000 7f1be4ab8640 1 -- 172.21.15.60:0/1530559202 wait complete. 2024-07-24T22:01:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.910+0000 7f1be4ab8640 1 Processor -- start 2024-07-24T22:01:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 -- start start 2024-07-24T22:01:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be011af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 0x7f1be011b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be011ba90 0x7f1be0114350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01214e0 con 0x7f1be011ba90 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0121360 con 0x7f1be0109b40 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.915+0000 7f1be4ab8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0121660 con 0x7f1be0109f20 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.916+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be011af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.916+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be011af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34596/0 (socket says 172.21.15.60:34596) 2024-07-24T22:01:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.916+0000 7f1bde59c640 1 -- 172.21.15.60:0/2454388611 learned_addr learned my addr 172.21.15.60:0/2454388611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.916+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 0x7f1be011b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.919+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be011ba90 0x7f1be0114350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 -- 172.21.15.60:0/2454388611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 msgr2=0x7f1be011af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be011af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 -- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be011ba90 msgr2=0x7f1be0114350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be011ba90 0x7f1be0114350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 -- 172.21.15.60:0/2454388611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be0114c10 con 0x7f1be0109f20 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.920+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 0x7f1be011b480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e790 tx=0x7f1bc800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.921+0000 7f1bd77fe640 1 -- 172.21.15.60:0/2454388611 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800eea0 con 0x7f1be0109f20 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.921+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be0114f00 con 0x7f1be0109f20 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.921+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be0119f90 con 0x7f1be0109f20 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.921+0000 7f1bd77fe640 1 -- 172.21.15.60:0/2454388611 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc800ccc0 con 0x7f1be0109f20 2024-07-24T22:01:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.921+0000 7f1bd77fe640 1 -- 172.21.15.60:0/2454388611 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80187a0 con 0x7f1be0109f20 2024-07-24T22:01:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.929+0000 7f1bd77fe640 1 -- 172.21.15.60:0/2454388611 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f1bc8018940 con 0x7f1be0109f20 2024-07-24T22:01:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.931+0000 7f1be780b740 1 Processor -- start 2024-07-24T22:01:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.931+0000 7f1bd77fe640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 0x7f1bcc0b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.931+0000 7f1bd77fe640 1 -- 172.21.15.60:0/2454388611 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f1bc80e5260 con 0x7f1be0109f20 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 -- start start 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 0x7f1bcc0b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f511e0 0x561ce3f5cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561ce3ec6280 con 0x561ce3f511e0 2024-07-24T22:01:21.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ce3eb5780 con 0x561ce3f508c0 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1be780b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ce3eb97c0 con 0x561ce3f5d170 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.932+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:34602/0 (socket says 172.21.15.60:34602) 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 -- 172.21.15.60:0/141975193 learned_addr learned my addr 172.21.15.60:0/141975193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bded9d640 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 -- 172.21.15.60:0/141975193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 msgr2=0x561ce3f5f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 -- 172.21.15.60:0/141975193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f511e0 msgr2=0x561ce3f5cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f511e0 0x561ce3f5cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 -- 172.21.15.60:0/141975193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce3c19790 con 0x561ce3f508c0 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.933+0000 7f1bde59c640 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009b10 tx=0x7f1bd000e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5a0e7ca57288f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.934+0000 7f1bd4ff9640 1 -- 172.21.15.60:0/141975193 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0017070 con 0x561ce3f508c0 2024-07-24T22:01:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.934+0000 7f1bd4ff9640 1 -- 172.21.15.60:0/141975193 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd000ebe0 con 0x561ce3f508c0 2024-07-24T22:01:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.934+0000 7f1bd4ff9640 1 -- 172.21.15.60:0/141975193 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0012630 con 0x561ce3f508c0 2024-07-24T22:01:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.937+0000 7f1be780b740 1 -- 172.21.15.60:0/141975193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 msgr2=0x561ce3f50ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.937+0000 7f1be780b740 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009b10 tx=0x7f1bd000e1d0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.938+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 0x7f1bcc0b6330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009b30 tx=0x7f1bd8005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.942+0000 7f1be780b740 1 -- 172.21.15.60:0/141975193 shutdown_connections 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.942+0000 7f1be780b740 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.942+0000 7f1be780b740 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f511e0 0x561ce3f5cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.942+0000 7f1be780b740 1 --2- 172.21.15.60:0/141975193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f508c0 0x561ce3f50ca0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.942+0000 7f1be780b740 1 -- 172.21.15.60:0/141975193 >> 172.21.15.60:0/141975193 conn(0x561ce3f4e6f0 msgr2=0x561ce3f4f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.943+0000 7f1be780b740 1 -- 172.21.15.60:0/141975193 shutdown_connections 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.943+0000 7f1be780b740 1 -- 172.21.15.60:0/141975193 wait complete. 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.943+0000 7f1be780b740 1 -- start start 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.944+0000 7f1be780b740 1 Processor -- start 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.944+0000 7f1be780b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.944+0000 7f1be780b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1be780b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1be780b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561ce3defab0 con 0x561ce3f508c0 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1be780b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ce3e3d9e0 con 0x561ce3f511e0 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1be780b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ce3f5fe50 con 0x561ce3f5d170 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47004/0 (socket says 172.21.15.60:47004) 2024-07-24T22:01:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1bded9d640 1 -- 172.21.15.60:0/341591900 learned_addr learned my addr 172.21.15.60:0/341591900 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.945+0000 7f1be780b740 10 client.0 _ll_register_callbacks cb 0x561ce3bc5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 -- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 msgr2=0x561ce3f5a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 -- 172.21.15.60:0/341591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 msgr2=0x561ce3f59e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f59e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.946+0000 7f1bde59c640 1 -- 172.21.15.60:0/341591900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce3c92620 con 0x561ce3f508c0 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.947+0000 7f1bde59c640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f5c930 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0000c00 tx=0x7f1bd00136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba68e54c8fab3a03 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.947+0000 7f1bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.947+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0017050 con 0x561ce3f508c0 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.947+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ce3e64ce0 con 0x561ce3f508c0 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.948+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f1bd001c650 con 0x561ce3f508c0 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.948+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1bd001c850 con 0x561ce3f508c0 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.948+0000 7f1be780b740 10 client.18543 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.948+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561ce3f6b6f0 con 0x561ce3f508c0 2024-07-24T22:01:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.949+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561ce3e64ce0 con 0x561ce3f508c0 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd0026950 con 0x561ce3f508c0 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 10 client.18543 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 10 client.18543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 20 client.18543 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3f6cf60 0x561ce3f6f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 4 client.18543 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.950+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ce3ec81d0 con 0x561ce3f6cf60 2024-07-24T22:01:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.951+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3f6cf60 0x561ce3f6f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.953+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3f6cf60 0x561ce3f6f380 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def765288c4aae0d server_cookie=5f6938ab6c6b8c8a in_seq=0 out_seq=0 2024-07-24T22:01:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:21.953+0000 7f1bc5ffb640 10 client.18543 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:21 smithi123 ceph-mon[37143]: pgmap v836: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-24T22:01:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:21 smithi086 ceph-mon[37128]: pgmap v836: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.076+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/341591900 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ce3ec81d0 con 0x561ce3f6cf60 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.076+0000 7f1bc5ffb640 10 client.18543 handle_command_reply: tid=0 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.076+0000 7f1be780b740 1 client.18543 shutdown 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.076+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 msgr2=0x561ce3f5c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.076+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f5c930 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0000c00 tx=0x7f1bd00136a0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 shutdown_connections 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3f6cf60 0x561ce3f6f380 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f5a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f59e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 --2- 172.21.15.60:0/341591900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f5c930 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 >> 172.21.15.60:0/341591900 conn(0x561ce3f4e6f0 msgr2=0x561ce3f54290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1bc5ffb640 0 client.18543 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 shutdown_connections 2024-07-24T22:01:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 1 -- 172.21.15.60:0/341591900 wait complete. 2024-07-24T22:01:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.077+0000 7f1be780b740 20 client.18543 trim_cache size 0 max 16384 2024-07-24T22:01:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.114+0000 7f1be780b740 1 Processor -- start 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 -- start start 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f69000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0011270 con 0x561ce3f508c0 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc000ea80 con 0x561ce3f511e0 2024-07-24T22:01:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.115+0000 7f1be780b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc000c980 con 0x561ce3f5d170 2024-07-24T22:01:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f69000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bde59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f69000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43210/0 (socket says 172.21.15.60:43210) 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47014/0 (socket says 172.21.15.60:47014) 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bde59c640 1 -- 172.21.15.60:0/2416094420 learned_addr learned my addr 172.21.15.60:0/2416094420 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.117+0000 7f1bddd9b640 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.118+0000 7f1bded9d640 1 -- 172.21.15.60:0/2416094420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 msgr2=0x561ce3f5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.118+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f5a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.118+0000 7f1bded9d640 1 -- 172.21.15.60:0/2416094420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 msgr2=0x561ce3f69000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.118+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f69000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.118+0000 7f1bded9d640 1 -- 172.21.15.60:0/2416094420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce3f5f9e0 con 0x561ce3f5d170 2024-07-24T22:01:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.119+0000 7f1bded9d640 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802ace0 tx=0x7f1bd803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f611ccd93cfc51ea server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.124+0000 7f1bc7fff640 1 -- 172.21.15.60:0/2416094420 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8046030 con 0x561ce3f5d170 2024-07-24T22:01:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.124+0000 7f1bc7fff640 1 -- 172.21.15.60:0/2416094420 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd8051070 con 0x561ce3f5d170 2024-07-24T22:01:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.124+0000 7f1bc7fff640 1 -- 172.21.15.60:0/2416094420 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8036040 con 0x561ce3f5d170 2024-07-24T22:01:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.125+0000 7f1be780b740 1 -- 172.21.15.60:0/2416094420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 msgr2=0x561ce3f4fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.125+0000 7f1be780b740 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802ace0 tx=0x7f1bd803a000 comp rx=0 tx=0).stop 2024-07-24T22:01:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.126+0000 7f1be780b740 1 -- 172.21.15.60:0/2416094420 shutdown_connections 2024-07-24T22:01:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.126+0000 7f1be780b740 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f5d170 0x561ce3f4fae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.126+0000 7f1be780b740 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f511e0 0x561ce3f5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.126+0000 7f1be780b740 1 --2- 172.21.15.60:0/2416094420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f69000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.126+0000 7f1be780b740 1 -- 172.21.15.60:0/2416094420 >> 172.21.15.60:0/2416094420 conn(0x561ce3f4e6f0 msgr2=0x561ce3f4f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.131+0000 7f1be780b740 1 -- 172.21.15.60:0/2416094420 shutdown_connections 2024-07-24T22:01:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.131+0000 7f1be780b740 1 -- 172.21.15.60:0/2416094420 wait complete. 2024-07-24T22:01:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.131+0000 7f1be780b740 1 -- start start 2024-07-24T22:01:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.132+0000 7f1be780b740 1 Processor -- start 2024-07-24T22:01:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.132+0000 7f1be780b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f65180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.132+0000 7f1be780b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 0x561ce3f656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1be780b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 0x561ce3f60130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1be780b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc000cf40 con 0x561ce3f508c0 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1bddd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 0x561ce3f656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1bddd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 0x561ce3f656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47030/0 (socket says 172.21.15.60:47030) 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1bddd9b640 1 -- 172.21.15.60:0/357580273 learned_addr learned my addr 172.21.15.60:0/357580273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1bded9d640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 0x561ce3f60130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc000e3b0 con 0x561ce3f5d170 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc000d610 con 0x561ce3f511e0 2024-07-24T22:01:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.133+0000 7f1be780b740 10 client.0 _ll_register_callbacks cb 0x7f1bd0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 -- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 msgr2=0x561ce3f656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 0x561ce3f656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 -- 172.21.15.60:0/357580273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 msgr2=0x561ce3f65180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f65180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce3f60670 con 0x561ce3f5d170 2024-07-24T22:01:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bded9d640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 0x561ce3f60130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8030910 tx=0x7f1bd8004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d790670d34a0c53 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.134+0000 7f1bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.137+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8059070 con 0x561ce3f5d170 2024-07-24T22:01:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.137+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd8051070 con 0x561ce3f5d170 2024-07-24T22:01:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.137+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f1bd8004040 con 0x561ce3f5d170 2024-07-24T22:01:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.137+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1bd8065050 con 0x561ce3f5d170 2024-07-24T22:01:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.140+0000 7f1be780b740 10 client.27911 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.140+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561ce3fd6340 con 0x561ce3f5d170 2024-07-24T22:01:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.142+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd8047040 con 0x561ce3f5d170 2024-07-24T22:01:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.142+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd805eaf0 con 0x561ce3f5d170 2024-07-24T22:01:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.142+0000 7f1be780b740 10 client.27911 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.142+0000 7f1be780b740 10 client.27911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.142+0000 7f1be780b740 20 client.27911 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.143+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3fd6a40 0x561ce3fd8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.143+0000 7f1be780b740 4 client.27911 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:01:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.143+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1bd8051070 con 0x561ce3fd6a40 2024-07-24T22:01:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.151+0000 7f1bde59c640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3fd6a40 0x561ce3fd8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.163+0000 7f1bde59c640 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3fd6a40 0x561ce3fd8e60 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666220e7cf35aaae server_cookie=ee63f0d30d235ca8 in_seq=0 out_seq=0 2024-07-24T22:01:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.165+0000 7f1bc5ffb640 10 client.27911 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:22.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7264ff9640 1 -- 172.21.15.60:0/1238582457 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5571bec3f090 con 0x5571bed74690 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7264ff9640 10 client.27896 handle_command_reply: tid=0 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 client.27896 shutdown 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 msgr2=0x5571becdb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f7274009960 comp rx=0 tx=0).stop 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 shutdown_connections 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5571bed74690 0x5571bed76ab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571becd77a0 0x5571becdb830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571beccced0 0x5571becdb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 --2- 172.21.15.60:0/1238582457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5571beccb970 0x5571becdfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7264ff9640 0 client.27896 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.288+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 >> 172.21.15.60:0/1238582457 conn(0x5571bec95750 msgr2=0x5571becc8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.289+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 shutdown_connections 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.289+0000 7f7285709740 1 -- 172.21.15.60:0/1238582457 wait complete. 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.289+0000 7f7285709740 20 client.27896 trim_cache size 0 max 16384 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:22.292 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 46, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "538e6ea8": { 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 46, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.6, 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:01:22.293 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:22.294 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 msgr2=0x7f72680b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 0x7f72680b6370 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f72700ab0c0 tx=0x7f72700048b0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 msgr2=0x7f727c073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f727000e970 tx=0x7f727000ed70 comp rx=0 tx=0).stop 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 shutdown_connections 2024-07-24T22:01:22.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f72680b3eb0 0x7f72680b6370 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c10f4c0 0x7f727c073700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f727c107850 0x7f727c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 --2- 172.21.15.60:0/1662088819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c106ed0 0x7f727c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.290+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 >> 172.21.15.60:0/1662088819 conn(0x7f727c0fcf90 msgr2=0x7f727c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.291+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 shutdown_connections 2024-07-24T22:01:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:22.291+0000 7f72820f6640 1 -- 172.21.15.60:0/1662088819 wait complete. 2024-07-24T22:01:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.354+0000 7f1bc5ffb640 1 -- 172.21.15.60:0/357580273 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1bd8051070 con 0x561ce3fd6a40 2024-07-24T22:01:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1bc5ffb640 10 client.27911 handle_command_reply: tid=0 2024-07-24T22:01:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 client.27911 shutdown 2024-07-24T22:01:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 msgr2=0x561ce3f60130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 0x561ce3f60130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8030910 tx=0x7f1bd8004950 comp rx=0 tx=0).stop 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 shutdown_connections 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x561ce3fd6a40 0x561ce3fd8e60 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ce3f5d170 0x561ce3f60130 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ce3f511e0 0x561ce3f656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 --2- 172.21.15.60:0/357580273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ce3f508c0 0x561ce3f65180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 >> 172.21.15.60:0/357580273 conn(0x561ce3f4e6f0 msgr2=0x561ce3f54220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.355+0000 7f1bc5ffb640 0 client.27911 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.358+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 shutdown_connections 2024-07-24T22:01:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.358+0000 7f1be780b740 1 -- 172.21.15.60:0/357580273 wait complete. 2024-07-24T22:01:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.358+0000 7f1be780b740 20 client.27911 trim_cache size 0 max 16384 2024-07-24T22:01:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:01:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:01:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:01:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:01:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 msgr2=0x7f1bcc0b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 0x7f1bcc0b6330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009b30 tx=0x7f1bd8005ec0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 msgr2=0x7f1be011b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 0x7f1be011b480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e790 tx=0x7f1bc800ec50 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 shutdown_connections 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f1bcc0b3e70 0x7f1bcc0b6330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be011ba90 0x7f1be0114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1be0109f20 0x7f1be011b480 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 --2- 172.21.15.60:0/2454388611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0109b40 0x7f1be011af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.359+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 >> 172.21.15.60:0/2454388611 conn(0x7f1be006d760 msgr2=0x7f1be0072570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.360+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 shutdown_connections 2024-07-24T22:01:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:22.360+0000 7f1be4ab8640 1 -- 172.21.15.60:0/2454388611 wait complete. 2024-07-24T22:01:22.514 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:01:22.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:01:23.206 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '538e6ea8', quiesce took 16.4 seconds. Will release after: 4.200000000000003 2024-07-24T22:01:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:23.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:23 smithi060 ceph-mon[33000]: pgmap v837: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 679 KiB/s wr, 147 op/s 2024-07-24T22:01:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:23 smithi123 ceph-mon[37143]: pgmap v837: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 679 KiB/s wr, 147 op/s 2024-07-24T22:01:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:23 smithi086 ceph-mon[37128]: pgmap v837: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 679 KiB/s wr, 147 op/s 2024-07-24T22:01:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.471+0000 7facba5cc640 1 -- 172.21.15.60:0/3070342595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 msgr2=0x7facb410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.471+0000 7facba5cc640 1 --2- 172.21.15.60:0/3070342595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb410a360 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7faca4009900 tx=0x7faca402a440 comp rx=0 tx=0).stop 2024-07-24T22:01:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 -- 172.21.15.60:0/3070342595 shutdown_connections 2024-07-24T22:01:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 --2- 172.21.15.60:0/3070342595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 0x7facb411de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 --2- 172.21.15.60:0/3070342595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb410a360 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 --2- 172.21.15.60:0/3070342595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 -- 172.21.15.60:0/3070342595 >> 172.21.15.60:0/3070342595 conn(0x7facb406c4f0 msgr2=0x7facb406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 -- 172.21.15.60:0/3070342595 shutdown_connections 2024-07-24T22:01:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.474+0000 7facba5cc640 1 -- 172.21.15.60:0/3070342595 wait complete. 2024-07-24T22:01:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 Processor -- start 2024-07-24T22:01:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 -- start start 2024-07-24T22:01:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4118de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb4119320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 0x7facb41120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7facb4121560 con 0x7facb4109f20 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facb41213e0 con 0x7facb4109b40 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.475+0000 7facba5cc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facb41216e0 con 0x7facb411ba90 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.477+0000 7facb8dc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb4119320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.477+0000 7facb8dc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb4119320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43222/0 (socket says 172.21.15.60:43222) 2024-07-24T22:01:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.477+0000 7facb8dc9640 1 -- 172.21.15.60:0/2551331252 learned_addr learned my addr 172.21.15.60:0/2551331252 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.477+0000 7facb95ca640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4118de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.477+0000 7facb9dcb640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 0x7facb41120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.478+0000 7facb95ca640 1 -- 172.21.15.60:0/2551331252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 msgr2=0x7facb41120b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.478+0000 7facb95ca640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 0x7facb41120b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.478+0000 7facb95ca640 1 -- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 msgr2=0x7facb4119320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.478+0000 7facb95ca640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb4119320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.478+0000 7facb95ca640 1 -- 172.21.15.60:0/2551331252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb4112970 con 0x7facb4109b40 2024-07-24T22:01:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facb95ca640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4118de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7facb0009fd0 tx=0x7facb000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0015020 con 0x7facb4109b40 2024-07-24T22:01:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb4112c60 con 0x7facb4109b40 2024-07-24T22:01:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb4117c20 con 0x7facb4109b40 2024-07-24T22:01:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb0011ce0 con 0x7facb4109b40 2024-07-24T22:01:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.480+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0010de0 con 0x7facb4109b40 2024-07-24T22:01:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.481+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac980052c0 con 0x7facb4109b40 2024-07-24T22:01:25.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.484+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7facb0018020 con 0x7facb4109b40 2024-07-24T22:01:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.485+0000 7facaa7fc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 0x7faca00b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.486+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7facb00e41f0 con 0x7facb4109b40 2024-07-24T22:01:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.487+0000 7facb8dc9640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 0x7faca00b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:25.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.487+0000 7facb8dc9640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 0x7faca00b63f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faca40096c0 tx=0x7faca40040d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:25.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.491+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7facb001f020 con 0x7facb4109b40 2024-07-24T22:01:25.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.934+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac980055b0 con 0x7facb4109b40 2024-07-24T22:01:25.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:25 smithi060 ceph-mon[33000]: pgmap v838: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 367 KiB/s wr, 108 op/s 2024-07-24T22:01:25.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.936+0000 7facaa7fc640 1 -- 172.21.15.60:0/2551331252 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7facb00ab550 con 0x7facb4109b40 2024-07-24T22:01:25.938 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:01:25.938 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:01:25.939 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:01:25.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.939+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 msgr2=0x7faca00b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:25.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.940+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 0x7faca00b63f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faca40096c0 tx=0x7faca40040d0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.940+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 msgr2=0x7facb4118de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:25.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.940+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4118de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7facb0009fd0 tx=0x7facb000b870 comp rx=0 tx=0).stop 2024-07-24T22:01:25.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.940+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 shutdown_connections 2024-07-24T22:01:25.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.941+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7faca00b3f30 0x7faca00b63f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.941+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facb411ba90 0x7facb41120b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.941+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4109f20 0x7facb4119320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.941+0000 7facba5cc640 1 --2- 172.21.15.60:0/2551331252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facb4109b40 0x7facb4118de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:25.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.941+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 >> 172.21.15.60:0/2551331252 conn(0x7facb406c4f0 msgr2=0x7facb411bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.945+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 shutdown_connections 2024-07-24T22:01:25.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:25.947+0000 7facba5cc640 1 -- 172.21.15.60:0/2551331252 wait complete. 2024-07-24T22:01:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:25 smithi123 ceph-mon[37143]: pgmap v838: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 367 KiB/s wr, 108 op/s 2024-07-24T22:01:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:25 smithi086 ceph-mon[37128]: pgmap v838: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 367 KiB/s wr, 108 op/s 2024-07-24T22:01:26.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:26 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2551331252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:26.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:01:26.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:26 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2551331252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:27.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:26 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2551331252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:27.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:01:27.481 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:27.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:27 smithi060 ceph-mon[33000]: pgmap v839: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 350 KiB/s wr, 80 op/s 2024-07-24T22:01:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:27 smithi123 ceph-mon[37143]: pgmap v839: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 350 KiB/s wr, 80 op/s 2024-07-24T22:01:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:27 smithi086 ceph-mon[37128]: pgmap v839: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 350 KiB/s wr, 80 op/s 2024-07-24T22:01:28.893 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:29.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:29 smithi060 ceph-mon[33000]: pgmap v840: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 71 KiB/s wr, 44 op/s 2024-07-24T22:01:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:29 smithi123 ceph-mon[37143]: pgmap v840: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 71 KiB/s wr, 44 op/s 2024-07-24T22:01:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:29 smithi086 ceph-mon[37128]: pgmap v840: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 71 KiB/s wr, 44 op/s 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 -- 172.21.15.60:0/2142446068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf0107970 msgr2=0x7f4cf0112eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/2142446068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf0107970 0x7f4cf0112eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009860 tx=0x7f4ce002a380 comp rx=0 tx=0).stop 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 -- 172.21.15.60:0/2142446068 shutdown_connections 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/2142446068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cf01133f0 0x7f4cf01157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/2142446068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf0107970 0x7f4cf0112eb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.376+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/2142446068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107050 0x7f4cf0107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.377+0000 7f4cf83bf640 1 -- 172.21.15.60:0/2142446068 >> 172.21.15.60:0/2142446068 conn(0x7f4cf00778d0 msgr2=0x7f4cf0079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.377+0000 7f4cf83bf640 1 -- 172.21.15.60:0/2142446068 shutdown_connections 2024-07-24T22:01:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.377+0000 7f4cf83bf640 1 -- 172.21.15.60:0/2142446068 wait complete. 2024-07-24T22:01:30.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 Processor -- start 2024-07-24T22:01:30.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 -- start start 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cf0107050 0x7f4cf019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf01133f0 0x7f4cf01a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0118d90 con 0x7f4cf0107050 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0118c10 con 0x7f4cf01133f0 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.378+0000 7f4cf83bf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0118f10 con 0x7f4cf0107970 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.379+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.379+0000 7f4cf695c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf01133f0 0x7f4cf01a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.379+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49766/0 (socket says 172.21.15.60:49766) 2024-07-24T22:01:30.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.379+0000 7f4cf595a640 1 -- 172.21.15.60:0/4215489219 learned_addr learned my addr 172.21.15.60:0/4215489219 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:30.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 -- 172.21.15.60:0/4215489219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf01133f0 msgr2=0x7f4cf01a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf01133f0 0x7f4cf01a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 -- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cf0107050 msgr2=0x7f4cf019f0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:01:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cf0107050 0x7f4cf019f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 -- 172.21.15.60:0/4215489219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf01a5170 con 0x7f4cf0107970 2024-07-24T22:01:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.381+0000 7f4cf595a640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00057c0 tx=0x7f4ce0030d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.383+0000 7f4ce77fe640 1 -- 172.21.15.60:0/4215489219 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0037050 con 0x7f4cf0107970 2024-07-24T22:01:30.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.383+0000 7f4ce77fe640 1 -- 172.21.15.60:0/4215489219 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0038040 con 0x7f4cf0107970 2024-07-24T22:01:30.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.383+0000 7f4ce77fe640 1 -- 172.21.15.60:0/4215489219 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00053c0 con 0x7f4cf0107970 2024-07-24T22:01:30.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.383+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf01a5400 con 0x7f4cf0107970 2024-07-24T22:01:30.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.383+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01a57d0 con 0x7f4cf0107970 2024-07-24T22:01:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.387+0000 7f4cfa770740 1 Processor -- start 2024-07-24T22:01:30.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.387+0000 7f4cfa770740 1 -- start start 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x564552331ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564552331ff0 0x56455233da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5645522a89b0 con 0x56455233df80 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564552297eb0 con 0x564552331ff0 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.388+0000 7f4cfa770740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56455229bef0 con 0x5645523316d0 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.390+0000 7f4ce77fe640 1 -- 172.21.15.60:0/4215489219 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4ce0045020 con 0x7f4cf0107970 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.390+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.390+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36658/0 (socket says 172.21.15.60:36658) 2024-07-24T22:01:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.390+0000 7f4cf595a640 1 -- 172.21.15.60:0/3262358339 learned_addr learned my addr 172.21.15.60:0/3262358339 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf615b640 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564552331ff0 0x56455233da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf695c640 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x564552331ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 -- 172.21.15.60:0/3262358339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 msgr2=0x564552331ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x564552331ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 -- 172.21.15.60:0/3262358339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564552331ff0 msgr2=0x56455233da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564552331ff0 0x56455233da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 -- 172.21.15.60:0/3262358339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564552221320 con 0x56455233df80 2024-07-24T22:01:30.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.391+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0101680 tx=0x7f4ce0107f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b530762c5ec8d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.392+0000 7f4ce4de4640 1 -- 172.21.15.60:0/3262358339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce010c840 con 0x56455233df80 2024-07-24T22:01:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.392+0000 7f4ce4de4640 1 -- 172.21.15.60:0/3262358339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0111030 con 0x56455233df80 2024-07-24T22:01:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.392+0000 7f4ce77fe640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 0x7f4cdc0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4ce4de4640 1 -- 172.21.15.60:0/3262358339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0106e40 con 0x56455233df80 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 -- 172.21.15.60:0/3262358339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 msgr2=0x564552340370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0101680 tx=0x7f4ce0107f30 comp rx=0 tx=0).stop 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 -- 172.21.15.60:0/3262358339 shutdown_connections 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x564552340370 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564552331ff0 0x56455233da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3262358339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x564552331ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 -- 172.21.15.60:0/3262358339 >> 172.21.15.60:0/3262358339 conn(0x56455232f500 msgr2=0x5645523306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 -- 172.21.15.60:0/3262358339 shutdown_connections 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cf695c640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 0x7f4cdc0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4cfa770740 1 -- 172.21.15.60:0/3262358339 wait complete. 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.394+0000 7f4cfa770740 1 -- start start 2024-07-24T22:01:30.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.394+0000 7f4cfa770740 1 Processor -- start 2024-07-24T22:01:30.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.393+0000 7f4ce77fe640 1 -- 172.21.15.60:0/4215489219 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f4ce0103fa0 con 0x7f4cf0107970 2024-07-24T22:01:30.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.394+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564552331ff0 0x56455233ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x56455233b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5645520fb700 con 0x56455233df80 2024-07-24T22:01:30.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56455223a5f0 con 0x5645523316d0 2024-07-24T22:01:30.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645523409c0 con 0x564552331ff0 2024-07-24T22:01:30.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54846/0 (socket says 172.21.15.60:54846) 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 -- 172.21.15.60:0/1216573497 learned_addr learned my addr 172.21.15.60:0/1216573497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cfa770740 10 client.0 _ll_register_callbacks cb 0x56455225a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf595a640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564552331ff0 0x56455233ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 -- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564552331ff0 msgr2=0x56455233ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564552331ff0 0x56455233ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 -- 172.21.15.60:0/1216573497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 msgr2=0x56455233b620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x56455233b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 -- 172.21.15.60:0/1216573497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455222d680 con 0x5645523316d0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cf615b640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011690 tx=0x7f4cec011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e16ddc3e619d45c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.395+0000 7f4cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:30.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec01f060 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce0111030 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f4cec00fac0 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4cec011e90 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cfa770740 10 client.27926 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.396+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564551ff8650 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0111030 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4cec022410 con 0x5645523316d0 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 10 client.27926 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 10 client.27926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 20 client.27926 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56455234dc40 0x564552350080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 4 client.27926 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:30.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.397+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645522aa900 con 0x56455234dc40 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.398+0000 7f4cf595a640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56455234dc40 0x564552350080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.399+0000 7f4cd9ffb640 10 client.27926 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.398+0000 7f4cf595a640 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56455234dc40 0x564552350080 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa55f7b83a9f7f19 server_cookie=1ff2a7cc68028909 in_seq=0 out_seq=0 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.399+0000 7f4cf695c640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 0x7f4cdc0b6410 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80099a0 tx=0x7f4ce8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.399+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/1216573497 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645522aa900 con 0x56455234dc40 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.399+0000 7f4cd9ffb640 10 client.27926 handle_command_reply: tid=0 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 client.27926 shutdown 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 msgr2=0x56455233da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011690 tx=0x7f4cec011b50 comp rx=0 tx=0).stop 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 shutdown_connections 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x56455234dc40 0x564552350080 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56455233df80 0x56455233b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564552331ff0 0x56455233ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 --2- 172.21.15.60:0/1216573497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 >> 172.21.15.60:0/1216573497 conn(0x56455232f500 msgr2=0x5645523350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cd9ffb640 0 client.27926 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 shutdown_connections 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.400+0000 7f4cfa770740 1 -- 172.21.15.60:0/1216573497 wait complete. 2024-07-24T22:01:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.401+0000 7f4cfa770740 20 client.27926 trim_cache size 0 max 16384 2024-07-24T22:01:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.424+0000 7f4cfa770740 1 Processor -- start 2024-07-24T22:01:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.424+0000 7f4cfa770740 1 -- start start 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552335820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b7140 con 0x564552331ff0 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b4950 con 0x56455233df80 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cfa770740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b27c0 con 0x5645523316d0 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552335820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cf595a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552335820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36668/0 (socket says 172.21.15.60:36668) 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cf595a640 1 -- 172.21.15.60:0/4048835691 learned_addr learned my addr 172.21.15.60:0/4048835691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.425+0000 7f4cf695c640 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 -- 172.21.15.60:0/4048835691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 msgr2=0x5645523396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 -- 172.21.15.60:0/4048835691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 msgr2=0x564552335820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552335820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.429+0000 7f4cf615b640 1 -- 172.21.15.60:0/4048835691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645523409c0 con 0x5645523316d0 2024-07-24T22:01:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.430+0000 7f4cf615b640 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x56455233da20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cec022800 tx=0x7f4cec009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc43cdae1ddc8e3f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.431+0000 7f4cdbfff640 1 -- 172.21.15.60:0/4048835691 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec022610 con 0x5645523316d0 2024-07-24T22:01:30.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.431+0000 7f4cdbfff640 1 -- 172.21.15.60:0/4048835691 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cec011e90 con 0x5645523316d0 2024-07-24T22:01:30.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.431+0000 7f4cdbfff640 1 -- 172.21.15.60:0/4048835691 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0035f0 con 0x5645523316d0 2024-07-24T22:01:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.432+0000 7f4cfa770740 1 -- 172.21.15.60:0/4048835691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 msgr2=0x56455233da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.432+0000 7f4cfa770740 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x56455233da20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cec022800 tx=0x7f4cec009fd0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.432+0000 7f4cfa770740 1 -- 172.21.15.60:0/4048835691 shutdown_connections 2024-07-24T22:01:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.432+0000 7f4cfa770740 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.432+0000 7f4cfa770740 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552335820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.433+0000 7f4cfa770740 1 --2- 172.21.15.60:0/4048835691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x56455233da20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.433+0000 7f4cfa770740 1 -- 172.21.15.60:0/4048835691 >> 172.21.15.60:0/4048835691 conn(0x56455232f500 msgr2=0x5645523303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.433+0000 7f4cfa770740 1 -- 172.21.15.60:0/4048835691 shutdown_connections 2024-07-24T22:01:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.433+0000 7f4cfa770740 1 -- 172.21.15.60:0/4048835691 wait complete. 2024-07-24T22:01:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.434+0000 7f4cfa770740 1 -- start start 2024-07-24T22:01:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.434+0000 7f4cfa770740 1 Processor -- start 2024-07-24T22:01:30.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.435+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.435+0000 7f4cfa770740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552341930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cf615b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cf615b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49788/0 (socket says 172.21.15.60:49788) 2024-07-24T22:01:30.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cf615b640 1 -- 172.21.15.60:0/3175538116 learned_addr learned my addr 172.21.15.60:0/3175538116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:30.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b2e10 con 0x564552331ff0 2024-07-24T22:01:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b4280 con 0x56455233df80 2024-07-24T22:01:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01b34e0 con 0x5645523316d0 2024-07-24T22:01:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.436+0000 7f4cf695c640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cfa770740 10 client.0 _ll_register_callbacks cb 0x7f4cec009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552341930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf615b640 1 -- 172.21.15.60:0/3175538116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 msgr2=0x5645523b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf615b640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf615b640 1 -- 172.21.15.60:0/3175538116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 msgr2=0x564552341930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf615b640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552341930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.437+0000 7f4cf615b640 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645523b6fc0 con 0x5645523316d0 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.438+0000 7f4cf615b640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011b90 tx=0x7f4cec023d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fc4bc94bb9801e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.438+0000 7f4cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:30.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.438+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec005520 con 0x5645523316d0 2024-07-24T22:01:30.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.438+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cec011e90 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.438+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f4cec00f860 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.439+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4cec00fa60 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.439+0000 7f4cfa770740 10 client.27838 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.439+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5645523b8260 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.439+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cec011e90 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4cec003be0 con 0x5645523316d0 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 10 client.27838 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 10 client.27838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 20 client.27838 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5645523b8900 0x5645523bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 4 client.27838 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:01:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.440+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645522a63a0 con 0x5645523b8900 2024-07-24T22:01:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.441+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5645523b8900 0x5645523bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.441+0000 7f4cf595a640 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5645523b8900 0x5645523bad20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d6a351499ddd95e server_cookie=c91ccea8601754fd in_seq=0 out_seq=0 2024-07-24T22:01:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.441+0000 7f4cd9ffb640 10 client.27838 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.441+0000 7f4cd9ffb640 1 -- 172.21.15.60:0/3175538116 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645522a63a0 con 0x5645523b8900 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.441+0000 7f4cd9ffb640 10 client.27838 handle_command_reply: tid=0 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 client.27838 shutdown 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 msgr2=0x5645523413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011b90 tx=0x7f4cec023d10 comp rx=0 tx=0).stop 2024-07-24T22:01:30.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 shutdown_connections 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x5645523b8900 0x5645523bad20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cd9ffb640 0 client.27838 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56455233df80 0x5645523b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564552331ff0 0x564552341930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 --2- 172.21.15.60:0/3175538116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645523316d0 0x5645523413f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.442+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 >> 172.21.15.60:0/3175538116 conn(0x56455232f500 msgr2=0x5645523396b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.443+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 shutdown_connections 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.443+0000 7f4cfa770740 1 -- 172.21.15.60:0/3175538116 wait complete. 2024-07-24T22:01:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.443+0000 7f4cfa770740 20 client.27838 trim_cache size 0 max 16384 2024-07-24T22:01:30.446 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:01:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.444+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 msgr2=0x7f4cdc0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.444+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 0x7f4cdc0b6410 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80099a0 tx=0x7f4ce8009500 comp rx=0 tx=0).stop 2024-07-24T22:01:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.444+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 msgr2=0x7f4cf019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.444+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00057c0 tx=0x7f4ce0030d70 comp rx=0 tx=0).stop 2024-07-24T22:01:30.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 shutdown_connections 2024-07-24T22:01:30.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4cdc0b3f50 0x7f4cdc0b6410 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf01133f0 0x7f4cf01a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf0107970 0x7f4cf019f600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 --2- 172.21.15.60:0/4215489219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cf0107050 0x7f4cf019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:30.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.445+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 >> 172.21.15.60:0/4215489219 conn(0x7f4cf00778d0 msgr2=0x7f4cf0114270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:30.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.446+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 shutdown_connections 2024-07-24T22:01:30.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:30.446+0000 7f4cf83bf640 1 -- 172.21.15.60:0/4215489219 wait complete. 2024-07-24T22:01:31.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:01:31.950 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:31 smithi060 ceph-mon[33000]: pgmap v841: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-07-24T22:01:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:31 smithi123 ceph-mon[37143]: pgmap v841: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-07-24T22:01:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:31 smithi086 ceph-mon[37128]: pgmap v841: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1803733092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 msgr2=0x7fafdc10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1803733092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc10e1e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009be0 tx=0x7fafd402a6c0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1803733092 shutdown_connections 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1803733092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc10e1e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1803733092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 0x7fafdc1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1803733092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 0x7fafdc101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.365+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1803733092 >> 172.21.15.60:0/1803733092 conn(0x7fafdc0fd0c0 msgr2=0x7fafdc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.366+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1803733092 shutdown_connections 2024-07-24T22:01:32.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.366+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1803733092 wait complete. 2024-07-24T22:01:32.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.366+0000 7fafe3dcf640 1 Processor -- start 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.366+0000 7fafe3dcf640 1 -- start start 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 0x7fafdc10d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 0x7fafdc10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1127a0 con 0x7fafdc107060 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc112620 con 0x7fafdc1079e0 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe3dcf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc112920 con 0x7fafdc101650 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe236c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe236c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54870/0 (socket says 172.21.15.60:54870) 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe236c640 1 -- 172.21.15.60:0/1194063190 learned_addr learned my addr 172.21.15.60:0/1194063190 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe1b6b640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 0x7fafdc10d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:32.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.367+0000 7fafe136a640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 0x7fafdc10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 -- 172.21.15.60:0/1194063190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 msgr2=0x7fafdc10d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 0x7fafdc10d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 -- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 msgr2=0x7fafdc10da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 0x7fafdc10da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 -- 172.21.15.60:0/1194063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc1a52d0 con 0x7fafdc1079e0 2024-07-24T22:01:32.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe236c640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004af0 tx=0x7fafd4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4041020 con 0x7fafdc1079e0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1a5560 con 0x7fafdc1079e0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1abc80 con 0x7fafdc1079e0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd4003e10 con 0x7fafdc1079e0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.368+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4036040 con 0x7fafdc1079e0 2024-07-24T22:01:32.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.370+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc101a30 con 0x7fafdc1079e0 2024-07-24T22:01:32.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.374+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fafd4044020 con 0x7fafdc1079e0 2024-07-24T22:01:32.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.375+0000 7fafd2ffd640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 0x7fafc80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:32.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.375+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fafd4105080 con 0x7fafdc1079e0 2024-07-24T22:01:32.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.376+0000 7fafe1b6b640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 0x7fafc80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.376+0000 7fafe1b6b640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 0x7fafc80b6280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009bd0 tx=0x7fafd8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:32.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.383+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fafd404b030 con 0x7fafdc1079e0 2024-07-24T22:01:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.769+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafdc0febb0 con 0x7fafdc1079e0 2024-07-24T22:01:32.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.771+0000 7fafd2ffd640 1 -- 172.21.15.60:0/1194063190 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11922 (secure 0 0 0) 0x7fafd40cc2d0 con 0x7fafdc1079e0 2024-07-24T22:01:32.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:01:32.777 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"btime":"2024-07-24T21:56:56:482319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":24536,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T21:56:56.482304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26284":{"gid":26284,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1697293905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1697293905},{"type":"v1","addr":"172.21.15.123:6833","nonce":1697293905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:01:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-07-24T22:01:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.775+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 msgr2=0x7fafc80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.775+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 0x7fafc80b6280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009bd0 tx=0x7fafd8005f90 comp rx=0 tx=0).stop 2024-07-24T22:01:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.775+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 msgr2=0x7fafdc1a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.775+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004af0 tx=0x7fafd4003c20 comp rx=0 tx=0).stop 2024-07-24T22:01:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 shutdown_connections 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafc80b3dc0 0x7fafc80b6280 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc1079e0 0x7fafdc1a4b00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc107060 0x7fafdc10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 --2- 172.21.15.60:0/1194063190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc101650 0x7fafdc10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.776+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 >> 172.21.15.60:0/1194063190 conn(0x7fafdc0fd0c0 msgr2=0x7fafdc103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.777+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 shutdown_connections 2024-07-24T22:01:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:32.777+0000 7fafe3dcf640 1 -- 172.21.15.60:0/1194063190 wait complete. 2024-07-24T22:01:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:33 smithi123 ceph-mon[37143]: pgmap v842: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-07-24T22:01:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:33 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1194063190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:33 smithi086 ceph-mon[37128]: pgmap v842: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-07-24T22:01:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:33 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1194063190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:33 smithi060 ceph-mon[33000]: pgmap v842: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-07-24T22:01:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:33 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1194063190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:01:35.182 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 538e6ea8 --release --await' 2024-07-24T22:01:35.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 538e6ea8 --release --await 2024-07-24T22:01:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.515+0000 7f8c52740640 1 -- 172.21.15.60:0/2234915631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 msgr2=0x7f8c4c109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.515+0000 7f8c52740640 1 --2- 172.21.15.60:0/2234915631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c109b40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009860 tx=0x7f8c4802a380 comp rx=0 tx=0).stop 2024-07-24T22:01:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.522+0000 7f8c52740640 1 -- 172.21.15.60:0/2234915631 shutdown_connections 2024-07-24T22:01:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.522+0000 7f8c52740640 1 --2- 172.21.15.60:0/2234915631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c1196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.522+0000 7f8c52740640 1 --2- 172.21.15.60:0/2234915631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c10a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.522+0000 7f8c52740640 1 --2- 172.21.15.60:0/2234915631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c109b40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.522+0000 7f8c52740640 1 -- 172.21.15.60:0/2234915631 >> 172.21.15.60:0/2234915631 conn(0x7f8c4c06d570 msgr2=0x7f8c4c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.523+0000 7f8c52740640 1 -- 172.21.15.60:0/2234915631 shutdown_connections 2024-07-24T22:01:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.523+0000 7f8c52740640 1 -- 172.21.15.60:0/2234915631 wait complete. 2024-07-24T22:01:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.524+0000 7f8c52740640 1 Processor -- start 2024-07-24T22:01:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.524+0000 7f8c52740640 1 -- start start 2024-07-24T22:01:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c52740640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c52740640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c116680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c52740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c122c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c116680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c116680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49830/0 (socket says 172.21.15.60:49830) 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c51f3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c122c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c51f3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c122c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54890/0 (socket says 172.21.15.60:54890) 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.525+0000 7f8c51f3f640 1 -- 172.21.15.60:0/372921325 learned_addr learned my addr 172.21.15.60:0/372921325 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.526+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c10ea90 con 0x7f8c4c074770 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.526+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c10e910 con 0x7f8c4c1172b0 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.526+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c10ec10 con 0x7f8c4c10a080 2024-07-24T22:01:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.526+0000 7f8c5173e640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 -- 172.21.15.60:0/372921325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 msgr2=0x7f8c4c116680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c116680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 -- 172.21.15.60:0/372921325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 msgr2=0x7f8c4c122c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c122c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c123410 con 0x7f8c4c074770 2024-07-24T22:01:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.527+0000 7f8c5173e640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c116140 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802a970 tx=0x7f8c48030720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.528+0000 7f8c427fc640 1 -- 172.21.15.60:0/372921325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037050 con 0x7f8c4c074770 2024-07-24T22:01:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.528+0000 7f8c427fc640 1 -- 172.21.15.60:0/372921325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c48038040 con 0x7f8c4c074770 2024-07-24T22:01:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.528+0000 7f8c427fc640 1 -- 172.21.15.60:0/372921325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48003d00 con 0x7f8c4c074770 2024-07-24T22:01:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.529+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c123640 con 0x7f8c4c074770 2024-07-24T22:01:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.529+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c11c860 con 0x7f8c4c074770 2024-07-24T22:01:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.533+0000 7f8c567d1740 1 Processor -- start 2024-07-24T22:01:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.534+0000 7f8c427fc640 1 -- 172.21.15.60:0/372921325 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8c48045030 con 0x7f8c4c074770 2024-07-24T22:01:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.536+0000 7f8c427fc640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 0x7f8c380b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.536+0000 7f8c427fc640 1 -- 172.21.15.60:0/372921325 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f8c48103d40 con 0x7f8c4c074770 2024-07-24T22:01:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.536+0000 7f8c567d1740 1 -- start start 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d34e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d35350 0x55ef96d37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ef96c9e280 con 0x55ef96d35350 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef96c8d780 con 0x55ef96d28aa0 2024-07-24T22:01:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c567d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef96c917c0 con 0x55ef96d293c0 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c5173e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c5173e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54892/0 (socket says 172.21.15.60:54892) 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.537+0000 7f8c5173e640 1 -- 172.21.15.60:0/153732382 learned_addr learned my addr 172.21.15.60:0/153732382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d35350 0x55ef96d37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 0x7f8c380b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 -- 172.21.15.60:0/153732382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 msgr2=0x55ef96d34e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d34e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 -- 172.21.15.60:0/153732382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d35350 msgr2=0x55ef96d37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d35350 0x55ef96d37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 -- 172.21.15.60:0/153732382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef969f1790 con 0x55ef96d28aa0 2024-07-24T22:01:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.538+0000 7f8c5173e640 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035040 tx=0x7f8c48004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74fcef93bfb7bd07 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.539+0000 7f8c377fe640 1 -- 172.21.15.60:0/153732382 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48030910 con 0x55ef96d28aa0 2024-07-24T22:01:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.539+0000 7f8c377fe640 1 -- 172.21.15.60:0/153732382 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c480d3390 con 0x55ef96d28aa0 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c377fe640 1 -- 172.21.15.60:0/153732382 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480d3a50 con 0x55ef96d28aa0 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 -- 172.21.15.60:0/153732382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 msgr2=0x55ef96d28e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 0x7f8c380b6410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c44009660 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035040 tx=0x7f8c48004250 comp rx=0 tx=0).stop 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 -- 172.21.15.60:0/153732382 shutdown_connections 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d35350 0x55ef96d37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d34e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 --2- 172.21.15.60:0/153732382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d28e80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.540+0000 7f8c567d1740 1 -- 172.21.15.60:0/153732382 >> 172.21.15.60:0/153732382 conn(0x55ef96d268d0 msgr2=0x55ef96d27a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.541+0000 7f8c567d1740 1 -- 172.21.15.60:0/153732382 shutdown_connections 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.541+0000 7f8c567d1740 1 -- 172.21.15.60:0/153732382 wait complete. 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.541+0000 7f8c567d1740 1 -- start start 2024-07-24T22:01:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.541+0000 7f8c567d1740 1 Processor -- start 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d32480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ef96bc7ab0 con 0x55ef96d28aa0 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef96c159e0 con 0x55ef96d35350 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef96d37df0 con 0x55ef96d293c0 2024-07-24T22:01:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c567d1740 10 client.0 _ll_register_callbacks cb 0x55ef9699da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.542+0000 7f8c5173e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36734/0 (socket says 172.21.15.60:36734) 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 -- 172.21.15.60:0/595528580 learned_addr learned my addr 172.21.15.60:0/595528580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d32480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 -- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 msgr2=0x55ef96d31f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d31f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 -- 172.21.15.60:0/595528580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 msgr2=0x55ef96d32480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d32480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 -- 172.21.15.60:0/595528580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef96a6a620 con 0x55ef96d28aa0 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.543+0000 7f8c5173e640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8c480ca730 tx=0x7f8c48032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5eecc9e6a6818 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.544+0000 7f8c34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.544+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48030910 con 0x55ef96d28aa0 2024-07-24T22:01:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.544+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef96c3cce0 con 0x55ef96d28aa0 2024-07-24T22:01:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.544+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f8c480d37d0 con 0x55ef96d28aa0 2024-07-24T22:01:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.544+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8c480caa30 con 0x55ef96d28aa0 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c567d1740 10 client.18582 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef96d43510 con 0x55ef96d28aa0 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ef96c3cce0 con 0x55ef96d28aa0 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f8c480d1260 con 0x55ef96d28aa0 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c567d1740 10 client.18582 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.545+0000 7f8c567d1740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.546+0000 7f8c567d1740 20 client.18582 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.546+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96d44a20 0x55ef96d46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.546+0000 7f8c567d1740 4 client.18582 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.546+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef96ca01d0 con 0x55ef96d44a20 2024-07-24T22:01:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.546+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96d44a20 0x55ef96d46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.547+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96d44a20 0x55ef96d46e60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e98dd5e1179c70b server_cookie=9990fabf943ae2eb in_seq=0 out_seq=0 2024-07-24T22:01:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.547+0000 7f8c34ff9640 10 client.18582 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c34ff9640 1 -- 172.21.15.60:0/595528580 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef96ca01d0 con 0x55ef96d44a20 2024-07-24T22:01:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c34ff9640 10 client.18582 handle_command_reply: tid=0 2024-07-24T22:01:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 client.18582 shutdown 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 msgr2=0x55ef96d34a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8c480ca730 tx=0x7f8c48032f10 comp rx=0 tx=0).stop 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 shutdown_connections 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96d44a20 0x55ef96d46e60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d31f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 --2- 172.21.15.60:0/595528580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d34a60 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 >> 172.21.15.60:0/595528580 conn(0x55ef96d268d0 msgr2=0x55ef96d2c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.548+0000 7f8c34ff9640 0 client.18582 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.549+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 shutdown_connections 2024-07-24T22:01:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.549+0000 7f8c567d1740 1 -- 172.21.15.60:0/595528580 wait complete. 2024-07-24T22:01:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.549+0000 7f8c567d1740 20 client.18582 trim_cache size 0 max 16384 2024-07-24T22:01:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.575+0000 7f8c567d1740 1 Processor -- start 2024-07-24T22:01:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.576+0000 7f8c567d1740 1 -- start start 2024-07-24T22:01:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.576+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.576+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c567d1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c8040 con 0x55ef96d28aa0 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49860/0 (socket says 172.21.15.60:49860) 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c50f3d640 1 -- 172.21.15.60:0/120344943 learned_addr learned my addr 172.21.15.60:0/120344943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c567d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c5850 con 0x55ef96d35350 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c36c0 con 0x55ef96d293c0 2024-07-24T22:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 -- 172.21.15.60:0/120344943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 msgr2=0x55ef96d4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 -- 172.21.15.60:0/120344943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 msgr2=0x55ef96d2f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d2f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 -- 172.21.15.60:0/120344943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef96d297a0 con 0x55ef96d28aa0 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.577+0000 7f8c5173e640 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d4c730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8c480cabf0 tx=0x7f8c48032850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c3d43dc838871 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.578+0000 7f8c36ffd640 1 -- 172.21.15.60:0/120344943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480c9f90 con 0x55ef96d28aa0 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.578+0000 7f8c36ffd640 1 -- 172.21.15.60:0/120344943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c480e5070 con 0x55ef96d28aa0 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.578+0000 7f8c36ffd640 1 -- 172.21.15.60:0/120344943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480cb130 con 0x55ef96d28aa0 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 msgr2=0x55ef96d4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d4c730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8c480cabf0 tx=0x7f8c48032850 comp rx=0 tx=0).stop 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 shutdown_connections 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d35350 0x55ef96d2f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d293c0 0x55ef96d4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 --2- 172.21.15.60:0/120344943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d28aa0 0x55ef96d4c730 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 >> 172.21.15.60:0/120344943 conn(0x55ef96d268d0 msgr2=0x55ef96d27dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 shutdown_connections 2024-07-24T22:01:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- 172.21.15.60:0/120344943 wait complete. 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.579+0000 7f8c567d1740 1 -- start start 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 Processor -- start 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 0x55ef96da66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c3d10 con 0x55ef96d293c0 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c5180 con 0x55ef96d28aa0 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c567d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1c43e0 con 0x55ef96d35350 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.580+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36742/0 (socket says 172.21.15.60:36742) 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 -- 172.21.15.60:0/1876910778 learned_addr learned my addr 172.21.15.60:0/1876910778 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 0x55ef96da66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c5173e640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c567d1740 10 client.0 _ll_register_callbacks cb 0x7f8c44009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 -- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 msgr2=0x55ef96da66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 0x55ef96da66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 -- 172.21.15.60:0/1876910778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 msgr2=0x55ef96d398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 -- 172.21.15.60:0/1876910778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef96da6e00 con 0x55ef96d293c0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c51f3f640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 0x55ef96da66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c50f3d640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004720 tx=0x7f8c3c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc897d72daf88f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.581+0000 7f8c34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c021020 con 0x55ef96d293c0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f8c3c017c30 con 0x55ef96d293c0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c480e5070 con 0x55ef96d293c0 2024-07-24T22:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c02c050 con 0x55ef96d293c0 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c567d1740 10 client.18588 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef96dae2a0 con 0x55ef96d293c0 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.582+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c017e30 con 0x55ef96d293c0 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f8c3c01d500 con 0x55ef96d293c0 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 10 client.18588 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 20 client.18588 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96daeb00 0x55ef96db0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 4 client.18588 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:01:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8c480e5070 con 0x55ef96daeb00 2024-07-24T22:01:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.583+0000 7f8c5173e640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96daeb00 0x55ef96db0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.584+0000 7f8c5173e640 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96daeb00 0x55ef96db0f20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643448e0b869a16c server_cookie=69cdac00d479ae6a in_seq=0 out_seq=0 2024-07-24T22:01:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.584+0000 7f8c34ff9640 10 client.18588 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c34ff9640 1 -- 172.21.15.60:0/1876910778 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8c480e5070 con 0x55ef96daeb00 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c34ff9640 10 client.18588 handle_command_reply: tid=0 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 client.18588 shutdown 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 msgr2=0x55ef96d39e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004720 tx=0x7f8c3c009d40 comp rx=0 tx=0).stop 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 shutdown_connections 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55ef96daeb00 0x55ef96db0f20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c34ff9640 0 client.18588 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef96d35350 0x55ef96da66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef96d293c0 0x55ef96d39e80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 --2- 172.21.15.60:0/1876910778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef96d28aa0 0x55ef96d398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 >> 172.21.15.60:0/1876910778 conn(0x55ef96d268d0 msgr2=0x55ef96d25210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 shutdown_connections 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.695+0000 7f8c567d1740 1 -- 172.21.15.60:0/1876910778 wait complete. 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.696+0000 7f8c567d1740 20 client.18588 trim_cache size 0 max 16384 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:01:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:01:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 msgr2=0x7f8c380b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 0x7f8c380b6410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c44009660 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 msgr2=0x7f8c4c116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c116140 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802a970 tx=0x7f8c48030720 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 shutdown_connections 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8c380b3f50 0x7f8c380b6410 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c4c1172b0 0x7f8c4c122c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4c10a080 0x7f8c4c116680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 --2- 172.21.15.60:0/372921325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c074770 0x7f8c4c116140 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 >> 172.21.15.60:0/372921325 conn(0x7f8c4c06d570 msgr2=0x7f8c4c118710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 shutdown_connections 2024-07-24T22:01:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:35.697+0000 7f8c52740640 1 -- 172.21.15.60:0/372921325 wait complete. 2024-07-24T22:01:35.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:35 smithi060 ceph-mon[33000]: pgmap v843: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-24T22:01:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.774+0000 7f195bfe5640 1 -- 172.21.15.60:0/287398989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108ae0 msgr2=0x7f195410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.775+0000 7f195bfe5640 1 --2- 172.21.15.60:0/287398989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108ae0 0x7f195410f2e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f194c009ca0 tx=0x7f194c02a8f0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.775+0000 7f195bfe5640 1 -- 172.21.15.60:0/287398989 shutdown_connections 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.775+0000 7f195bfe5640 1 --2- 172.21.15.60:0/287398989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108ae0 0x7f195410f2e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.775+0000 7f195bfe5640 1 --2- 172.21.15.60:0/287398989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954108160 0x7f19541085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.775+0000 7f195bfe5640 1 --2- 172.21.15.60:0/287398989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954102750 0x7f1954102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.776+0000 7f195bfe5640 1 -- 172.21.15.60:0/287398989 >> 172.21.15.60:0/287398989 conn(0x7f19540fe200 msgr2=0x7f1954100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.776+0000 7f195bfe5640 1 -- 172.21.15.60:0/287398989 shutdown_connections 2024-07-24T22:01:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.776+0000 7f195bfe5640 1 -- 172.21.15.60:0/287398989 wait complete. 2024-07-24T22:01:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.777+0000 7f195bfe5640 1 Processor -- start 2024-07-24T22:01:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.777+0000 7f195bfe5640 1 -- start start 2024-07-24T22:01:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.778+0000 7f195bfe5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 0x7f19541967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.778+0000 7f195bfe5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 0x7f1954196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.778+0000 7f195bfe5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 0x7f195419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.778+0000 7f1959d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 0x7f19541967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.778+0000 7f1959d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 0x7f19541967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:54920/0 (socket says 172.21.15.60:54920) 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f1959580640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 0x7f1954196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f1959580640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 0x7f1954196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:49900/0 (socket says 172.21.15.60:49900) 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f195bfe5640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19541105b0 con 0x7f1954108ae0 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f195bfe5640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1954110430 con 0x7f1954102750 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f195bfe5640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1954110730 con 0x7f1954108160 2024-07-24T22:01:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f1959d81640 1 -- 172.21.15.60:0/2417677022 learned_addr learned my addr 172.21.15.60:0/2417677022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.779+0000 7f195a582640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 0x7f195419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 -- 172.21.15.60:0/2417677022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 msgr2=0x7f1954196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 0x7f1954196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 -- 172.21.15.60:0/2417677022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 msgr2=0x7f19541967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 0x7f19541967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195419c880 con 0x7f1954108ae0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.780+0000 7f195a582640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 0x7f195419c0b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f194c009bf0 tx=0x7f194c031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.781+0000 7f194affd640 1 -- 172.21.15.60:0/2417677022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c030cb0 con 0x7f1954108ae0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.781+0000 7f194affd640 1 -- 172.21.15.60:0/2417677022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194c02d070 con 0x7f1954108ae0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.781+0000 7f195bfe5640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195419cb10 con 0x7f1954108ae0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.781+0000 7f194affd640 1 -- 172.21.15.60:0/2417677022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c03a660 con 0x7f1954108ae0 2024-07-24T22:01:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.781+0000 7f195bfe5640 1 -- 172.21.15.60:0/2417677022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19541a3230 con 0x7f1954108ae0 2024-07-24T22:01:35.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.785+0000 7f195e396740 1 Processor -- start 2024-07-24T22:01:35.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.798+0000 7f194affd640 1 -- 172.21.15.60:0/2417677022 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f194c04d8e0 con 0x7f1954108ae0 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.800+0000 7f194affd640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 0x7f19400b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.801+0000 7f194affd640 1 -- 172.21.15.60:0/2417677022 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f194c10d890 con 0x7f1954108ae0 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.801+0000 7f195e396740 1 -- start start 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a5419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54dd20 0x56117a550110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56117a4b89b0 con 0x56117a541470 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56117a4a7eb0 con 0x56117a541ef0 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f195e396740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56117a4abef0 con 0x56117a54dd20 2024-07-24T22:01:35.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.802+0000 7f1959580640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 0x7f19400b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.803+0000 7f1959580640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.803+0000 7f195a582640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54dd20 0x56117a550110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.803+0000 7f1959580640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41710/0 (socket says 172.21.15.60:41710) 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.803+0000 7f1959580640 1 -- 172.21.15.60:0/526818001 learned_addr learned my addr 172.21.15.60:0/526818001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 0x7f19400b6410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1944014660 tx=0x7f1944014a60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 -- 172.21.15.60:0/526818001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54dd20 msgr2=0x56117a550110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54dd20 0x56117a550110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 -- 172.21.15.60:0/526818001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 msgr2=0x56117a5419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a5419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.804+0000 7f1959580640 1 -- 172.21.15.60:0/526818001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117a431320 con 0x56117a541ef0 2024-07-24T22:01:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.805+0000 7f1959580640 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1944010970 tx=0x7f194401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f21e37317309df2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.807+0000 7f195bfe5640 1 -- 172.21.15.60:0/526818001 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944014230 con 0x56117a541ef0 2024-07-24T22:01:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.807+0000 7f195bfe5640 1 -- 172.21.15.60:0/526818001 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19440054d0 con 0x56117a541ef0 2024-07-24T22:01:35.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 -- 172.21.15.60:0/526818001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 msgr2=0x56117a542310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1944010970 tx=0x7f194401e000 comp rx=0 tx=0).stop 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 -- 172.21.15.60:0/526818001 shutdown_connections 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54dd20 0x56117a550110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541ef0 0x56117a542310 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1944010970 tx=0x7f194401e000 comp rx=0 tx=0).stop 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 --2- 172.21.15.60:0/526818001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a5419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.808+0000 7f195e396740 1 -- 172.21.15.60:0/526818001 >> 172.21.15.60:0/526818001 conn(0x56117a53f7e0 msgr2=0x56117a540980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.809+0000 7f195e396740 1 -- 172.21.15.60:0/526818001 shutdown_connections 2024-07-24T22:01:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.809+0000 7f195e396740 1 -- 172.21.15.60:0/526818001 wait complete. 2024-07-24T22:01:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.809+0000 7f195e396740 1 -- start start 2024-07-24T22:01:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.810+0000 7f195e396740 1 Processor -- start 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a54ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56117a30b700 con 0x56117a54dd20 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56117a44a5f0 con 0x56117a541470 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56117a561bb0 con 0x56117a54b8c0 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.811+0000 7f195e396740 10 client.0 _ll_register_callbacks cb 0x56117a46a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53096/0 (socket says 172.21.15.60:53096) 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 -- 172.21.15.60:0/3351962453 learned_addr learned my addr 172.21.15.60:0/3351962453 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 -- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 msgr2=0x56117a54ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a54ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 -- 172.21.15.60:0/3351962453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 msgr2=0x56117a54b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.813+0000 7f195a582640 1 -- 172.21.15.60:0/3351962453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117a43d680 con 0x56117a54b8c0 2024-07-24T22:01:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.815+0000 7f195a582640 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f194c10b090 tx=0x7f194c0f71a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd66149b3c10fb0a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.815+0000 7f193d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c0d4000 con 0x56117a54b8c0 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56117a4ba900 con 0x56117a54b8c0 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f194c0e9da0 con 0x56117a54b8c0 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f194c0d4bf0 con 0x56117a54b8c0 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f195e396740 10 client.27865 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.817+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56117a208650 con 0x56117a54b8c0 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56117a4ba900 con 0x56117a54b8c0 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f194c0f4020 con 0x56117a54b8c0 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 10 client.27865 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 10 client.27865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 20 client.27865 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a56f3e0 0x56117a571880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 4 client.27865 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.819+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117a4b63a0 con 0x56117a56f3e0 2024-07-24T22:01:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.820+0000 7f1959d81640 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a56f3e0 0x56117a571880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.821+0000 7f1959d81640 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a56f3e0 0x56117a571880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2032ff2d67526e55 server_cookie=182347d107f30fd3 in_seq=0 out_seq=0 2024-07-24T22:01:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.822+0000 7f193d7fa640 10 client.27865 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:01:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.824+0000 7f193d7fa640 1 -- 172.21.15.60:0/3351962453 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56117a4b63a0 con 0x56117a56f3e0 2024-07-24T22:01:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.824+0000 7f193d7fa640 10 client.27865 handle_command_reply: tid=0 2024-07-24T22:01:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.824+0000 7f195e396740 1 client.27865 shutdown 2024-07-24T22:01:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.824+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 msgr2=0x56117a56af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.824+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f194c10b090 tx=0x7f194c0f71a0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 shutdown_connections 2024-07-24T22:01:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a56f3e0 0x56117a571880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a56af60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 --2- 172.21.15.60:0/3351962453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a54ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.825+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 >> 172.21.15.60:0/3351962453 conn(0x56117a53f7e0 msgr2=0x56117a544a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.826+0000 7f193d7fa640 0 client.27865 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.826+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 shutdown_connections 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.826+0000 7f195e396740 1 -- 172.21.15.60:0/3351962453 wait complete. 2024-07-24T22:01:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.826+0000 7f195e396740 20 client.27865 trim_cache size 0 max 16384 2024-07-24T22:01:35.853 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:35.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.853+0000 7f195e396740 1 Processor -- start 2024-07-24T22:01:35.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 -- start start 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a56ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a54ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19541ae940 con 0x56117a54dd20 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19541ac150 con 0x56117a541470 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195e396740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19541a9fc0 con 0x56117a54b8c0 2024-07-24T22:01:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f1959d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a56ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195a582640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195a582640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45776/0 (socket says 172.21.15.60:45776) 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.855+0000 7f195a582640 1 -- 172.21.15.60:0/2137780599 learned_addr learned my addr 172.21.15.60:0/2137780599 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 -- 172.21.15.60:0/2137780599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 msgr2=0x56117a54ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a54ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 -- 172.21.15.60:0/2137780599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 msgr2=0x56117a56ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a56ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 -- 172.21.15.60:0/2137780599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117a53aab0 con 0x56117a54dd20 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.856+0000 7f195a582640 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f194c0d4480 tx=0x7f194c0d3940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8dfa154606c084 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.857+0000 7f193f7fe640 1 -- 172.21.15.60:0/2137780599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c03f070 con 0x56117a54dd20 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.857+0000 7f193f7fe640 1 -- 172.21.15.60:0/2137780599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194c036070 con 0x56117a54dd20 2024-07-24T22:01:35.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.857+0000 7f193f7fe640 1 -- 172.21.15.60:0/2137780599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c0d8af0 con 0x56117a54dd20 2024-07-24T22:01:35.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.858+0000 7f195e396740 1 -- 172.21.15.60:0/2137780599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 msgr2=0x56117a54b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.859+0000 7f195e396740 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f194c0d4480 tx=0x7f194c0d3940 comp rx=0 tx=0).stop 2024-07-24T22:01:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.862+0000 7f195e396740 1 -- 172.21.15.60:0/2137780599 shutdown_connections 2024-07-24T22:01:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.862+0000 7f195e396740 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a54dd20 0x56117a54b350 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.862+0000 7f195e396740 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a54ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.862+0000 7f195e396740 1 --2- 172.21.15.60:0/2137780599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a541470 0x56117a56ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.862+0000 7f195e396740 1 -- 172.21.15.60:0/2137780599 >> 172.21.15.60:0/2137780599 conn(0x56117a53f7e0 msgr2=0x56117a545250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.864+0000 7f195e396740 1 -- 172.21.15.60:0/2137780599 shutdown_connections 2024-07-24T22:01:35.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.864+0000 7f195e396740 1 -- 172.21.15.60:0/2137780599 wait complete. 2024-07-24T22:01:35.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.864+0000 7f195e396740 1 -- start start 2024-07-24T22:01:35.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.865+0000 7f195e396740 1 Processor -- start 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a567110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a567690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aa610 con 0x56117a541470 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aba80 con 0x56117a54dd20 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aace0 con 0x56117a54b8c0 2024-07-24T22:01:35.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.866+0000 7f195e396740 10 client.0 _ll_register_callbacks cb 0x7f194c0d04a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.867+0000 7f195a582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.867+0000 7f1959580640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a567690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.867+0000 7f195a582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41722/0 (socket says 172.21.15.60:41722) 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.867+0000 7f195a582640 1 -- 172.21.15.60:0/1671763371 learned_addr learned my addr 172.21.15.60:0/1671763371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.868+0000 7f195a582640 1 -- 172.21.15.60:0/1671763371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 msgr2=0x56117a567690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.868+0000 7f195a582640 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a567690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.868+0000 7f195a582640 1 -- 172.21.15.60:0/1671763371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 msgr2=0x56117a567110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.868+0000 7f195a582640 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a567110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.868+0000 7f195a582640 1 -- 172.21.15.60:0/1671763371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117a5480d0 con 0x56117a54dd20 2024-07-24T22:01:35.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.870+0000 7f195a582640 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f194c0dd070 tx=0x7f194c004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c520fe44dda9e7fe server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:35.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.873+0000 7f193d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:35.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.873+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c03f070 con 0x56117a54dd20 2024-07-24T22:01:35.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.873+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f194c0d0070 con 0x56117a54dd20 2024-07-24T22:01:35.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.873+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f194c036070 con 0x56117a54dd20 2024-07-24T22:01:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.875+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f194c0d3e40 con 0x56117a54dd20 2024-07-24T22:01:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.876+0000 7f195e396740 10 client.27971 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.876+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56117a5480d0 con 0x56117a54dd20 2024-07-24T22:01:35.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.878+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f194c0d3a70 con 0x56117a54dd20 2024-07-24T22:01:35.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.879+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f194c0f4030 con 0x56117a54dd20 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 10 client.27971 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 10 client.27971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 20 client.27971 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a5da000 0x56117a5dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 4 client.27971 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:01:35.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.880+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f194c036070 con 0x56117a5da000 2024-07-24T22:01:35.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.883+0000 7f1959d81640 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a5da000 0x56117a5dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:35.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.886+0000 7f1959d81640 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a5da000 0x56117a5dc420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df4569280ee394b8 server_cookie=a3283b76c011884d in_seq=0 out_seq=0 2024-07-24T22:01:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.886+0000 7f193d7fa640 10 client.27971 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:01:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.886+0000 7f193d7fa640 1 -- 172.21.15.60:0/1671763371 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f194c036070 con 0x56117a5da000 2024-07-24T22:01:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.886+0000 7f193d7fa640 10 client.27971 handle_command_reply: tid=0 2024-07-24T22:01:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.886+0000 7f195e396740 1 client.27971 shutdown 2024-07-24T22:01:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.887+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 msgr2=0x56117a5478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.887+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f194c0dd070 tx=0x7f194c004d50 comp rx=0 tx=0).stop 2024-07-24T22:01:35.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.887+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 shutdown_connections 2024-07-24T22:01:35.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.887+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x56117a5da000 0x56117a5dc420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56117a54dd20 0x56117a5478a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f193d7fa640 0 client.27971 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:01:35.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56117a54b8c0 0x56117a567690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 --2- 172.21.15.60:0/1671763371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56117a541470 0x56117a567110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 >> 172.21.15.60:0/1671763371 conn(0x56117a53f7e0 msgr2=0x56117a54ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 shutdown_connections 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 1 -- 172.21.15.60:0/1671763371 wait complete. 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.888+0000 7f195e396740 20 client.27971 trim_cache size 0 max 16384 2024-07-24T22:01:35.891 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:01:35.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.890+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 msgr2=0x7f19400b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.890+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 0x7f19400b6410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1944014660 tx=0x7f1944014a60 comp rx=0 tx=0).stop 2024-07-24T22:01:35.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.890+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 msgr2=0x7f195419c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:35.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.890+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 0x7f195419c0b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f194c009bf0 tx=0x7f194c031f80 comp rx=0 tx=0).stop 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.890+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 shutdown_connections 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f19400b3f50 0x7f19400b6410 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1954108ae0 0x7f195419c0b0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954108160 0x7f1954196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 --2- 172.21.15.60:0/2417677022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1954102750 0x7f19541967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 >> 172.21.15.60:0/2417677022 conn(0x7f19540fe200 msgr2=0x7f19540fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 shutdown_connections 2024-07-24T22:01:35.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:35.891+0000 7f1948ff9640 1 -- 172.21.15.60:0/2417677022 wait complete. 2024-07-24T22:01:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:35 smithi123 ceph-mon[37143]: pgmap v843: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-24T22:01:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:35 smithi086 ceph-mon[37128]: pgmap v843: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-24T22:01:37.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:01:37.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:37 smithi060 ceph-mon[33000]: pgmap v844: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-07-24T22:01:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:37 smithi123 ceph-mon[37143]: pgmap v844: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-07-24T22:01:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:37 smithi086 ceph-mon[37128]: pgmap v844: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-07-24T22:01:38.478 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:39.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:39 smithi123 ceph-mon[37143]: pgmap v845: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-07-24T22:01:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:39 smithi086 ceph-mon[37128]: pgmap v845: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-07-24T22:01:40.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:39 smithi060 ceph-mon[33000]: pgmap v845: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-07-24T22:01:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 -- 172.21.15.60:0/3871448700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc40409a3e0 msgr2=0x7fc40409a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 --2- 172.21.15.60:0/3871448700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc40409a3e0 0x7fc40409a7c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a420 comp rx=0 tx=0).stop 2024-07-24T22:01:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 -- 172.21.15.60:0/3871448700 shutdown_connections 2024-07-24T22:01:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 --2- 172.21.15.60:0/3871448700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4040057f0 0x7fc404098fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 --2- 172.21.15.60:0/3871448700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc404004d10 0x7fc404005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 --2- 172.21.15.60:0/3871448700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc40409a3e0 0x7fc40409a7c0 secure :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a420 comp rx=0 tx=0).stop 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.293+0000 7fc410846640 1 -- 172.21.15.60:0/3871448700 >> 172.21.15.60:0/3871448700 conn(0x7fc40400a150 msgr2=0x7fc40400c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.294+0000 7fc410846640 1 -- 172.21.15.60:0/3871448700 shutdown_connections 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.294+0000 7fc410846640 1 -- 172.21.15.60:0/3871448700 wait complete. 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.295+0000 7fc410846640 1 Processor -- start 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.296+0000 7fc410846640 1 -- start start 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4040057f0 0x7fc4040a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4040a2440 0x7fc4040a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040abe80 con 0x7fc4040057f0 2024-07-24T22:01:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040abd00 con 0x7fc404004d10 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc410846640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040ac000 con 0x7fc4040a2440 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc40b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc40b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41738/0 (socket says 172.21.15.60:41738) 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.297+0000 7fc40b7fe640 1 -- 172.21.15.60:0/1603242628 learned_addr learned my addr 172.21.15.60:0/1603242628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40bfff640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4040a2440 0x7fc4040a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40b7fe640 1 -- 172.21.15.60:0/1603242628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4040a2440 msgr2=0x7fc4040a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40b7fe640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4040a2440 0x7fc4040a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40b7fe640 1 -- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4040057f0 msgr2=0x7fc4040a1f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40b7fe640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4040057f0 0x7fc4040a1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.298+0000 7fc40b7fe640 1 -- 172.21.15.60:0/1603242628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040a7940 con 0x7fc404004d10 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.300+0000 7fc40b7fe640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc40002aa10 tx=0x7fc400003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.301+0000 7fc408ff9640 1 -- 172.21.15.60:0/1603242628 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40003f030 con 0x7fc404004d10 2024-07-24T22:01:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.301+0000 7fc408ff9640 1 -- 172.21.15.60:0/1603242628 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400032bf0 con 0x7fc404004d10 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.301+0000 7fc408ff9640 1 -- 172.21.15.60:0/1603242628 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400035070 con 0x7fc404004d10 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.303+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4040a7bd0 con 0x7fc404004d10 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.303+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4040a8030 con 0x7fc404004d10 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 Processor -- start 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 -- start start 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1c520 0x555bcef27f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef284b0 0x555bcef2a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555bcee92dd0 con 0x555bcef1c520 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.307+0000 7fc413e59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555bcee822d0 con 0x555bcef1bc00 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc413e59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bcee86310 con 0x555bcef284b0 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41744/0 (socket says 172.21.15.60:41744) 2024-07-24T22:01:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 -- 172.21.15.60:0/2177144406 learned_addr learned my addr 172.21.15.60:0/2177144406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40affd640 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef284b0 0x555bcef2a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 -- 172.21.15.60:0/2177144406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef284b0 msgr2=0x555bcef2a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef284b0 0x555bcef2a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 -- 172.21.15.60:0/2177144406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1c520 msgr2=0x555bcef27f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1c520 0x555bcef27f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 -- 172.21.15.60:0/2177144406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcebe2b00 con 0x555bcef1bc00 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.308+0000 7fc40bfff640 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc40c02e590 tx=0x7fc40c0640c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b640e5d1e4e0ec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.309+0000 7fc408ff9640 1 -- 172.21.15.60:0/1603242628 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc400042030 con 0x7fc404004d10 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.309+0000 7fc3f5ffb640 1 -- 172.21.15.60:0/2177144406 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c082470 con 0x555bcef1bc00 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.309+0000 7fc3f5ffb640 1 -- 172.21.15.60:0/2177144406 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c082ac0 con 0x555bcef1bc00 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.309+0000 7fc3f5ffb640 1 -- 172.21.15.60:0/2177144406 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0817b0 con 0x555bcef1bc00 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.310+0000 7fc408ff9640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 0x7fc3fc0b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.310+0000 7fc408ff9640 1 -- 172.21.15.60:0/1603242628 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc4001048e0 con 0x7fc404004d10 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.310+0000 7fc413e59740 1 -- 172.21.15.60:0/2177144406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 msgr2=0x555bcef1bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.310+0000 7fc413e59740 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc40c02e590 tx=0x7fc40c0640c0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.310+0000 7fc413e59740 1 -- 172.21.15.60:0/2177144406 shutdown_connections 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc413e59740 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef284b0 0x555bcef2a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc413e59740 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1c520 0x555bcef27f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc413e59740 1 --2- 172.21.15.60:0/2177144406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef1bfe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc413e59740 1 -- 172.21.15.60:0/2177144406 >> 172.21.15.60:0/2177144406 conn(0x555bcef19a30 msgr2=0x555bcef1abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc413e59740 1 -- 172.21.15.60:0/2177144406 shutdown_connections 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.311+0000 7fc40b7fe640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 0x7fc3fc0b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.312+0000 7fc413e59740 1 -- 172.21.15.60:0/2177144406 wait complete. 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.312+0000 7fc413e59740 1 -- start start 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.312+0000 7fc40b7fe640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 0x7fc3fc0b6230 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x555bcef1a910 tx=0x7fc40002a420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.312+0000 7fc413e59740 1 Processor -- start 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc413e59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc413e59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 0x555bcef25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc413e59740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53148/0 (socket says 172.21.15.60:53148) 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc40affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 0x555bcef25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc40affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 0x555bcef25910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41750/0 (socket says 172.21.15.60:41750) 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.313+0000 7fc40bfff640 1 -- 172.21.15.60:0/3875544446 learned_addr learned my addr 172.21.15.60:0/3875544446 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555bcece5bb0 con 0x555bcef284b0 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555bcee24900 con 0x555bcef1c520 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bcef2b130 con 0x555bcef1bc00 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc413e59740 10 client.0 _ll_register_callbacks cb 0x555bcee2ec30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 -- 172.21.15.60:0/3875544446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 msgr2=0x555bcef25910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 0x555bcef25910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 -- 172.21.15.60:0/3875544446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 msgr2=0x555bcef25e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef25e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.314+0000 7fc40bfff640 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcec59960 con 0x555bcef1bc00 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.315+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc40c068320 tx=0x7fc40c064020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc0294e62b0330 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.315+0000 7fc3eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.315+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c095060 con 0x555bcef1bc00 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.315+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40c082ac0 con 0x555bcef1bc00 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.315+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc40c0816b0 con 0x555bcef1bc00 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.316+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc40c0818b0 con 0x555bcef1bc00 2024-07-24T22:01:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.316+0000 7fc413e59740 10 client.27871 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.316+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555bcef36c60 con 0x555bcef1bc00 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.316+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c06a040 con 0x555bcef1bc00 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.316+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc40c08d030 con 0x555bcef1bc00 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 10 client.27871 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 10 client.27871 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 20 client.27871 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcef3a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 4 client.27871 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc40c082ac0 con 0x555bcef37fb0 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.317+0000 7fc40affd640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcef3a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.320+0000 7fc40affd640 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcef3a3a0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea3cd08ad485477 server_cookie=dfc08b3034e6cbd in_seq=0 out_seq=0 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.320+0000 7fc3eaffd640 10 client.27871 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc3eaffd640 1 -- 172.21.15.60:0/3875544446 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc40c082ac0 con 0x555bcef37fb0 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc3eaffd640 10 client.27871 handle_command_reply: tid=0 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 client.27871 shutdown 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 msgr2=0x555bcef253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc40c068320 tx=0x7fc40c064020 comp rx=0 tx=0).stop 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 shutdown_connections 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcef3a3a0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc3eaffd640 0 client.27871 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef25e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.321+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1c520 0x555bcef25910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.322+0000 7fc413e59740 1 --2- 172.21.15.60:0/3875544446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1bc00 0x555bcef253d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.322+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 >> 172.21.15.60:0/3875544446 conn(0x555bcef19a30 msgr2=0x555bcef1f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.322+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 shutdown_connections 2024-07-24T22:01:40.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.322+0000 7fc413e59740 1 -- 172.21.15.60:0/3875544446 wait complete. 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.322+0000 7fc413e59740 20 client.27871 trim_cache size 0 max 16384 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.348+0000 7fc413e59740 1 Processor -- start 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 -- start start 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef284b0 0x555bcef3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc404152fe0 con 0x555bcef1bc00 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45822/0 (socket says 172.21.15.60:45822) 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 -- 172.21.15.60:0/3150190410 learned_addr learned my addr 172.21.15.60:0/3150190410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041507f0 con 0x555bcef284b0 2024-07-24T22:01:40.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc40414e660 con 0x555bcef1c520 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40affd640 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 -- 172.21.15.60:0/3150190410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 msgr2=0x555bcef27110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef27110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 -- 172.21.15.60:0/3150190410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef284b0 msgr2=0x555bcef3fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef284b0 0x555bcef3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.349+0000 7fc40bfff640 1 -- 172.21.15.60:0/3150190410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcef238f0 con 0x555bcef1bc00 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.350+0000 7fc40bfff640 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc40c066390 tx=0x7fc40c051200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baa4b4ca3c2894a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.350+0000 7fc3f57fa640 1 -- 172.21.15.60:0/3150190410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c082930 con 0x555bcef1bc00 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.350+0000 7fc3f57fa640 1 -- 172.21.15.60:0/3150190410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c069fa0 con 0x555bcef1bc00 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.350+0000 7fc3f57fa640 1 -- 172.21.15.60:0/3150190410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c082ad0 con 0x555bcef1bc00 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 msgr2=0x555bcef1fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc40c066390 tx=0x7fc40c051200 comp rx=0 tx=0).stop 2024-07-24T22:01:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 shutdown_connections 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef284b0 0x555bcef3fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 --2- 172.21.15.60:0/3150190410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef1bc00 0x555bcef1fd50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 >> 172.21.15.60:0/3150190410 conn(0x555bcef19a30 msgr2=0x555bcef26a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 shutdown_connections 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- 172.21.15.60:0/3150190410 wait complete. 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.351+0000 7fc413e59740 1 -- start start 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 Processor -- start 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc40414ecb0 con 0x555bcef284b0 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc404150120 con 0x555bcef1bc00 2024-07-24T22:01:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc40414f380 con 0x555bcef1c520 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45834/0 (socket says 172.21.15.60:45834) 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc40bfff640 1 -- 172.21.15.60:0/408283660 learned_addr learned my addr 172.21.15.60:0/408283660 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc413e59740 10 client.0 _ll_register_callbacks cb 0x7fc40c069ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.352+0000 7fc40affd640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 -- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 msgr2=0x555bcef220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 -- 172.21.15.60:0/408283660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 msgr2=0x555bcef309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 -- 172.21.15.60:0/408283660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcef22d90 con 0x555bcef284b0 2024-07-24T22:01:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc40bfff640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc40c065f10 tx=0x7fc40c084fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a8f469c602dc4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc3eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0ab020 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bcef2ace0 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc40c09f030 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc40c095020 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc413e59740 10 client.18609 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.353+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555bcefa2bf0 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555bcef2ace0 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc40c08d070 con 0x555bcef284b0 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 10 client.18609 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 10 client.18609 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 20 client.18609 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcefa51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:40.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 4 client.18609 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "538e6ea8", "release": true, "await": true}] 2024-07-24T22:01:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "538e6ea8", "release": true, "await": true}) -- 0x555bcee94d20 con 0x555bcef37fb0 2024-07-24T22:01:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.354+0000 7fc40affd640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcefa51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.355+0000 7fc40affd640 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcefa51a0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870b47984ee7d000 server_cookie=35d91e4dab45ea7 in_seq=0 out_seq=0 2024-07-24T22:01:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:39.355+0000 7fc3eaffd640 10 client.18609 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc3eaffd640 1 -- 172.21.15.60:0/408283660 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555bcee94d20 con 0x555bcef37fb0 2024-07-24T22:01:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc3eaffd640 10 client.18609 handle_command_reply: tid=0 2024-07-24T22:01:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 client.18609 shutdown 2024-07-24T22:01:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 msgr2=0x555bcef22620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc40c065f10 tx=0x7fc40c084fc0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 shutdown_connections 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555bcef37fb0 0x555bcefa51a0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc3eaffd640 0 client.18609 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bcef284b0 0x555bcef22620 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bcef1c520 0x555bcef220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 --2- 172.21.15.60:0/408283660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcef1bc00 0x555bcef309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.061+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 >> 172.21.15.60:0/408283660 conn(0x555bcef19a30 msgr2=0x555bcef25b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 44, 2024-07-24T22:01:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 48, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "538e6ea8": { 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 48, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 140.6, 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:01:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:41.066 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 shutdown_connections 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc413e59740 1 -- 172.21.15.60:0/408283660 wait complete. 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc413e59740 20 client.18609 trim_cache size 0 max 16384 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 msgr2=0x7fc3fc0b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 0x7fc3fc0b6230 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x555bcef1a910 tx=0x7fc40002a420 comp rx=0 tx=0).stop 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 msgr2=0x7fc4040a8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.062+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc40002aa10 tx=0x7fc400003e30 comp rx=0 tx=0).stop 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 shutdown_connections 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3fc0b3d70 0x7fc3fc0b6230 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4040a2440 0x7fc4040a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4040057f0 0x7fc4040a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 --2- 172.21.15.60:0/1603242628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc404004d10 0x7fc4040a8cd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 >> 172.21.15.60:0/1603242628 conn(0x7fc40400a150 msgr2=0x7fc4040972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 shutdown_connections 2024-07-24T22:01:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.063+0000 7fc410846640 1 -- 172.21.15.60:0/1603242628 wait complete. 2024-07-24T22:01:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:41 smithi060 ceph-mon[33000]: pgmap v846: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-07-24T22:01:41.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.984+0000 7fa202ffd640 1 -- 172.21.15.60:0/392936814 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec030830 con 0x7fa204109b40 2024-07-24T22:01:41.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 -- 172.21.15.60:0/392936814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204109b40 msgr2=0x7fa204110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/392936814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204109b40 0x7fa204110ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009890 tx=0x7fa1ec02a390 comp rx=0 tx=0).stop 2024-07-24T22:01:41.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 -- 172.21.15.60:0/392936814 shutdown_connections 2024-07-24T22:01:41.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/392936814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20411ba90 0x7fa20411de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/392936814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109f20 0x7fa20410a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/392936814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204109b40 0x7fa204110ba0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 -- 172.21.15.60:0/392936814 >> 172.21.15.60:0/392936814 conn(0x7fa20406c500 msgr2=0x7fa20406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.985+0000 7fa20a9a7640 1 -- 172.21.15.60:0/392936814 shutdown_connections 2024-07-24T22:01:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.986+0000 7fa20a9a7640 1 -- 172.21.15.60:0/392936814 wait complete. 2024-07-24T22:01:41.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 Processor -- start 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 -- start start 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 0x7fa2041a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa204109f20 0x7fa2041a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 0x7fa2041ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041213b0 con 0x7fa204109f20 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa204121230 con 0x7fa204109b40 2024-07-24T22:01:41.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.987+0000 7fa20a9a7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa204121530 con 0x7fa20411ba90 2024-07-24T22:01:41.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.988+0000 7fa208f44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 0x7fa2041ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:41.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.988+0000 7fa208f44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 0x7fa2041ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53184/0 (socket says 172.21.15.60:53184) 2024-07-24T22:01:41.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.988+0000 7fa208f44640 1 -- 172.21.15.60:0/261758192 learned_addr learned my addr 172.21.15.60:0/261758192 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:41.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.988+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 0x7fa2041a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 -- 172.21.15.60:0/261758192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 msgr2=0x7fa2041ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 0x7fa2041ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 -- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa204109f20 msgr2=0x7fa2041a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa204109f20 0x7fa2041a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 -- 172.21.15.60:0/261758192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2041ad950 con 0x7fa204109b40 2024-07-24T22:01:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.989+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 0x7fa2041a78a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec02a8b0 tx=0x7fa1ec030d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:41 smithi123 ceph-mon[37143]: pgmap v846: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-07-24T22:01:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.992+0000 7fa2017fa640 1 -- 172.21.15.60:0/261758192 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec040020 con 0x7fa204109b40 2024-07-24T22:01:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.992+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041adbe0 con 0x7fa204109b40 2024-07-24T22:01:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.992+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa204115290 con 0x7fa204109b40 2024-07-24T22:01:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.996+0000 7fa20cd58740 1 Processor -- start 2024-07-24T22:01:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.996+0000 7fa2017fa640 1 -- 172.21.15.60:0/261758192 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec038040 con 0x7fa204109b40 2024-07-24T22:01:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.996+0000 7fa2017fa640 1 -- 172.21.15.60:0/261758192 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa204109b40 2024-07-24T22:01:41.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.997+0000 7fa20cd58740 1 -- start start 2024-07-24T22:01:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb50170 0x5634adb5bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb5e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634adac69b0 con 0x5634adb50170 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634adab5eb0 con 0x5634adb4f850 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.998+0000 7fa20cd58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634adab9ef0 con 0x5634adb5c100 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.999+0000 7fa2037fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb5e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.999+0000 7fa208f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.999+0000 7fa208f44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41784/0 (socket says 172.21.15.60:41784) 2024-07-24T22:01:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:41.999+0000 7fa208f44640 1 -- 172.21.15.60:0/78487646 learned_addr learned my addr 172.21.15.60:0/78487646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:42.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.000+0000 7fa208f44640 1 -- 172.21.15.60:0/78487646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 msgr2=0x5634adb5e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.000+0000 7fa208f44640 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb5e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.000+0000 7fa208f44640 1 -- 172.21.15.60:0/78487646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb50170 msgr2=0x5634adb5bbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.000+0000 7fa208f44640 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb50170 0x5634adb5bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.000+0000 7fa208f44640 1 -- 172.21.15.60:0/78487646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ada3f320 con 0x5634adb4f850 2024-07-24T22:01:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.001+0000 7fa2017fa640 1 -- 172.21.15.60:0/261758192 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa1ec043020 con 0x7fa204109b40 2024-07-24T22:01:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa208f44640 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fb0 tx=0x7fa1fc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397061a19c72dd0c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa2017fa640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 0x7fa1f00b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa2017fa640 1 -- 172.21.15.60:0/261758192 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fa1ec103d10 con 0x7fa204109b40 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa1fa7fc640 1 -- 172.21.15.60:0/78487646 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc016070 con 0x5634adb4f850 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa1fa7fc640 1 -- 172.21.15.60:0/78487646 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc012bc0 con 0x5634adb4f850 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.002+0000 7fa1fa7fc640 1 -- 172.21.15.60:0/78487646 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc01bc20 con 0x5634adb4f850 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 -- 172.21.15.60:0/78487646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 msgr2=0x5634adb4fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fb0 tx=0x7fa1fc010800 comp rx=0 tx=0).stop 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 0x7fa1f00b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 -- 172.21.15.60:0/78487646 shutdown_connections 2024-07-24T22:01:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb5e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb50170 0x5634adb5bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 --2- 172.21.15.60:0/78487646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb4f850 0x5634adb4fc30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.004+0000 7fa20cd58740 1 -- 172.21.15.60:0/78487646 >> 172.21.15.60:0/78487646 conn(0x5634adb4d680 msgr2=0x5634adb4e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.005+0000 7fa203fff640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 0x7fa1f00b6480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5634adb4e560 tx=0x7fa1ec0ca8e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:42.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.006+0000 7fa20cd58740 1 -- 172.21.15.60:0/78487646 shutdown_connections 2024-07-24T22:01:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.006+0000 7fa20cd58740 1 -- 172.21.15.60:0/78487646 wait complete. 2024-07-24T22:01:42.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.007+0000 7fa20cd58740 1 -- start start 2024-07-24T22:01:42.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.007+0000 7fa20cd58740 1 Processor -- start 2024-07-24T22:01:42.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb59610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb59b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634ad919700 con 0x5634adb4f850 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634ada585f0 con 0x5634adb50170 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634adb5ed80 con 0x5634adb5c100 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa20cd58740 10 client.0 _ll_register_callbacks cb 0x5634ada78960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa2037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb59610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa2037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb59610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41798/0 (socket says 172.21.15.60:41798) 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa2037fe640 1 -- 172.21.15.60:0/568803611 learned_addr learned my addr 172.21.15.60:0/568803611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.009+0000 7fa208f44640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb59b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 -- 172.21.15.60:0/568803611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 msgr2=0x5634adb59610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb59610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 -- 172.21.15.60:0/568803611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 msgr2=0x5634adb590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 -- 172.21.15.60:0/568803611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ada4b680 con 0x5634adb5c100 2024-07-24T22:01:42.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa208f44640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb59b50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fb0 tx=0x7fa1fc01b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd56be1f9e6024a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.010+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc016070 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc012bc0 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fa1fc012510 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc012730 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa20cd58740 10 client.27886 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5634ad816650 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.011+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc012bc0 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa1fc00f040 con 0x5634adb5c100 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 10 client.27886 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:42.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 10 client.27886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:42.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 20 client.27886 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:42.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adb6df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 4 client.27886 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:42.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.012+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634adac8900 con 0x5634adb6bb30 2024-07-24T22:01:42.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.013+0000 7fa2037fe640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adb6df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.013+0000 7fa2037fe640 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adb6df20 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f46b354384b8a1 server_cookie=a59488d5def7853f in_seq=0 out_seq=0 2024-07-24T22:01:42.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.013+0000 7fa1e37fe640 10 client.27886 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:42.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.015+0000 7fa1e37fe640 1 -- 172.21.15.60:0/568803611 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634adac8900 con 0x5634adb6bb30 2024-07-24T22:01:42.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.015+0000 7fa1e37fe640 10 client.27886 handle_command_reply: tid=0 2024-07-24T22:01:42.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.015+0000 7fa20cd58740 1 client.27886 shutdown 2024-07-24T22:01:42.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 msgr2=0x5634adb59b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb59b50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fb0 tx=0x7fa1fc01b800 comp rx=0 tx=0).stop 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 shutdown_connections 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adb6df20 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb59b50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb59610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa1e37fe640 0 client.27886 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 --2- 172.21.15.60:0/568803611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 >> 172.21.15.60:0/568803611 conn(0x5634adb4d680 msgr2=0x5634adb53220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 shutdown_connections 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.016+0000 7fa20cd58740 1 -- 172.21.15.60:0/568803611 wait complete. 2024-07-24T22:01:42.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.017+0000 7fa20cd58740 20 client.27886 trim_cache size 0 max 16384 2024-07-24T22:01:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:41 smithi086 ceph-mon[37128]: pgmap v846: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-07-24T22:01:42.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.044+0000 7fa20cd58740 1 Processor -- start 2024-07-24T22:01:42.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.044+0000 7fa20cd58740 1 -- start start 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.044+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb73780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa20cd58740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e40112c0 con 0x5634adb4f850 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa20cd58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400ead0 con 0x5634adb50170 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa20cd58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400c940 con 0x5634adb5c100 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa2037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa2037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb73cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41812/0 (socket says 172.21.15.60:41812) 2024-07-24T22:01:42.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa2037fe640 1 -- 172.21.15.60:0/2623626762 learned_addr learned my addr 172.21.15.60:0/2623626762 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:42.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa208f44640 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa208f44640 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb73780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa208f44640 1 -- 172.21.15.60:0/2623626762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 msgr2=0x5634adb562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.045+0000 7fa208f44640 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa208f44640 1 -- 172.21.15.60:0/2623626762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 msgr2=0x5634adb73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa208f44640 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa208f44640 1 -- 172.21.15.60:0/2623626762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634adb4d4b0 con 0x5634adb4f850 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa208f44640 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb73780 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x5634adac6b00 tx=0x7fa1fc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f665fc056988730 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa1f9ffb640 1 -- 172.21.15.60:0/2623626762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc016070 con 0x5634adb4f850 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.046+0000 7fa1f9ffb640 1 -- 172.21.15.60:0/2623626762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc02c070 con 0x5634adb4f850 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa1f9ffb640 1 -- 172.21.15.60:0/2623626762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc027940 con 0x5634adb4f850 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 -- 172.21.15.60:0/2623626762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 msgr2=0x5634adb73780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb73780 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x5634adac6b00 tx=0x7fa1fc000f40 comp rx=0 tx=0).stop 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 -- 172.21.15.60:0/2623626762 shutdown_connections 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 --2- 172.21.15.60:0/2623626762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb73780 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.047+0000 7fa20cd58740 1 -- 172.21.15.60:0/2623626762 >> 172.21.15.60:0/2623626762 conn(0x5634adb4d680 msgr2=0x5634adb4c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.048+0000 7fa20cd58740 1 -- 172.21.15.60:0/2623626762 shutdown_connections 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.048+0000 7fa20cd58740 1 -- 172.21.15.60:0/2623626762 wait complete. 2024-07-24T22:01:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.048+0000 7fa20cd58740 1 -- start start 2024-07-24T22:01:42.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.048+0000 7fa20cd58740 1 Processor -- start 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb64730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400cf90 con 0x5634adb4f850 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400e400 con 0x5634adb50170 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400d660 con 0x5634adb5c100 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa20cd58740 10 client.0 _ll_register_callbacks cb 0x7fa1fc009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa208f44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb64730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa208f44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb64730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45904/0 (socket says 172.21.15.60:45904) 2024-07-24T22:01:42.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa208f44640 1 -- 172.21.15.60:0/3288408435 learned_addr learned my addr 172.21.15.60:0/3288408435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.049+0000 7fa2037fe640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa208f44640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 -- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 msgr2=0x5634adb601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 -- 172.21.15.60:0/3288408435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 msgr2=0x5634adb64730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb64730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 -- 172.21.15.60:0/3288408435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634adb60a10 con 0x5634adb50170 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa2037fe640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb5fca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000c00 tx=0x7fa1f4012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f501ac4eadac094a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.050+0000 7fa1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401b6a0 con 0x5634adb50170 2024-07-24T22:01:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc02c070 con 0x5634adb50170 2024-07-24T22:01:42.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fa1f401b840 con 0x5634adb50170 2024-07-24T22:01:42.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa1f401baa0 con 0x5634adb50170 2024-07-24T22:01:42.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa20cd58740 10 client.28001 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:42.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.051+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5634adbd6860 con 0x5634adb50170 2024-07-24T22:01:42.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.052+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc02c070 con 0x5634adb50170 2024-07-24T22:01:42.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.052+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa1f400f040 con 0x5634adb50170 2024-07-24T22:01:42.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 10 client.28001 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:42.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 10 client.28001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 20 client.28001 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adbd9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 4 client.28001 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634adac43a0 con 0x5634adb6bb30 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.053+0000 7fa208f44640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adbd9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa208f44640 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adbd9030 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb781663442e82d0 server_cookie=8771a2f14d3cc323 in_seq=0 out_seq=0 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa1e37fe640 10 client.28001 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa1e37fe640 1 -- 172.21.15.60:0/3288408435 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634adac43a0 con 0x5634adb6bb30 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa1e37fe640 10 client.28001 handle_command_reply: tid=0 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa20cd58740 1 client.28001 shutdown 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 msgr2=0x5634adb5fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb5fca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000c00 tx=0x7fa1f4012820 comp rx=0 tx=0).stop 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 shutdown_connections 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.055+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634adb6bb30 0x5634adbd9030 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa1e37fe640 0 client.28001 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634adb5c100 0x5634adb601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634adb50170 0x5634adb5fca0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 --2- 172.21.15.60:0/3288408435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634adb4f850 0x5634adb64730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 >> 172.21.15.60:0/3288408435 conn(0x5634adb4d680 msgr2=0x5634adb4beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 shutdown_connections 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 1 -- 172.21.15.60:0/3288408435 wait complete. 2024-07-24T22:01:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.056+0000 7fa20cd58740 20 client.28001 trim_cache size 0 max 16384 2024-07-24T22:01:42.061 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:01:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 msgr2=0x7fa1f00b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 0x7fa1f00b6480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5634adb4e560 tx=0x7fa1ec0ca8e0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 msgr2=0x7fa2041a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 0x7fa2041a78a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec02a8b0 tx=0x7fa1ec030d60 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 shutdown_connections 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa1f00b3fc0 0x7fa1f00b6480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20411ba90 0x7fa2041ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa204109f20 0x7fa2041a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 --2- 172.21.15.60:0/261758192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204109b40 0x7fa2041a78a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.057+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 >> 172.21.15.60:0/261758192 conn(0x7fa20406c500 msgr2=0x7fa20411bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.058+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 shutdown_connections 2024-07-24T22:01:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:42.058+0000 7fa20a9a7640 1 -- 172.21.15.60:0/261758192 wait complete. 2024-07-24T22:01:43.091 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '538e6ea8', total seconds elapsed: 36.3 2024-07-24T22:01:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:43 smithi123 ceph-mon[37143]: pgmap v847: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-07-24T22:01:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:43 smithi086 ceph-mon[37128]: pgmap v847: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-07-24T22:01:44.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:43 smithi060 ceph-mon[33000]: pgmap v847: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-07-24T22:01:44.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T22:01:44.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T22:01:45.314 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:45.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:45 smithi060 ceph-mon[33000]: pgmap v848: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:01:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:45 smithi123 ceph-mon[37143]: pgmap v848: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:01:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:45 smithi086 ceph-mon[37128]: pgmap v848: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4019914559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4000a6580 msgr2=0x7fc4000a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4019914559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4000a6580 0x7fc4000a8970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4080685c0 tx=0x7fc4080978b0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4019914559 shutdown_connections 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4019914559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4000a6580 0x7fc4000a8970 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4019914559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400007430 0x7fc4000a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4019914559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc400006b10 0x7fc400006ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.315+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4019914559 >> 172.21.15.60:0/4019914559 conn(0x7fc400091250 msgr2=0x7fc400093670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.318+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4019914559 shutdown_connections 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.318+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4019914559 wait complete. 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.319+0000 7fc40ef6d640 1 Processor -- start 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.319+0000 7fc40ef6d640 1 -- start start 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc400007430 0x7fc400143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000a6580 0x7fc400148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000abd70 con 0x7fc400006b10 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000abbf0 con 0x7fc400007430 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40ef6d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000abef0 con 0x7fc4000a6580 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40df6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40df6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48872/0 (socket says 172.21.15.60:48872) 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40df6b640 1 -- 172.21.15.60:0/4233174311 learned_addr learned my addr 172.21.15.60:0/4233174311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40d76a640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc400007430 0x7fc400143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.320+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000a6580 0x7fc400148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 -- 172.21.15.60:0/4233174311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000a6580 msgr2=0x7fc400148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000a6580 0x7fc400148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 -- 172.21.15.60:0/4233174311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc400007430 msgr2=0x7fc400143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc400007430 0x7fc400143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 -- 172.21.15.60:0/4233174311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4001494a0 con 0x7fc400006b10 2024-07-24T22:01:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.321+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9f0 tx=0x7fc40400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.325+0000 7fc3faffd640 1 -- 172.21.15.60:0/4233174311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40400ccc0 con 0x7fc400006b10 2024-07-24T22:01:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.325+0000 7fc3faffd640 1 -- 172.21.15.60:0/4233174311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4040108e0 con 0x7fc400006b10 2024-07-24T22:01:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.325+0000 7fc3faffd640 1 -- 172.21.15.60:0/4233174311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404017780 con 0x7fc400006b10 2024-07-24T22:01:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.325+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc400149790 con 0x7fc400006b10 2024-07-24T22:01:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.325+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc400099810 con 0x7fc400006b10 2024-07-24T22:01:47.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.331+0000 7fc412580740 1 Processor -- start 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.331+0000 7fc412580740 1 -- start start 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5a20 0x5631556e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556e19b0 0x5631556e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56315564c9f0 con 0x5631556e19b0 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56315563bef0 con 0x5631556d5100 2024-07-24T22:01:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc412580740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56315563ff30 con 0x5631556d5a20 2024-07-24T22:01:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc3faffd640 1 -- 172.21.15.60:0/4233174311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc404010a90 con 0x7fc400006b10 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.332+0000 7fc40e76c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:58022/0 (socket says 172.21.15.60:58022) 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 -- 172.21.15.60:0/442812496 learned_addr learned my addr 172.21.15.60:0/442812496 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc3faffd640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 0x7fc3f40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 -- 172.21.15.60:0/442812496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5a20 msgr2=0x5631556e1470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5a20 0x5631556e1470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 -- 172.21.15.60:0/442812496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556e19b0 msgr2=0x5631556e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556e19b0 0x5631556e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.333+0000 7fc40e76c640 1 -- 172.21.15.60:0/442812496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315539c930 con 0x5631556d5100 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.334+0000 7fc3faffd640 1 -- 172.21.15.60:0/4233174311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc4040a7740 con 0x7fc400006b10 2024-07-24T22:01:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.334+0000 7fc40e76c640 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc408051b40 tx=0x7fc408068b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10efcaf29a2b149f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.335+0000 7fc3f3fff640 1 -- 172.21.15.60:0/442812496 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408069ca0 con 0x5631556d5100 2024-07-24T22:01:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.335+0000 7fc3f3fff640 1 -- 172.21.15.60:0/442812496 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40809eb80 con 0x5631556d5100 2024-07-24T22:01:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 -- 172.21.15.60:0/442812496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 msgr2=0x5631556d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc3f3fff640 1 -- 172.21.15.60:0/442812496 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408069ca0 con 0x5631556d5100 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 0x7fc3f40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc408051b40 tx=0x7fc408068b80 comp rx=0 tx=0).stop 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 -- 172.21.15.60:0/442812496 shutdown_connections 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556e19b0 0x5631556e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5a20 0x5631556e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 --2- 172.21.15.60:0/442812496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556d54e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.336+0000 7fc412580740 1 -- 172.21.15.60:0/442812496 >> 172.21.15.60:0/442812496 conn(0x5631556d2f30 msgr2=0x5631556d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.337+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 0x7fc3f40b6390 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc408051280 tx=0x7fc408068b80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.337+0000 7fc412580740 1 -- 172.21.15.60:0/442812496 shutdown_connections 2024-07-24T22:01:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.337+0000 7fc412580740 1 -- 172.21.15.60:0/442812496 wait complete. 2024-07-24T22:01:47.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.338+0000 7fc412580740 1 -- start start 2024-07-24T22:01:47.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.338+0000 7fc412580740 1 Processor -- start 2024-07-24T22:01:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.338+0000 7fc412580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.338+0000 7fc412580740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x5631556de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.338+0000 7fc412580740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc412580740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56315549f890 con 0x5631556d5a20 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc412580740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631555de390 con 0x5631556d5100 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc412580740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5631556e4620 con 0x5631556e19b0 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc412580740 10 client.0 _ll_register_callbacks cb 0x5631555e8500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc40e76c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc40e76c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55010/0 (socket says 172.21.15.60:55010) 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc40e76c640 1 -- 172.21.15.60:0/4057021125 learned_addr learned my addr 172.21.15.60:0/4057021125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.339+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.340+0000 7fc40e76c640 1 -- 172.21.15.60:0/4057021125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 msgr2=0x5631556e1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.340+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556e1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.340+0000 7fc40e76c640 1 -- 172.21.15.60:0/4057021125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 msgr2=0x5631556de900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.340+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x5631556de900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.340+0000 7fc40e76c640 1 -- 172.21.15.60:0/4057021125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631555d8d60 con 0x5631556e19b0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.341+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4080521d0 tx=0x7fc408065db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c790d8674e9e9ff7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.341+0000 7fc3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.341+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4080c4060 con 0x5631556e19b0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.341+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40809eb80 con 0x5631556e19b0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.341+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc40806c560 con 0x5631556e19b0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.342+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc40806b610 con 0x5631556e19b0 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.342+0000 7fc412580740 10 client.27901 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.342+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563155712520 con 0x5631556e19b0 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.343+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40809eb80 con 0x5631556e19b0 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc4080746d0 con 0x5631556e19b0 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 10 client.27901 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 10 client.27901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 20 client.27901 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563155713870 0x563155715c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 4 client.27901 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:47.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.344+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315564e940 con 0x563155713870 2024-07-24T22:01:47.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.346+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563155713870 0x563155715c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.357+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563155713870 0x563155715c60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1899525391ad596 server_cookie=7bee7cd25cd193b7 in_seq=0 out_seq=0 2024-07-24T22:01:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.357+0000 7fc3f17fa640 10 client.27901 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.364+0000 7fc3f17fa640 1 -- 172.21.15.60:0/4057021125 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56315564e940 con 0x563155713870 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.364+0000 7fc3f17fa640 10 client.27901 handle_command_reply: tid=0 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.364+0000 7fc412580740 1 client.27901 shutdown 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.364+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 msgr2=0x5631556df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.364+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4080521d0 tx=0x7fc408065db0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 shutdown_connections 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563155713870 0x563155715c60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x5631556df090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x5631556de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 --2- 172.21.15.60:0/4057021125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5100 0x5631556e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc3f17fa640 0 client.27901 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:47.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.365+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 >> 172.21.15.60:0/4057021125 conn(0x5631556d2f30 msgr2=0x5631556d8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.367+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 shutdown_connections 2024-07-24T22:01:47.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.367+0000 7fc412580740 1 -- 172.21.15.60:0/4057021125 wait complete. 2024-07-24T22:01:47.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.367+0000 7fc412580740 20 client.27901 trim_cache size 0 max 16384 2024-07-24T22:01:47.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.404+0000 7fc412580740 1 Processor -- start 2024-07-24T22:01:47.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.404+0000 7fc412580740 1 -- start start 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc412580740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5100 0x5631556d4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc412580740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc412580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 0x56315571b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc40d76a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc40d76a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48900/0 (socket says 172.21.15.60:48900) 2024-07-24T22:01:47.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc40d76a640 1 -- 172.21.15.60:0/4238276606 learned_addr learned my addr 172.21.15.60:0/4238276606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.405+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc400133cf0 con 0x5631556d5a20 2024-07-24T22:01:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.406+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc400131540 con 0x5631556e19b0 2024-07-24T22:01:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.406+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 0x56315571b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.406+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc400006ef0 con 0x5631556d5100 2024-07-24T22:01:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.407+0000 7fc40df6b640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5100 0x5631556d4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 -- 172.21.15.60:0/4238276606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5100 msgr2=0x5631556d4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5100 0x5631556d4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 -- 172.21.15.60:0/4238276606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 msgr2=0x56315571b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 -- 172.21.15.60:0/4238276606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631556dd820 con 0x5631556e19b0 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.410+0000 7fc40e76c640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 0x56315571b800 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc408052280 tx=0x7fc40806cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168f1489bce1b8e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.411+0000 7fc40d76a640 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.411+0000 7fc3f37fe640 1 -- 172.21.15.60:0/4238276606 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4080ce080 con 0x5631556e19b0 2024-07-24T22:01:47.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.412+0000 7fc3f37fe640 1 -- 172.21.15.60:0/4238276606 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40809e6d0 con 0x5631556e19b0 2024-07-24T22:01:47.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.412+0000 7fc3f37fe640 1 -- 172.21.15.60:0/4238276606 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408074220 con 0x5631556e19b0 2024-07-24T22:01:47.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.413+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 msgr2=0x56315571b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.413+0000 7fc412580740 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 0x56315571b800 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc408052280 tx=0x7fc40806cbd0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.414+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 shutdown_connections 2024-07-24T22:01:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.414+0000 7fc412580740 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556e19b0 0x56315571b800 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.414+0000 7fc412580740 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5a20 0x56315571b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.414+0000 7fc412580740 1 --2- 172.21.15.60:0/4238276606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556d5100 0x5631556d4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.414+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 >> 172.21.15.60:0/4238276606 conn(0x5631556d2f30 msgr2=0x5631556d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.417+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 shutdown_connections 2024-07-24T22:01:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.417+0000 7fc412580740 1 -- 172.21.15.60:0/4238276606 wait complete. 2024-07-24T22:01:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.417+0000 7fc412580740 1 -- start start 2024-07-24T22:01:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.418+0000 7fc412580740 1 Processor -- start 2024-07-24T22:01:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.418+0000 7fc412580740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5100 0x5631556e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc412580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc412580740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x56315577d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc40d76a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc40d76a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:58036/0 (socket says 172.21.15.60:58036) 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc40d76a640 1 -- 172.21.15.60:0/2494272047 learned_addr learned my addr 172.21.15.60:0/2494272047 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000a0820 con 0x5631556d5100 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000a1c40 con 0x5631556d5a20 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.419+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4000a0ea0 con 0x5631556e19b0 2024-07-24T22:01:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.420+0000 7fc412580740 10 client.0 _ll_register_callbacks cb 0x7fc40809ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.420+0000 7fc40e76c640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x56315577d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.420+0000 7fc40d76a640 1 -- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 msgr2=0x56315577d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.420+0000 7fc40d76a640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x56315577d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.420+0000 7fc40d76a640 1 -- 172.21.15.60:0/2494272047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5100 msgr2=0x5631556e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.421+0000 7fc40d76a640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5100 0x5631556e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.421+0000 7fc40d76a640 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631556e17e0 con 0x5631556d5a20 2024-07-24T22:01:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.421+0000 7fc40d76a640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0102c0 tx=0x7fc3fc010780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc859c32dd9114e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.422+0000 7fc3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.422+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc018070 con 0x5631556d5a20 2024-07-24T22:01:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.422+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7fc3fc014800 con 0x5631556d5a20 2024-07-24T22:01:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.422+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40809e6d0 con 0x5631556d5a20 2024-07-24T22:01:47.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.424+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc3fc024050 con 0x5631556d5a20 2024-07-24T22:01:47.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.424+0000 7fc412580740 10 client.28019 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:47.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.424+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56315577e9c0 con 0x5631556d5a20 2024-07-24T22:01:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc01c3b0 con 0x5631556d5a20 2024-07-24T22:01:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fc3fc013370 con 0x5631556d5a20 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 10 client.28019 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 10 client.28019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 20 client.28019 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56315577f1b0 0x5631557815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 4 client.28019 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.425+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc40809e6d0 con 0x56315577f1b0 2024-07-24T22:01:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.426+0000 7fc40df6b640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56315577f1b0 0x5631557815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.427+0000 7fc3f17fa640 10 client.28019 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.428+0000 7fc40df6b640 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56315577f1b0 0x5631557815d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7664c21cc876b512 server_cookie=c2eea077f51b2321 in_seq=0 out_seq=0 2024-07-24T22:01:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc3f17fa640 1 -- 172.21.15.60:0/2494272047 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc40809e6d0 con 0x56315577f1b0 2024-07-24T22:01:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc3f17fa640 10 client.28019 handle_command_reply: tid=0 2024-07-24T22:01:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 client.28019 shutdown 2024-07-24T22:01:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 msgr2=0x5631556e6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0102c0 tx=0x7fc3fc010780 comp rx=0 tx=0).stop 2024-07-24T22:01:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 shutdown_connections 2024-07-24T22:01:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56315577f1b0 0x5631557815d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5631556e19b0 0x56315577d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631556d5a20 0x5631556e6450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 --2- 172.21.15.60:0/2494272047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5631556d5100 0x5631556e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 >> 172.21.15.60:0/2494272047 conn(0x5631556d2f30 msgr2=0x5631556dfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.464+0000 7fc3f17fa640 0 client.28019 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.465+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 shutdown_connections 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.465+0000 7fc412580740 1 -- 172.21.15.60:0/2494272047 wait complete. 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.465+0000 7fc412580740 20 client.28019 trim_cache size 0 max 16384 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T22:01:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "54f5cd12-22db-4b60-97f1-94128537550b", 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 msgr2=0x7fc3f40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 0x7fc3f40b6390 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc408051280 tx=0x7fc408068b80 comp rx=0 tx=0).stop 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 msgr2=0x7fc4001433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9f0 tx=0x7fc40400edf0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 shutdown_connections 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc3f40b3ed0 0x7fc3f40b6390 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000a6580 0x7fc400148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc400007430 0x7fc400143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 --2- 172.21.15.60:0/4233174311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc400006b10 0x7fc4001433f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 >> 172.21.15.60:0/4233174311 conn(0x7fc400091250 msgr2=0x7fc400091f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 shutdown_connections 2024-07-24T22:01:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:47.466+0000 7fc40ef6d640 1 -- 172.21.15.60:0/4233174311 wait complete. 2024-07-24T22:01:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:47 smithi123 ceph-mon[37143]: pgmap v849: 97 pgs: 26 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 363 KiB/s wr, 33 op/s 2024-07-24T22:01:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:47 smithi086 ceph-mon[37128]: pgmap v849: 97 pgs: 26 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 363 KiB/s wr, 33 op/s 2024-07-24T22:01:48.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:47 smithi060 ceph-mon[33000]: pgmap v849: 97 pgs: 26 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 58 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 363 KiB/s wr, 33 op/s 2024-07-24T22:01:48.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 -- 172.21.15.60:0/583798380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36300aa090 msgr2=0x7f36300aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 --2- 172.21.15.60:0/583798380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36300aa090 0x7f36300aa4d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3638068420 tx=0x7f363808ed10 comp rx=0 tx=0).stop 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 -- 172.21.15.60:0/583798380 shutdown_connections 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 --2- 172.21.15.60:0/583798380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 0x7f36300ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 --2- 172.21.15.60:0/583798380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36300aa090 0x7f36300aa4d0 secure :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3638068420 tx=0x7f363808ed10 comp rx=0 tx=0).stop 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 --2- 172.21.15.60:0/583798380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 0x7f36300a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.854+0000 7f363f576640 1 -- 172.21.15.60:0/583798380 >> 172.21.15.60:0/583798380 conn(0x7f363001a080 msgr2=0x7f363001c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.855+0000 7f363f576640 1 -- 172.21.15.60:0/583798380 shutdown_connections 2024-07-24T22:01:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.855+0000 7f363f576640 1 -- 172.21.15.60:0/583798380 wait complete. 2024-07-24T22:01:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 Processor -- start 2024-07-24T22:01:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 -- start start 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 0x7f3630143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 0x7f3630143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36300bce70 con 0x7f3630148f10 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36300bccf0 con 0x7f36300a4a10 2024-07-24T22:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.856+0000 7f363f576640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36300bcff0 con 0x7f36300aaa10 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.857+0000 7f363ed75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.857+0000 7f363ed75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48932/0 (socket says 172.21.15.60:48932) 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.857+0000 7f363ed75640 1 -- 172.21.15.60:0/3185161727 learned_addr learned my addr 172.21.15.60:0/3185161727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.857+0000 7f363e574640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 0x7f3630143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.857+0000 7f363dd73640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 0x7f3630143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 -- 172.21.15.60:0/3185161727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 msgr2=0x7f3630143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 0x7f3630143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 -- 172.21.15.60:0/3185161727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 msgr2=0x7f3630143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 0x7f3630143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 -- 172.21.15.60:0/3185161727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3630149af0 con 0x7f3630148f10 2024-07-24T22:01:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.858+0000 7f363ed75640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3628009fd0 tx=0x7f362800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.859+0000 7f362f7fe640 1 -- 172.21.15.60:0/3185161727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628011830 con 0x7f3630148f10 2024-07-24T22:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.859+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3630150090 con 0x7f3630148f10 2024-07-24T22:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.859+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3630150510 con 0x7f3630148f10 2024-07-24T22:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.859+0000 7f362f7fe640 1 -- 172.21.15.60:0/3185161727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3628016070 con 0x7f3630148f10 2024-07-24T22:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.859+0000 7f362f7fe640 1 -- 172.21.15.60:0/3185161727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628010df0 con 0x7f3630148f10 2024-07-24T22:01:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.863+0000 7f3643605740 1 Processor -- start 2024-07-24T22:01:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.864+0000 7f362f7fe640 1 -- 172.21.15.60:0/3185161727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3628010380 con 0x7f3630148f10 2024-07-24T22:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.864+0000 7f3643605740 1 -- start start 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8c3c0 0x560acdf97e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560acdf01280 con 0x560acdf98350 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560acdef0780 con 0x560acdf8c3c0 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f3643605740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560acdef47c0 con 0x560acdf8baa0 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f363dd73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f363dd73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48942/0 (socket says 172.21.15.60:48942) 2024-07-24T22:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f363dd73640 1 -- 172.21.15.60:0/3633965549 learned_addr learned my addr 172.21.15.60:0/3633965549 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.865+0000 7f363dd73640 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 -- 172.21.15.60:0/3633965549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 msgr2=0x560acdf8be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf8be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 -- 172.21.15.60:0/3633965549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8c3c0 msgr2=0x560acdf97e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8c3c0 0x560acdf97e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 -- 172.21.15.60:0/3633965549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560acdc54790 con 0x560acdf98350 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f362f7fe640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 0x7f36240b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f362f7fe640 1 -- 172.21.15.60:0/3185161727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f36280e3480 con 0x7f3630148f10 2024-07-24T22:01:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.868+0000 7f363dd73640 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f36380521d0 tx=0x7f363809a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5db7e1902fc99ab server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.869+0000 7f362cff9640 1 -- 172.21.15.60:0/3633965549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380697d0 con 0x560acdf98350 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.869+0000 7f362cff9640 1 -- 172.21.15.60:0/3633965549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3638093070 con 0x560acdf98350 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 -- 172.21.15.60:0/3633965549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 msgr2=0x560acdf9a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f36380521d0 tx=0x7f363809a040 comp rx=0 tx=0).stop 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 -- 172.21.15.60:0/3633965549 shutdown_connections 2024-07-24T22:01:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf9a740 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8c3c0 0x560acdf97e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 --2- 172.21.15.60:0/3633965549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf8be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.870+0000 7f3643605740 1 -- 172.21.15.60:0/3633965549 >> 172.21.15.60:0/3633965549 conn(0x560acdf898d0 msgr2=0x560acdf8aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.874+0000 7f3643605740 1 -- 172.21.15.60:0/3633965549 shutdown_connections 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.874+0000 7f3643605740 1 -- 172.21.15.60:0/3633965549 wait complete. 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.874+0000 7f3643605740 1 -- start start 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 Processor -- start 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560acde2aab0 con 0x560acdf98350 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560acde789e0 con 0x560acdf8baa0 2024-07-24T22:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560acdf9b020 con 0x560acdf95740 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f3643605740 10 client.0 _ll_register_callbacks cb 0x560acdc00a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f363ed75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f363ed75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55062/0 (socket says 172.21.15.60:55062) 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.875+0000 7f363ed75640 1 -- 172.21.15.60:0/1974227455 learned_addr learned my addr 172.21.15.60:0/1974227455 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.876+0000 7f363e574640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363e574640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 0x7f36240b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 -- 172.21.15.60:0/1974227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 msgr2=0x560acdf97cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf97cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 -- 172.21.15.60:0/1974227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 msgr2=0x560acdf95110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 -- 172.21.15.60:0/1974227455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560acdccd620 con 0x560acdf95740 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f363ed75640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36280aa070 tx=0x7f36280a9d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fb89389df14fd2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.877+0000 7f3621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.878+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280b3190 con 0x560acdf95740 2024-07-24T22:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.879+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560acde9fce0 con 0x560acdf95740 2024-07-24T22:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.879+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f36280a9340 con 0x560acdf95740 2024-07-24T22:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.879+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f36280a9580 con 0x560acdf95740 2024-07-24T22:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.880+0000 7f363e574640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 0x7f36240b6350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x560acdf8a5b0 tx=0x7f3634006ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.881+0000 7f3643605740 10 client.27916 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.881+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560acdfa6e80 con 0x560acdf95740 2024-07-24T22:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.882+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560acde9fce0 con 0x560acdf95740 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f36280a8e40 con 0x560acdf95740 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 10 client.27916 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 10 client.27916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 20 client.27916 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560acdfa88e0 0x560acdfaad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 4 client.27916 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.883+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560acdf031d0 con 0x560acdfa88e0 2024-07-24T22:01:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.884+0000 7f363e574640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560acdfa88e0 0x560acdfaad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.885+0000 7f3621ffb640 10 client.27916 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.885+0000 7f363e574640 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560acdfa88e0 0x560acdfaad40 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b7b4a7fe8e94a7 server_cookie=bcbfb6b2a9eb378b in_seq=0 out_seq=0 2024-07-24T22:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.891+0000 7f3621ffb640 1 -- 172.21.15.60:0/1974227455 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560acdf031d0 con 0x560acdfa88e0 2024-07-24T22:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.891+0000 7f3621ffb640 10 client.27916 handle_command_reply: tid=0 2024-07-24T22:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.891+0000 7f3643605740 1 client.27916 shutdown 2024-07-24T22:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 msgr2=0x560acdf95bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36280aa070 tx=0x7f36280a9d30 comp rx=0 tx=0).stop 2024-07-24T22:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 shutdown_connections 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560acdfa88e0 0x560acdfaad40 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf95740 0x560acdf95bc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf98350 0x560acdf95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 --2- 172.21.15.60:0/1974227455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf97cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 >> 172.21.15.60:0/1974227455 conn(0x560acdf898d0 msgr2=0x560acdf8f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3621ffb640 0 client.27916 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 shutdown_connections 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 1 -- 172.21.15.60:0/1974227455 wait complete. 2024-07-24T22:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.892+0000 7f3643605740 20 client.27916 trim_cache size 0 max 16384 2024-07-24T22:01:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 Processor -- start 2024-07-24T22:01:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 -- start start 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf93970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdf93eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f361c011200 con 0x560acdf95740 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f361c00ea10 con 0x560acdf8baa0 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.930+0000 7f3643605740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f361c00c910 con 0x560acdf98350 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55076/0 (socket says 172.21.15.60:55076) 2024-07-24T22:01:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 -- 172.21.15.60:0/3624788182 learned_addr learned my addr 172.21.15.60:0/3624788182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 -- 172.21.15.60:0/3624788182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 msgr2=0x560acdf93970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf93970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 -- 172.21.15.60:0/3624788182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 msgr2=0x560acdf93eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdf93eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 -- 172.21.15.60:0/3624788182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560acdf89700 con 0x560acdf98350 2024-07-24T22:01:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.931+0000 7f363ed75640 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36280a9de0 tx=0x7f36280047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13fec9b92c24a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.933+0000 7f3623fff640 1 -- 172.21.15.60:0/3624788182 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280c9070 con 0x560acdf98350 2024-07-24T22:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.933+0000 7f3623fff640 1 -- 172.21.15.60:0/3624788182 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36280b3190 con 0x560acdf98350 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 -- 172.21.15.60:0/3624788182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 msgr2=0x560acdf924a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36280a9de0 tx=0x7f36280047d0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 -- 172.21.15.60:0/3624788182 shutdown_connections 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf98350 0x560acdf924a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdf93eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 --2- 172.21.15.60:0/3624788182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdf8baa0 0x560acdf93970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 -- 172.21.15.60:0/3624788182 >> 172.21.15.60:0/3624788182 conn(0x560acdf898d0 msgr2=0x560acdf8a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.934+0000 7f3643605740 1 -- 172.21.15.60:0/3624788182 shutdown_connections 2024-07-24T22:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.935+0000 7f3643605740 1 -- 172.21.15.60:0/3624788182 wait complete. 2024-07-24T22:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 -- start start 2024-07-24T22:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 Processor -- start 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf92c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdfa0940 0x560acdf91490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f361c00ced0 con 0x560acdf95740 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f361c00e340 con 0x560acdfa0940 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f361c00d5a0 con 0x560acdf8baa0 2024-07-24T22:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f3643605740 10 client.0 _ll_register_callbacks cb 0x7f36280a6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.936+0000 7f363dd73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48958/0 (socket says 172.21.15.60:48958) 2024-07-24T22:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 -- 172.21.15.60:0/1846023089 learned_addr learned my addr 172.21.15.60:0/1846023089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363ed75640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdfa0940 0x560acdf91490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363e574640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf92c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 -- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 msgr2=0x560acdf92c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf92c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 -- 172.21.15.60:0/1846023089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdfa0940 msgr2=0x560acdf91490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdfa0940 0x560acdf91490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363dd73640 1 -- 172.21.15.60:0/1846023089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560acdf919d0 con 0x560acdf95740 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.937+0000 7f363e574640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf92c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.938+0000 7f363dd73640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3638051b40 tx=0x7f363808ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e6af7ab67d1f3f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.938+0000 7f3621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.939+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363808fa20 con 0x560acdf95740 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.940+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560acdefec70 con 0x560acdf95740 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.940+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8609+0+0 (secure 0 0 0) 0x7f363808fbc0 con 0x560acdf95740 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.940+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f363808fdc0 con 0x560acdf95740 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.940+0000 7f3643605740 10 client.18636 fetch_fsmap learned FSMap version 44 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.940+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560ace012d80 con 0x560acdf95740 2024-07-24T22:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.941+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560acdefec70 con 0x560acdf95740 2024-07-24T22:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f36380b8430 con 0x560acdf95740 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 10 client.18636 fetch_fsmap finished waiting for FSMap version 44 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 20 client.18636 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560ace013060 0x560ace015480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 4 client.18636 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.942+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560acdefc3b0 con 0x560ace013060 2024-07-24T22:01:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.944+0000 7f363e574640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560ace013060 0x560ace015480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.956+0000 7f363e574640 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560ace013060 0x560ace015480 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4130e512dd9527f server_cookie=98b4771d7d20ef35 in_seq=0 out_seq=0 2024-07-24T22:01:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:48.957+0000 7f3621ffb640 10 client.18636 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:48 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T22:01:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:48 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:48 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:01:49.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:48 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T22:01:49.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:48 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:49.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:48 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:01:49.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:48 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T22:01:49.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:48 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:49.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:48 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:01:49.258 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:01:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3621ffb640 1 -- 172.21.15.60:0/1846023089 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560acdefc3b0 con 0x560ace013060 2024-07-24T22:01:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3621ffb640 10 client.18636 handle_command_reply: tid=0 2024-07-24T22:01:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 client.18636 shutdown 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 msgr2=0x560acdfa0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3638051b40 tx=0x7f363808ed10 comp rx=0 tx=0).stop 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 shutdown_connections 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x560ace013060 0x560ace015480 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560acdfa0940 0x560acdf91490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560acdf95740 0x560acdfa0360 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 --2- 172.21.15.60:0/1846023089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560acdf8baa0 0x560acdf92c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 >> 172.21.15.60:0/1846023089 conn(0x560acdf898d0 msgr2=0x560acdf938b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3621ffb640 0 client.18636 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.400+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 shutdown_connections 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.401+0000 7f3643605740 1 -- 172.21.15.60:0/1846023089 wait complete. 2024-07-24T22:01:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.401+0000 7f3643605740 20 client.18636 trim_cache size 0 max 16384 2024-07-24T22:01:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:01:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:01:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:01:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:01:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.402+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 msgr2=0x7f36240b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.402+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 0x7f36240b6350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x560acdf8a5b0 tx=0x7f3634006ab0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.402+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 msgr2=0x7f36301493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.402+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3628009fd0 tx=0x7f362800b8e0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 shutdown_connections 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f36240b3e90 0x7f36240b6350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3630148f10 0x7f36301493c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36300aaa10 0x7f3630143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 --2- 172.21.15.60:0/3185161727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36300a4a10 0x7f3630143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.403+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 >> 172.21.15.60:0/3185161727 conn(0x7f363001a080 msgr2=0x7f36300a3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.404+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 shutdown_connections 2024-07-24T22:01:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:01:49.404+0000 7f363f576640 1 -- 172.21.15.60:0/3185161727 wait complete. 2024-07-24T22:01:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:49 smithi123 ceph-mon[37143]: pgmap v850: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 577 KiB/s wr, 73 op/s 2024-07-24T22:01:49.993 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:01:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: ignoring --setuser ceph since I am not root 2024-07-24T22:01:49.993 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:01:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: ignoring --setgroup ceph since I am not root 2024-07-24T22:01:49.993 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:01:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: starting mds.l at 2024-07-24T22:01:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:49 smithi086 ceph-mon[37128]: pgmap v850: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 577 KiB/s wr, 73 op/s 2024-07-24T22:01:50.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:49 smithi060 ceph-mon[33000]: pgmap v850: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 577 KiB/s wr, 73 op/s 2024-07-24T22:01:50.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:50 smithi060 ceph-mon[33000]: daemon mds.l restarted 2024-07-24T22:01:50.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:50 smithi123 ceph-mon[37143]: daemon mds.l restarted 2024-07-24T22:01:51.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:50 smithi086 ceph-mon[37128]: daemon mds.l restarted 2024-07-24T22:01:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 -- 172.21.15.60:0/1078565422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc1079e0 msgr2=0x7feffc10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 --2- 172.21.15.60:0/1078565422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc1079e0 0x7feffc10e1e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fefec009860 tx=0x7fefec02a360 comp rx=0 tx=0).stop 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 -- 172.21.15.60:0/1078565422 shutdown_connections 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 --2- 172.21.15.60:0/1078565422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc1079e0 0x7feffc10e1e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 --2- 172.21.15.60:0/1078565422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc107060 0x7feffc1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 --2- 172.21.15.60:0/1078565422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc101650 0x7feffc101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.652+0000 7ff001907640 1 -- 172.21.15.60:0/1078565422 >> 172.21.15.60:0/1078565422 conn(0x7feffc0fd0c0 msgr2=0x7feffc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.653+0000 7ff001907640 1 -- 172.21.15.60:0/1078565422 shutdown_connections 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.653+0000 7ff001907640 1 -- 172.21.15.60:0/1078565422 wait complete. 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.653+0000 7ff001907640 1 Processor -- start 2024-07-24T22:01:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.654+0000 7ff001907640 1 -- start start 2024-07-24T22:01:51.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.654+0000 7ff001907640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc101650 0x7feffc0741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.654+0000 7ff001907640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 0x7feffc074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7ff001907640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 0x7feffc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7ff001907640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feffc111610 con 0x7feffc107060 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7ff001907640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feffc111490 con 0x7feffc101650 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7ff001907640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feffc111790 con 0x7feffc1079e0 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7feffb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 0x7feffc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7feffb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 0x7feffc06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:55094/0 (socket says 172.21.15.60:55094) 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7feffb7fe640 1 -- 172.21.15.60:0/1331992776 learned_addr learned my addr 172.21.15.60:0/1331992776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7feffa7fc640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 0x7feffc074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.655+0000 7feffaffd640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc101650 0x7feffc0741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 -- 172.21.15.60:0/1331992776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 msgr2=0x7feffc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 0x7feffc06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 -- 172.21.15.60:0/1331992776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc101650 msgr2=0x7feffc0741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc101650 0x7feffc0741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 -- 172.21.15.60:0/1331992776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc06de60 con 0x7feffc107060 2024-07-24T22:01:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.656+0000 7feffa7fc640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 0x7feffc074710 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feff000c970 tx=0x7feff000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.657+0000 7ff000905640 1 -- 172.21.15.60:0/1331992776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0007bb0 con 0x7feffc107060 2024-07-24T22:01:51.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.657+0000 7ff000905640 1 -- 172.21.15.60:0/1331992776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff0007d50 con 0x7feffc107060 2024-07-24T22:01:51.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.657+0000 7ff000905640 1 -- 172.21.15.60:0/1331992776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0004e40 con 0x7feffc107060 2024-07-24T22:01:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.657+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc06e150 con 0x7feffc107060 2024-07-24T22:01:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.657+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc0730c0 con 0x7feffc107060 2024-07-24T22:01:51.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.664+0000 7ff000905640 1 -- 172.21.15.60:0/1331992776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7feff00046c0 con 0x7feffc107060 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 Processor -- start 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 -- start start 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b9fd0 0x5628c02c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c02c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5628c0230990 con 0x5628c02b9fd0 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5628c021fe90 con 0x5628c02b96b0 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.669+0000 7ff003cb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628c0223ed0 con 0x5628c02c5f60 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.670+0000 7feffa7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c02c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.670+0000 7feffb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.670+0000 7feffb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:58084/0 (socket says 172.21.15.60:58084) 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.670+0000 7feffb7fe640 1 -- 172.21.15.60:0/1820510792 learned_addr learned my addr 172.21.15.60:0/1820510792 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 -- 172.21.15.60:0/1820510792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 msgr2=0x5628c02c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c02c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 -- 172.21.15.60:0/1820510792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b9fd0 msgr2=0x5628c02c5a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b9fd0 0x5628c02c5a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 -- 172.21.15.60:0/1820510792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c01a9300 con 0x5628c02b96b0 2024-07-24T22:01:51.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.671+0000 7feffb7fe640 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fefec0094c0 tx=0x7fefec0094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af18967232374c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.672+0000 7ff000905640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 0x7fefe40b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.672+0000 7ff000905640 1 -- 172.21.15.60:0/1331992776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7feff00e3a40 con 0x7feffc107060 2024-07-24T22:01:51.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.673+0000 7fefe1ffb640 1 -- 172.21.15.60:0/1820510792 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec037040 con 0x5628c02b96b0 2024-07-24T22:01:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.673+0000 7fefe1ffb640 1 -- 172.21.15.60:0/1820510792 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec0304f0 con 0x5628c02b96b0 2024-07-24T22:01:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.673+0000 7feffaffd640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 0x7fefe40b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.673+0000 7fefe1ffb640 1 -- 172.21.15.60:0/1820510792 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec0054b0 con 0x5628c02b96b0 2024-07-24T22:01:51.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.674+0000 7ff003cb8740 1 -- 172.21.15.60:0/1820510792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 msgr2=0x5628c02b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.674+0000 7ff003cb8740 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fefec0094c0 tx=0x7fefec0094f0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 -- 172.21.15.60:0/1820510792 shutdown_connections 2024-07-24T22:01:51.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c02c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b9fd0 0x5628c02c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 --2- 172.21.15.60:0/1820510792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b96b0 0x5628c02b9a90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 -- 172.21.15.60:0/1820510792 >> 172.21.15.60:0/1820510792 conn(0x5628c02b74e0 msgr2=0x5628c02b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7feffaffd640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 0x7fefe40b6410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5628c02b83c0 tx=0x7feff4009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.675+0000 7ff003cb8740 1 -- 172.21.15.60:0/1820510792 shutdown_connections 2024-07-24T22:01:51.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.676+0000 7ff003cb8740 1 -- 172.21.15.60:0/1820510792 wait complete. 2024-07-24T22:01:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.676+0000 7ff003cb8740 1 -- start start 2024-07-24T22:01:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.677+0000 7ff003cb8740 1 Processor -- start 2024-07-24T22:01:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.677+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.677+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02c2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48990/0 (socket says 172.21.15.60:48990) 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7feffaffd640 1 -- 172.21.15.60:0/2131682247 learned_addr learned my addr 172.21.15.60:0/2131682247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7feffb7fe640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7feffa7fc640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5628c0083700 con 0x5628c02b96b0 2024-07-24T22:01:51.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5628c01c25d0 con 0x5628c02c5f60 2024-07-24T22:01:51.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.678+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628c02c8bd0 con 0x5628c02b9fd0 2024-07-24T22:01:51.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7ff003cb8740 10 client.0 _ll_register_callbacks cb 0x5628c01e2940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffb7fe640 1 -- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 msgr2=0x5628c02c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffb7fe640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffb7fe640 1 -- 172.21.15.60:0/2131682247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 msgr2=0x5628c02c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffb7fe640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffb7fe640 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c01b5660 con 0x5628c02c5f60 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.679+0000 7feffa7fc640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.680+0000 7feffb7fe640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c3970 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefec004650 tx=0x7fefec005150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671f0a523142a3b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.680+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefec0304f0 con 0x5628c02c5f60 2024-07-24T22:01:51.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.684+0000 7fefdaffd640 10 client.28040 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:01:51.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.684+0000 7fefdaffd640 1 -- 172.21.15.60:0/2131682247 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec037030 con 0x5628c02c5f60 2024-07-24T22:01:51.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.684+0000 7fefdaffd640 1 -- 172.21.15.60:0/2131682247 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7fefec0046c0 con 0x5628c02c5f60 2024-07-24T22:01:51.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.684+0000 7fefdaffd640 1 -- 172.21.15.60:0/2131682247 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fefec04a070 con 0x5628c02c5f60 2024-07-24T22:01:51.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.686+0000 7ff003cb8740 10 client.28040 fetch_fsmap learned FSMap version 46 2024-07-24T22:01:51.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.686+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=46}) -- 0x5628bff80650 con 0x5628c02c5f60 2024-07-24T22:01:51.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.687+0000 7fefdaffd640 1 -- 172.21.15.60:0/2131682247 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fefec0048c0 con 0x5628c02c5f60 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 10 client.28040 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 10 client.28040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 20 client.28040 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c02d57d0 0x5628c02d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 4 client.28040 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:01:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.688+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c02328e0 con 0x5628c02d57d0 2024-07-24T22:01:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.689+0000 7feffaffd640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c02d57d0 0x5628c02d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.689+0000 7fefdaffd640 10 client.28040 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.689+0000 7feffaffd640 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c02d57d0 0x5628c02d7bc0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee852e8f198f98c server_cookie=acfca085472d3413 in_seq=0 out_seq=0 2024-07-24T22:01:51.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: pgmap v851: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-07-24T22:01:51.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: osdmap e107: 12 total, 12 up, 12 in 2024-07-24T22:01:51.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6832/3168301857,v1:172.21.15.123:6833/3168301857] up:boot 2024-07-24T22:01:51.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T22:01:51.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T22:01:51.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:51 smithi060 ceph-mon[33000]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T22:01:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7fefdaffd640 1 -- 172.21.15.60:0/2131682247 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628c02328e0 con 0x5628c02d57d0 2024-07-24T22:01:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7fefdaffd640 10 client.28040 handle_command_reply: tid=0 2024-07-24T22:01:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 client.28040 shutdown 2024-07-24T22:01:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 msgr2=0x5628c02c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c3970 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefec004650 tx=0x7fefec005150 comp rx=0 tx=0).stop 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 shutdown_connections 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c02d57d0 0x5628c02d7bc0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c3970 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7fefdaffd640 0 client.28040 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 --2- 172.21.15.60:0/2131682247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 >> 172.21.15.60:0/2131682247 conn(0x5628c02b74e0 msgr2=0x5628c02bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.786+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 shutdown_connections 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.787+0000 7ff003cb8740 1 -- 172.21.15.60:0/2131682247 wait complete. 2024-07-24T22:01:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.787+0000 7ff003cb8740 20 client.28040 trim_cache size 0 max 16384 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 Processor -- start 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 -- start start 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8011220 con 0x5628c02b96b0 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefe800ea30 con 0x5628c02c5f60 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.823+0000 7ff003cb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefe800c880 con 0x5628c02b9fd0 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.824+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.824+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48994/0 (socket says 172.21.15.60:48994) 2024-07-24T22:01:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.824+0000 7feffaffd640 1 -- 172.21.15.60:0/3281860693 learned_addr learned my addr 172.21.15.60:0/3281860693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 -- 172.21.15.60:0/3281860693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 msgr2=0x5628c02dd6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02dd6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 -- 172.21.15.60:0/3281860693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 msgr2=0x5628c02c0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffb7fe640 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 -- 172.21.15.60:0/3281860693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c02b7310 con 0x5628c02b96b0 2024-07-24T22:01:51.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.825+0000 7feffaffd640 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feff4005fd0 tx=0x7feff4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80680b005be0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.828+0000 7fefe17fa640 1 -- 172.21.15.60:0/3281860693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff401d310 con 0x5628c02b96b0 2024-07-24T22:01:51.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.829+0000 7fefe17fa640 1 -- 172.21.15.60:0/3281860693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff401d6d0 con 0x5628c02b96b0 2024-07-24T22:01:51.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.829+0000 7fefe17fa640 1 -- 172.21.15.60:0/3281860693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4028680 con 0x5628c02b96b0 2024-07-24T22:01:51.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.830+0000 7ff003cb8740 1 -- 172.21.15.60:0/3281860693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 msgr2=0x5628c02dd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.830+0000 7ff003cb8740 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feff4005fd0 tx=0x7feff4014000 comp rx=0 tx=0).stop 2024-07-24T22:01:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 -- 172.21.15.60:0/3281860693 shutdown_connections 2024-07-24T22:01:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02c5f60 0x5628c02c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02b9fd0 0x5628c02dd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 --2- 172.21.15.60:0/3281860693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02dd190 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 -- 172.21.15.60:0/3281860693 >> 172.21.15.60:0/3281860693 conn(0x5628c02b74e0 msgr2=0x5628c02c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 -- 172.21.15.60:0/3281860693 shutdown_connections 2024-07-24T22:01:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.833+0000 7ff003cb8740 1 -- 172.21.15.60:0/3281860693 wait complete. 2024-07-24T22:01:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.834+0000 7ff003cb8740 1 -- start start 2024-07-24T22:01:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.834+0000 7ff003cb8740 1 Processor -- start 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b9fd0 0x5628c02cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c0347970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fefe800cef0 con 0x5628c02b96b0 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefe800e360 con 0x5628c02b9fd0 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fefe800d5c0 con 0x5628c02c5f60 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.835+0000 7ff003cb8740 10 client.0 _ll_register_callbacks cb 0x7feff00e93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49006/0 (socket says 172.21.15.60:49006) 2024-07-24T22:01:51.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 -- 172.21.15.60:0/4272823828 learned_addr learned my addr 172.21.15.60:0/4272823828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:01:51.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 -- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 msgr2=0x5628c0347970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c0347970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 -- 172.21.15.60:0/4272823828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b9fd0 msgr2=0x5628c02cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b9fd0 0x5628c02cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.836+0000 7feffaffd640 1 -- 172.21.15.60:0/4272823828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c02b7ee0 con 0x5628c02b96b0 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.837+0000 7feffaffd640 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feff4009d10 tx=0x7feff4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb311b991d55089c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.837+0000 7fefdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.838+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff401f2d0 con 0x5628c02b96b0 2024-07-24T22:01:51.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.838+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff401d6d0 con 0x5628c02b96b0 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.838+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7feff401f470 con 0x5628c02b96b0 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.839+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7feff4064050 con 0x5628c02b96b0 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.839+0000 7ff003cb8740 10 client.18657 fetch_fsmap learned FSMap version 46 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.839+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5628c0339390 con 0x5628c02b96b0 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff401d6d0 con 0x5628c02b96b0 2024-07-24T22:01:51.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7feff4067020 con 0x5628c02b96b0 2024-07-24T22:01:51.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 10 client.18657 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:01:51.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 10 client.18657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 20 client.18657 populate_metadata read hostname 'smithi060' 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c0339b50 0x5628c033bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 4 client.18657 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5628c022e380 con 0x5628c0339b50 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.840+0000 7feffa7fc640 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c0339b50 0x5628c033bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.842+0000 7feffa7fc640 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c0339b50 0x5628c033bf70 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236aec8373f05024 server_cookie=c2a4d3489bc6849a in_seq=0 out_seq=0 2024-07-24T22:01:51.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.842+0000 7fefdaffd640 10 client.18657 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:51.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.842+0000 7fefdaffd640 1 -- 172.21.15.60:0/4272823828 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5628c022e380 con 0x5628c0339b50 2024-07-24T22:01:51.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.842+0000 7fefdaffd640 10 client.18657 handle_command_reply: tid=0 2024-07-24T22:01:51.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.842+0000 7ff003cb8740 1 client.18657 shutdown 2024-07-24T22:01:51.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 msgr2=0x5628c02ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feff4009d10 tx=0x7feff4004b40 comp rx=0 tx=0).stop 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 shutdown_connections 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5628c0339b50 0x5628c033bf70 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628c02c5f60 0x5628c0347970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5628c02b9fd0 0x5628c02cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 --2- 172.21.15.60:0/4272823828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628c02b96b0 0x5628c02ca620 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 >> 172.21.15.60:0/4272823828 conn(0x5628c02b74e0 msgr2=0x5628c02bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.843+0000 7fefdaffd640 0 client.18657 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:01:51.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.844+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 shutdown_connections 2024-07-24T22:01:51.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.844+0000 7ff003cb8740 1 -- 172.21.15.60:0/4272823828 wait complete. 2024-07-24T22:01:51.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.844+0000 7ff003cb8740 20 client.18657 trim_cache size 0 max 16384 2024-07-24T22:01:51.846 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:01:51.846 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (17688 inodes in the stack)", 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: "54f5cd12-22db-4b60-97f1-94128537550b": { 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "54f5cd12-22db-4b60-97f1-94128537550b", 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:01:51.847 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 msgr2=0x7fefe40b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 0x7fefe40b6410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5628c02b83c0 tx=0x7feff4009680 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 msgr2=0x7feffc074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 0x7feffc074710 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feff000c970 tx=0x7feff000ce30 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 shutdown_connections 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fefe40b3f50 0x7fefe40b6410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feffc1079e0 0x7feffc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.845+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc107060 0x7feffc074710 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.846+0000 7ff001907640 1 --2- 172.21.15.60:0/1331992776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc101650 0x7feffc0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:01:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.846+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 >> 172.21.15.60:0/1331992776 conn(0x7feffc0fd0c0 msgr2=0x7feffc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:01:51.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.846+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 shutdown_connections 2024-07-24T22:01:51.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:01:51.846+0000 7ff001907640 1 -- 172.21.15.60:0/1331992776 wait complete. 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: pgmap v851: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: osdmap e107: 12 total, 12 up, 12 in 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6832/3168301857,v1:172.21.15.123:6833/3168301857] up:boot 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T22:01:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:51 smithi123 ceph-mon[37143]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T22:01:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: pgmap v851: 97 pgs: 23 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 63 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-07-24T22:01:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: osdmap e107: 12 total, 12 up, 12 in 2024-07-24T22:01:52.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6832/3168301857,v1:172.21.15.123:6833/3168301857] up:boot 2024-07-24T22:01:52.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-24T22:01:52.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-24T22:01:52.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:51 smithi086 ceph-mon[37128]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-24T22:01:52.670 INFO:tasks.cephfs.filesystem:scrub status for tag:54f5cd12-22db-4b60-97f1-94128537550b - {'path': '/', 'tag': '54f5cd12-22db-4b60-97f1-94128537550b', 'options': 'recursive,force'} 2024-07-24T22:01:52.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:52 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:01:52.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:52 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:52 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:01:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:52 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:53.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:52 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:01:53.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:52 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:01:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:53 smithi123 ceph-mon[37143]: pgmap v853: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 293 op/s 2024-07-24T22:01:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:53 smithi086 ceph-mon[37128]: pgmap v853: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 293 op/s 2024-07-24T22:01:54.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:53 smithi060 ceph-mon[33000]: pgmap v853: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 293 op/s 2024-07-24T22:01:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:55 smithi123 ceph-mon[37143]: pgmap v854: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 470 op/s 2024-07-24T22:01:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:55 smithi086 ceph-mon[37128]: pgmap v854: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 470 op/s 2024-07-24T22:01:56.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:55 smithi060 ceph-mon[33000]: pgmap v854: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 470 op/s 2024-07-24T22:01:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:57 smithi086 ceph-mon[37128]: pgmap v855: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-07-24T22:01:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:57 smithi060 ceph-mon[33000]: pgmap v855: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-07-24T22:01:58.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:57 smithi123 ceph-mon[37143]: pgmap v855: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-07-24T22:02:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:01:59 smithi086 ceph-mon[37128]: pgmap v856: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2024-07-24T22:02:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:01:59 smithi060 ceph-mon[33000]: pgmap v856: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2024-07-24T22:02:00.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:01:59 smithi123 ceph-mon[37143]: pgmap v856: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2024-07-24T22:02:01.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.051 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.053 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.057 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.059 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.060 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.064 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.065 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.067 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.068 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.071 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:00 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:01.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:00 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:01.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:00 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: pgmap v857: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 847 op/s 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.049 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:01 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: pgmap v857: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 847 op/s 2024-07-24T22:02:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:01 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: pgmap v857: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 847 op/s 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:01 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 -- 172.21.15.60:0/129220418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0107910 msgr2=0x7f91d010e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 --2- 172.21.15.60:0/129220418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0107910 0x7f91d010e110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ca0 tx=0x7f91c802a8c0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 -- 172.21.15.60:0/129220418 shutdown_connections 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 --2- 172.21.15.60:0/129220418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0107910 0x7f91d010e110 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 --2- 172.21.15.60:0/129220418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d0106f90 0x7f91d01073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 --2- 172.21.15.60:0/129220418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d01015e0 0x7f91d01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 -- 172.21.15.60:0/129220418 >> 172.21.15.60:0/129220418 conn(0x7f91d00782e0 msgr2=0x7f91d00ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 -- 172.21.15.60:0/129220418 shutdown_connections 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.537+0000 7f91d5379640 1 -- 172.21.15.60:0/129220418 wait complete. 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 Processor -- start 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 -- start start 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 0x7f91d01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d0107910 0x7f91d019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0111540 con 0x7f91d0107910 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01113c0 con 0x7f91d0106f90 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91ce7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 0x7f91d01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:59368/0 (socket says 172.21.15.60:59368) 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91d5379640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01116c0 con 0x7f91d01015e0 2024-07-24T22:02:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91ceffd640 1 -- 172.21.15.60:0/2192610130 learned_addr learned my addr 172.21.15.60:0/2192610130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.538+0000 7f91cf7fe640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d0107910 0x7f91d019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 -- 172.21.15.60:0/2192610130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 msgr2=0x7f91d0198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 -- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d0107910 msgr2=0x7f91d019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d0107910 0x7f91d019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 -- 172.21.15.60:0/2192610130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d019ec50 con 0x7f91d0106f90 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ceffd640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 0x7f91d01990f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e9f0 tx=0x7f91c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91b7fff640 1 -- 172.21.15.60:0/2192610130 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400ccc0 con 0x7f91d0106f90 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91b7fff640 1 -- 172.21.15.60:0/2192610130 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c40108e0 con 0x7f91d0106f90 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d019ef40 con 0x7f91d0106f90 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91b7fff640 1 -- 172.21.15.60:0/2192610130 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4018880 con 0x7f91d0106f90 2024-07-24T22:02:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.539+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a5600 con 0x7f91d0106f90 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 Processor -- start 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 -- start start 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91b7fff640 1 -- 172.21.15.60:0/2192610130 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f91c4010430 con 0x7f91d0106f90 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9e8c0 0x555c56b9eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56bab170 0x555c56bad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555c56b14280 con 0x555c56bab170 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c56b03780 con 0x555c56b9f1e0 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91d81ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c56b077c0 con 0x555c56b9e8c0 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56bab170 0x555c56bad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9e8c0 0x555c56b9eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36440/0 (socket says 172.21.15.60:36440) 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.543+0000 7f91ceffd640 1 -- 172.21.15.60:0/812481034 learned_addr learned my addr 172.21.15.60:0/812481034 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 -- 172.21.15.60:0/812481034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9e8c0 msgr2=0x555c56b9eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9e8c0 0x555c56b9eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 -- 172.21.15.60:0/812481034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56bab170 msgr2=0x555c56bad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56bab170 0x555c56bad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 -- 172.21.15.60:0/812481034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c56867790 con 0x555c56b9f1e0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91ceffd640 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f91c0011650 tx=0x7f91c0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96bc820c090c21a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91b57fa640 1 -- 172.21.15.60:0/812481034 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c001e070 con 0x555c56b9f1e0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91b7fff640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 0x7f91b80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91b57fa640 1 -- 172.21.15.60:0/812481034 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c000fb70 con 0x555c56b9f1e0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91cf7fe640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 0x7f91b80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91b57fa640 1 -- 172.21.15.60:0/812481034 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0019c60 con 0x555c56b9f1e0 2024-07-24T22:02:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.544+0000 7f91b7fff640 1 -- 172.21.15.60:0/2192610130 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7f91c40e43b0 con 0x7f91d0106f90 2024-07-24T22:02:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- 172.21.15.60:0/812481034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 msgr2=0x555c56baac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91cf7fe640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 0x7f91b80b6200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c80099a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f91c0011650 tx=0x7f91c0009f90 comp rx=0 tx=0).stop 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- 172.21.15.60:0/812481034 shutdown_connections 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56bab170 0x555c56bad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56baac30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 --2- 172.21.15.60:0/812481034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9e8c0 0x555c56b9eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- 172.21.15.60:0/812481034 >> 172.21.15.60:0/812481034 conn(0x555c56b9c6f0 msgr2=0x555c56b9d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- 172.21.15.60:0/812481034 shutdown_connections 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- 172.21.15.60:0/812481034 wait complete. 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 -- start start 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.545+0000 7f91d81ab740 1 Processor -- start 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56ba82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50430/0 (socket says 172.21.15.60:50430) 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ceffd640 1 -- 172.21.15.60:0/1540429596 learned_addr learned my addr 172.21.15.60:0/1540429596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555c56a3dab0 con 0x555c56b9e8c0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c56a8b9e0 con 0x555c56bab170 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c56badea0 con 0x555c56b9f1e0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91cf7fe640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56ba82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91d81ab740 10 client.0 _ll_register_callbacks cb 0x555c56813a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 -- 172.21.15.60:0/1540429596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 msgr2=0x555c56ba82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56ba82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 -- 172.21.15.60:0/1540429596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 msgr2=0x555c56baa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.546+0000 7f91ce7fc640 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c568e0620 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba7da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91c40e1ad0 tx=0x7f91c400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2346a36ad80b63c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40aa7b0 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c56ab2ce0 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7f91c40aa2a0 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f91c4010720 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 10 client.27943 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555c56bb9660 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555c56ab2ce0 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f91c40a69b0 con 0x555c56b9f1e0 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 10 client.27943 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 10 client.27943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 20 client.27943 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.547+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56bbab20 0x555c56bbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.548+0000 7f91d81ab740 4 client.27943 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.548+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c56b161d0 con 0x555c56bbab20 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.548+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56bbab20 0x555c56bbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.548+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56bbab20 0x555c56bbcf70 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9651e053599f764 server_cookie=eb289ba341652977 in_seq=0 out_seq=0 2024-07-24T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.548+0000 7f91b2ff5640 10 client.27943 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1540429596 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c56b161d0 con 0x555c56bbab20 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91b2ff5640 10 client.27943 handle_command_reply: tid=0 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 client.27943 shutdown 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 msgr2=0x555c56ba7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba7da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91c40e1ad0 tx=0x7f91c400ce60 comp rx=0 tx=0).stop 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 shutdown_connections 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56bbab20 0x555c56bbcf70 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56ba82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91b2ff5640 0 client.27943 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba7da0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1540429596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56baa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 >> 172.21.15.60:0/1540429596 conn(0x555c56b9c6f0 msgr2=0x555c56ba2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 shutdown_connections 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 1 -- 172.21.15.60:0/1540429596 wait complete. 2024-07-24T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.551+0000 7f91d81ab740 20 client.27943 trim_cache size 0 max 16384 2024-07-24T22:02:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.567+0000 7f91d81ab740 1 Processor -- start 2024-07-24T22:02:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.567+0000 7f91d81ab740 1 -- start start 2024-07-24T22:02:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56bc2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01ae900 con 0x555c56b9e8c0 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01ac110 con 0x555c56bab170 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91d81ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01a9f60 con 0x555c56b9f1e0 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50442/0 (socket says 172.21.15.60:50442) 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56bc2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 -- 172.21.15.60:0/3067517850 learned_addr learned my addr 172.21.15.60:0/3067517850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 -- 172.21.15.60:0/3067517850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 msgr2=0x555c56ba6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 -- 172.21.15.60:0/3067517850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 msgr2=0x555c56bc2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56bc2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 -- 172.21.15.60:0/3067517850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c56b9d160 con 0x555c56b9e8c0 2024-07-24T22:02:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91ceffd640 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f60 tx=0x7f91c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e051e010bd947b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91b4ff9640 1 -- 172.21.15.60:0/3067517850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c001e040 con 0x555c56b9e8c0 2024-07-24T22:02:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91b4ff9640 1 -- 172.21.15.60:0/3067517850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0019e20 con 0x555c56b9e8c0 2024-07-24T22:02:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.568+0000 7f91b4ff9640 1 -- 172.21.15.60:0/3067517850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000f6c0 con 0x555c56b9e8c0 2024-07-24T22:02:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 -- 172.21.15.60:0/3067517850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 msgr2=0x555c56ba6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f60 tx=0x7f91c0009da0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 -- 172.21.15.60:0/3067517850 shutdown_connections 2024-07-24T22:02:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56bab170 0x555c56bc2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56b9f1e0 0x555c56ba6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 --2- 172.21.15.60:0/3067517850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56ba6600 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 -- 172.21.15.60:0/3067517850 >> 172.21.15.60:0/3067517850 conn(0x555c56b9c6f0 msgr2=0x555c56b9d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 -- 172.21.15.60:0/3067517850 shutdown_connections 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.569+0000 7f91d81ab740 1 -- 172.21.15.60:0/3067517850 wait complete. 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 -- start start 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 Processor -- start 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56bab170 0x555c56bb0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01aa5d0 con 0x555c56b9e8c0 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01aba40 con 0x555c56b9f1e0 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01aaca0 con 0x555c56bab170 2024-07-24T22:02:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91d81ab740 10 client.0 _ll_register_callbacks cb 0x7f91c40e7bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ce7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ce7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:36474/0 (socket says 172.21.15.60:36474) 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50458/0 (socket says 172.21.15.60:50458) 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ce7fc640 1 -- 172.21.15.60:0/1710636685 learned_addr learned my addr 172.21.15.60:0/1710636685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91cf7fe640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56bab170 0x555c56bb0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 -- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56bab170 msgr2=0x555c56bb0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56bab170 0x555c56bb0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 -- 172.21.15.60:0/1710636685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 msgr2=0x555c56bb0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ceffd640 1 -- 172.21.15.60:0/1710636685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c56c23810 con 0x555c56b9e8c0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.570+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91ceffd640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f60 tx=0x7f91c00198a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19d702ac7ebe558 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0032070 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c0019e20 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8565+0+0 (secure 0 0 0) 0x7f91c002a040 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f91c003a050 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 10 client.18675 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555c56c24930 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0019e20 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f91c0035030 con 0x555c56b9e8c0 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 10 client.18675 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 20 client.18675 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56c24e90 0x555c56c272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 4 client.18675 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.571+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c56b11c70 con 0x555c56c24e90 2024-07-24T22:02:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.572+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56c24e90 0x555c56c272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.572+0000 7f91b2ff5640 10 client.18675 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:02.572+0000 7f91ce7fc640 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56c24e90 0x555c56c272b0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877e7c8ae44d77b5 server_cookie=85248e2bb153af30 in_seq=0 out_seq=0 2024-07-24T22:02:02.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.785 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.787 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.788 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.789 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.790 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.793 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.794 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.798 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.799 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.802 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.803 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.804 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.805 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.806 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.809 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.810 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.811 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.812 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.814 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.818 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.821 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.822 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.823 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.824 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.825 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.828 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.829 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.830 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.831 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.834 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.837 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.838 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.842 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.843 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.845 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.846 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.850 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:02.851 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:02.852 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:02 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.061+0000 7f91b2ff5640 1 -- 172.21.15.60:0/1710636685 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c56b11c70 con 0x555c56c24e90 2024-07-24T22:02:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.061+0000 7f91b2ff5640 10 client.18675 handle_command_reply: tid=0 2024-07-24T22:02:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.061+0000 7f91d81ab740 1 client.18675 shutdown 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.061+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 msgr2=0x555c56bb3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.061+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f60 tx=0x7f91c00198a0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 shutdown_connections 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555c56c24e90 0x555c56c272b0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91b2ff5640 0 client.18675 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c56bab170 0x555c56bb0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c56b9f1e0 0x555c56bb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 --2- 172.21.15.60:0/1710636685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555c56b9e8c0 0x555c56bb3880 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 >> 172.21.15.60:0/1710636685 conn(0x555c56b9c6f0 msgr2=0x555c56b9d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 shutdown_connections 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.062+0000 7f91d81ab740 1 -- 172.21.15.60:0/1710636685 wait complete. 2024-07-24T22:02:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.063+0000 7f91d81ab740 20 client.18675 trim_cache size 0 max 16384 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.063+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 msgr2=0x7f91b80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.064+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 0x7f91b80b6200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c80099a0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.064+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 msgr2=0x7f91d01990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.064+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 0x7f91d01990f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e9f0 tx=0x7f91c400eeb0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 shutdown_connections 2024-07-24T22:02:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b80b3d40 0x7f91b80b6200 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91d0107910 0x7f91d019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d0106f90 0x7f91d01990f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 --2- 172.21.15.60:0/2192610130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91d01015e0 0x7f91d0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 >> 172.21.15.60:0/2192610130 conn(0x7f91d00782e0 msgr2=0x7f91d01058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 shutdown_connections 2024-07-24T22:02:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:03.065+0000 7f91d5379640 1 -- 172.21.15.60:0/2192610130 wait complete. 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:02 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:03.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:03.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:02 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: pgmap v858: 97 pgs: 97 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 1.07k op/s 2024-07-24T22:02:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: osdmap e108: 12 total, 12 up, 12 in 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:04.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:03 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: pgmap v858: 97 pgs: 97 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 1.07k op/s 2024-07-24T22:02:04.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: osdmap e108: 12 total, 12 up, 12 in 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:04.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:03 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: pgmap v858: 97 pgs: 97 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 1.07k op/s 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.300 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.301 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.302 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.303 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.304 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.305 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.306 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.309 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.310 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.311 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.312 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.313 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: osdmap e108: 12 total, 12 up, 12 in 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:04.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:04.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:03 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.074 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.077 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.080 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.081 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.095 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.096 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.102 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.103 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.110 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.111 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.128 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.132 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.135 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.136 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:04 smithi060 ceph-mon[33000]: osdmap e109: 12 total, 12 up, 12 in 2024-07-24T22:02:05.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.246 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.247 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.248 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.249 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.250 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.251 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.253 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.254 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.255 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.256 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.257 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.258 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.259 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.260 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.261 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.262 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.263 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.264 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.265 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.266 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.267 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.268 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.269 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.270 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.274 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.275 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.276 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.277 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.278 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.279 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.280 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.283 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.284 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.285 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.286 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.287 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.288 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.290 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.291 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.292 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.293 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.294 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.295 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.296 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.297 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.301 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.302 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.303 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.304 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.308 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.309 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.310 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:05.311 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.312 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.312 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:04 smithi123 ceph-mon[37143]: osdmap e109: 12 total, 12 up, 12 in 2024-07-24T22:02:05.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.314 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.315 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.316 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.317 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.318 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.319 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.320 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.321 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.322 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.323 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.324 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.325 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.326 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.327 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.328 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.330 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.331 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.332 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.334 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.337 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.339 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.340 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.341 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.342 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.343 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.347 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.348 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.349 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.351 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.352 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.353 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.354 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.355 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.356 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.357 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.361 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.365 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.366 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.367 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.368 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.370 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.371 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.372 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.374 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.376 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:05.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:04 smithi086 ceph-mon[37128]: osdmap e109: 12 total, 12 up, 12 in 2024-07-24T22:02:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:05 smithi060 ceph-mon[33000]: pgmap v861: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 1.65k op/s 2024-07-24T22:02:06.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:05 smithi123 ceph-mon[37143]: pgmap v861: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 1.65k op/s 2024-07-24T22:02:06.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:05 smithi086 ceph-mon[37128]: pgmap v861: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 1.65k op/s 2024-07-24T22:02:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:07 smithi060 ceph-mon[33000]: pgmap v862: 97 pgs: 17 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.00k op/s 2024-07-24T22:02:08.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:07 smithi123 ceph-mon[37143]: pgmap v862: 97 pgs: 17 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.00k op/s 2024-07-24T22:02:08.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:07 smithi086 ceph-mon[37128]: pgmap v862: 97 pgs: 17 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 72 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 2.00k op/s 2024-07-24T22:02:10.097 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:09 smithi086 ceph-mon[37128]: pgmap v863: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.36k op/s 2024-07-24T22:02:10.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:09 smithi123 ceph-mon[37143]: pgmap v863: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.36k op/s 2024-07-24T22:02:10.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:09 smithi060 ceph-mon[33000]: pgmap v863: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.36k op/s 2024-07-24T22:02:12.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:11 smithi123 ceph-mon[37143]: pgmap v864: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 1.74k op/s 2024-07-24T22:02:12.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:11 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:02:12.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:11 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:02:12.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:11 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:02:12.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:11 smithi086 ceph-mon[37128]: pgmap v864: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 1.74k op/s 2024-07-24T22:02:12.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:11 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:02:12.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:11 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:02:12.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:11 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:02:12.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:11 smithi060 ceph-mon[33000]: pgmap v864: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 1.74k op/s 2024-07-24T22:02:12.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:11 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:02:12.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:11 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:02:12.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:11 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:02:13.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:02:13.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:02:13.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:02:13.244 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:12 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:02:13.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:02:13.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:02:13.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:02:13.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:12 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:02:13.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:12 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:02:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:13 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T22:02:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:13 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T22:02:14.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:13 smithi123 ceph-mon[37143]: pgmap v865: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-07-24T22:02:14.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:13 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T22:02:14.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:13 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T22:02:14.296 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:13 smithi086 ceph-mon[37128]: pgmap v865: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-07-24T22:02:14.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:13 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi060 to 177.1M 2024-07-24T22:02:14.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:13 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi060 to 185704960: error parsing value: Value '185704960' is below minimum 939524096 2024-07-24T22:02:14.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:13 smithi060 ceph-mon[33000]: pgmap v865: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.195+0000 7f9f2ff83640 1 -- 172.21.15.60:0/1388697599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109f20 msgr2=0x7f9f2810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/1388697599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109f20 0x7f9f2810a360 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f1802a410 comp rx=0 tx=0).stop 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 -- 172.21.15.60:0/1388697599 shutdown_connections 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/1388697599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2811ba90 0x7f9f2811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/1388697599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109f20 0x7f9f2810a360 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/1388697599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f28109b40 0x7f9f28110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 -- 172.21.15.60:0/1388697599 >> 172.21.15.60:0/1388697599 conn(0x7f9f2806d760 msgr2=0x7f9f2806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 -- 172.21.15.60:0/1388697599 shutdown_connections 2024-07-24T22:02:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 -- 172.21.15.60:0/1388697599 wait complete. 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 Processor -- start 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.196+0000 7f9f2ff83640 1 -- start start 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28109f20 0x7f9f281a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2811ba90 0x7f9f281ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281214f0 con 0x7f9f28109b40 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28121370 con 0x7f9f28109f20 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2ff83640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28121670 con 0x7f9f2811ba90 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44112/0 (socket says 172.21.15.60:44112) 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 -- 172.21.15.60:0/2984538888 learned_addr learned my addr 172.21.15.60:0/2984538888 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28109f20 0x7f9f281a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2e520640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2811ba90 0x7f9f281ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 -- 172.21.15.60:0/2984538888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2811ba90 msgr2=0x7f9f281ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2811ba90 0x7f9f281ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 -- 172.21.15.60:0/2984538888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28109f20 msgr2=0x7f9f281a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28109f20 0x7f9f281a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 -- 172.21.15.60:0/2984538888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f281adbd0 con 0x7f9f28109b40 2024-07-24T22:02:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.197+0000 7f9f2dd1f640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400b4a0 tx=0x7f9f2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.198+0000 7f9f1effd640 1 -- 172.21.15.60:0/2984538888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24014070 con 0x7f9f28109b40 2024-07-24T22:02:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.198+0000 7f9f1effd640 1 -- 172.21.15.60:0/2984538888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f24010ac0 con 0x7f9f28109b40 2024-07-24T22:02:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.198+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f281adec0 con 0x7f9f28109b40 2024-07-24T22:02:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.198+0000 7f9f1effd640 1 -- 172.21.15.60:0/2984538888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24017c30 con 0x7f9f28109b40 2024-07-24T22:02:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.198+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281b4580 con 0x7f9f28109b40 2024-07-24T22:02:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.200+0000 7f9f32db3740 1 Processor -- start 2024-07-24T22:02:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.200+0000 7f9f32db3740 1 -- start start 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825f1e0 0x55f73826ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f73826d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f7381d4280 con 0x55f73825e8c0 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f7381c3780 con 0x55f73826b170 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f32db3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7381c77c0 con 0x55f73825f1e0 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44122/0 (socket says 172.21.15.60:44122) 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 -- 172.21.15.60:0/3305991878 learned_addr learned my addr 172.21.15.60:0/3305991878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 -- 172.21.15.60:0/3305991878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825f1e0 msgr2=0x55f73826ac30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825f1e0 0x55f73826ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 -- 172.21.15.60:0/3305991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 msgr2=0x55f73826d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f73826d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.201+0000 7f9f2e520640 1 -- 172.21.15.60:0/3305991878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f737f27790 con 0x55f73825e8c0 2024-07-24T22:02:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.202+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009f90 tx=0x7f9f20010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d50b8f767fccfd3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.202+0000 7f9f13fff640 1 -- 172.21.15.60:0/3305991878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20016070 con 0x55f73825e8c0 2024-07-24T22:02:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.202+0000 7f9f1effd640 1 -- 172.21.15.60:0/2984538888 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9f24017070 con 0x7f9f28109b40 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.202+0000 7f9f13fff640 1 -- 172.21.15.60:0/3305991878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20004d20 con 0x55f73825e8c0 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.202+0000 7f9f13fff640 1 -- 172.21.15.60:0/3305991878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200053e0 con 0x55f73825e8c0 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f1effd640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 0x7f9f140b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 -- 172.21.15.60:0/3305991878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 msgr2=0x55f73825eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009f90 tx=0x7f9f20010800 comp rx=0 tx=0).stop 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 -- 172.21.15.60:0/3305991878 shutdown_connections 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f73826d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825f1e0 0x55f73826ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3305991878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73825e8c0 0x55f73825eca0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.203+0000 7f9f32db3740 1 -- 172.21.15.60:0/3305991878 >> 172.21.15.60:0/3305991878 conn(0x55f73825c6f0 msgr2=0x55f73825d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.204+0000 7f9f1effd640 1 -- 172.21.15.60:0/2984538888 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f9f240a6560 con 0x7f9f28109b40 2024-07-24T22:02:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.204+0000 7f9f32db3740 1 -- 172.21.15.60:0/3305991878 shutdown_connections 2024-07-24T22:02:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.206+0000 7f9f2dd1f640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 0x7f9f140b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.206+0000 7f9f32db3740 1 -- 172.21.15.60:0/3305991878 wait complete. 2024-07-24T22:02:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.207+0000 7f9f32db3740 1 -- start start 2024-07-24T22:02:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.207+0000 7f9f32db3740 1 Processor -- start 2024-07-24T22:02:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73826b170 0x55f738267d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7382682b0 0x55f738268730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f7380fdab0 con 0x55f73826b170 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49956/0 (socket says 172.21.15.60:49956) 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f2e520640 1 -- 172.21.15.60:0/3774461548 learned_addr learned my addr 172.21.15.60:0/3774461548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f73814b9e0 con 0x55f73825e8c0 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.208+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f73826de50 con 0x55f7382682b0 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.209+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7382682b0 0x55f738268730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.209+0000 7f9f32db3740 10 client.0 _ll_register_callbacks cb 0x55f737ed3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.209+0000 7f9f2e520640 1 -- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7382682b0 msgr2=0x55f738268730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.209+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7382682b0 0x55f738268730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f2e520640 1 -- 172.21.15.60:0/3774461548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73826b170 msgr2=0x55f738267d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73826b170 0x55f738267d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f2e520640 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f737fa0620 con 0x55f73825e8c0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f2dd1f640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 0x7f9f140b6280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f240a45f0 tx=0x7f9f240ae6b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f200127b0 tx=0x7f9f20012f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8dc427765e4d41 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.210+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2002e060 con 0x55f73825e8c0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f738172ce0 con 0x55f73825e8c0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f9f20012ae0 con 0x55f73825e8c0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9f20016040 con 0x55f73825e8c0 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f32db3740 10 client.28067 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f738279a80 con 0x55f73825e8c0 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.211+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f738172ce0 con 0x55f73825e8c0 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9f20028040 con 0x55f73825e8c0 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 10 client.28067 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 10 client.28067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 20 client.28067 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f73827b390 0x55f73827d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 4 client.28067 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.212+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7381d61d0 con 0x55f73827b390 2024-07-24T22:02:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.213+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f73827b390 0x55f73827d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.213+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f73827b390 0x55f73827d7f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76a939f4e5cb80f server_cookie=2c39d356dfdd4cd0 in_seq=0 out_seq=0 2024-07-24T22:02:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.213+0000 7f9f117fa640 10 client.28067 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.214+0000 7f9f117fa640 1 -- 172.21.15.60:0/3774461548 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7381d61d0 con 0x55f73827b390 2024-07-24T22:02:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.214+0000 7f9f117fa640 10 client.28067 handle_command_reply: tid=0 2024-07-24T22:02:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.215+0000 7f9f32db3740 1 client.28067 shutdown 2024-07-24T22:02:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.215+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 msgr2=0x55f73826a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.215+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f200127b0 tx=0x7f9f20012f60 comp rx=0 tx=0).stop 2024-07-24T22:02:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 shutdown_connections 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f73827b390 0x55f73827d7f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7382682b0 0x55f738268730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f73826b170 0x55f738267d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f117fa640 0 client.28067 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3774461548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f73826a930 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 >> 172.21.15.60:0/3774461548 conn(0x55f73825c6f0 msgr2=0x55f738262220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 shutdown_connections 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 1 -- 172.21.15.60:0/3774461548 wait complete. 2024-07-24T22:02:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.216+0000 7f9f32db3740 20 client.28067 trim_cache size 0 max 16384 2024-07-24T22:02:16.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:15 smithi123 ceph-mon[37143]: pgmap v866: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 935 op/s 2024-07-24T22:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.244+0000 7f9f32db3740 1 Processor -- start 2024-07-24T22:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.244+0000 7f9f32db3740 1 -- start start 2024-07-24T22:02:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825e8c0 0x55f7382830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f738283600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f738273130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bfa90 con 0x55f7382682b0 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bd2a0 con 0x55f73826b170 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f32db3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bb110 con 0x55f73825e8c0 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825e8c0 0x55f7382830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f738273130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f2d51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f738283600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f2e520640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f738273130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49966/0 (socket says 172.21.15.60:49966) 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.245+0000 7f9f2e520640 1 -- 172.21.15.60:0/95479906 learned_addr learned my addr 172.21.15.60:0/95479906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 -- 172.21.15.60:0/95479906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825e8c0 msgr2=0x55f7382830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825e8c0 0x55f7382830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 -- 172.21.15.60:0/95479906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 msgr2=0x55f738273130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f738273130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 -- 172.21.15.60:0/95479906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f738269040 con 0x55f7382682b0 2024-07-24T22:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f738283600 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009230 tx=0x7f9f18031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e252d660cfa913ae server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f137fe640 1 -- 172.21.15.60:0/95479906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1804a070 con 0x55f7382682b0 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f137fe640 1 -- 172.21.15.60:0/95479906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f18005040 con 0x55f7382682b0 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.246+0000 7f9f137fe640 1 -- 172.21.15.60:0/95479906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18035040 con 0x55f7382682b0 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- 172.21.15.60:0/95479906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 msgr2=0x55f738283600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f738283600 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009230 tx=0x7f9f18031400 comp rx=0 tx=0).stop 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- 172.21.15.60:0/95479906 shutdown_connections 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73826b170 0x55f738273130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f738283600 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 --2- 172.21.15.60:0/95479906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73825e8c0 0x55f7382830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- 172.21.15.60:0/95479906 >> 172.21.15.60:0/95479906 conn(0x55f73825c6f0 msgr2=0x55f7382665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- 172.21.15.60:0/95479906 shutdown_connections 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- 172.21.15.60:0/95479906 wait complete. 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.247+0000 7f9f32db3740 1 -- start start 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 Processor -- start 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f7382737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73826b170 0x55f7382701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bb760 con 0x55f7382682b0 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bcbd0 con 0x55f73825e8c0 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281bbe30 con 0x55f73826b170 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f32db3740 10 client.0 _ll_register_callbacks cb 0x7f9f20009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f2d51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f2d51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44150/0 (socket says 172.21.15.60:44150) 2024-07-24T22:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f2d51e640 1 -- 172.21.15.60:0/3489813064 learned_addr learned my addr 172.21.15.60:0/3489813064 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f7382737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.248+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73826b170 0x55f7382701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 -- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73826b170 msgr2=0x55f7382701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73826b170 0x55f7382701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 -- 172.21.15.60:0/3489813064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 msgr2=0x55f7382737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f7382737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 -- 172.21.15.60:0/3489813064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7382709c0 con 0x55f7382682b0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f2d51e640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009850 tx=0x7f9f18032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9cf11f28c132a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18035070 con 0x55f7382682b0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f9f18004350 con 0x55f7382682b0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f18005040 con 0x55f7382682b0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9f1804a070 con 0x55f7382682b0 2024-07-24T22:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f32db3740 10 client.18690 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.249+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f7382e50d0 con 0x55f7382682b0 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f18005040 con 0x55f7382682b0 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9f180035f0 con 0x55f7382682b0 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 10 client.18690 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 20 client.18690 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f7382e58b0 0x55f7382e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 4 client.18690 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7381d1c70 con 0x55f7382e58b0 2024-07-24T22:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f7382e58b0 0x55f7382e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f2e520640 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f7382e58b0 0x55f7382e7cd0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1339962806a7084 server_cookie=cc561a78a072b26b in_seq=0 out_seq=0 2024-07-24T22:02:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:16.250+0000 7f9f117fa640 10 client.18690 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:16.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:15 smithi086 ceph-mon[37128]: pgmap v866: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 935 op/s 2024-07-24T22:02:16.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:15 smithi060 ceph-mon[33000]: pgmap v866: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 935 op/s 2024-07-24T22:02:18.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:17 smithi123 ceph-mon[37143]: pgmap v867: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 814 op/s 2024-07-24T22:02:18.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:17 smithi086 ceph-mon[37128]: pgmap v867: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 814 op/s 2024-07-24T22:02:18.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:17 smithi060 ceph-mon[33000]: pgmap v867: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 814 op/s 2024-07-24T22:02:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f117fa640 1 -- 172.21.15.60:0/3489813064 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7381d1c70 con 0x55f7382e58b0 2024-07-24T22:02:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f117fa640 10 client.18690 handle_command_reply: tid=0 2024-07-24T22:02:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 client.18690 shutdown 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 msgr2=0x55f73826fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009850 tx=0x7f9f18032d60 comp rx=0 tx=0).stop 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 shutdown_connections 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f7382e58b0 0x55f7382e7cd0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f73826b170 0x55f7382701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f7382682b0 0x55f73826fc80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f117fa640 0 client.18690 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 --2- 172.21.15.60:0/3489813064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f73825e8c0 0x55f7382737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 >> 172.21.15.60:0/3489813064 conn(0x55f73825c6f0 msgr2=0x55f7382693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 shutdown_connections 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.330+0000 7f9f32db3740 1 -- 172.21.15.60:0/3489813064 wait complete. 2024-07-24T22:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f32db3740 20 client.18690 trim_cache size 0 max 16384 2024-07-24T22:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 msgr2=0x7f9f140b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 0x7f9f140b6280 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f240a45f0 tx=0x7f9f240ae6b0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 msgr2=0x7f9f281a7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400b4a0 tx=0x7f9f2400b960 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 shutdown_connections 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9f140b3dc0 0x7f9f140b6280 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2811ba90 0x7f9f281ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28109f20 0x7f9f281a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 --2- 172.21.15.60:0/2984538888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f28109b40 0x7f9f281a7b30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 >> 172.21.15.60:0/2984538888 conn(0x7f9f2806d760 msgr2=0x7f9f2811c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.331+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 shutdown_connections 2024-07-24T22:02:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:18.332+0000 7f9f2ff83640 1 -- 172.21.15.60:0/2984538888 wait complete. 2024-07-24T22:02:20.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:19 smithi123 ceph-mon[37143]: pgmap v868: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.0 MiB/s wr, 424 op/s 2024-07-24T22:02:20.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:19 smithi086 ceph-mon[37128]: pgmap v868: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.0 MiB/s wr, 424 op/s 2024-07-24T22:02:20.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:19 smithi060 ceph-mon[33000]: pgmap v868: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.0 MiB/s wr, 424 op/s 2024-07-24T22:02:22.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:21 smithi060 ceph-mon[33000]: pgmap v869: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-07-24T22:02:22.245 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:21 smithi123 ceph-mon[37143]: pgmap v869: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-07-24T22:02:22.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:21 smithi086 ceph-mon[37128]: pgmap v869: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-07-24T22:02:22.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:02:23.158 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:24.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:23 smithi123 ceph-mon[37143]: pgmap v870: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-07-24T22:02:24.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:23 smithi086 ceph-mon[37128]: pgmap v870: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-07-24T22:02:24.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:23 smithi060 ceph-mon[33000]: pgmap v870: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 -- 172.21.15.60:0/778842476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81009ddc0 msgr2=0x7fe8100a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 --2- 172.21.15.60:0/778842476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81009ddc0 0x7fe8100a45c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe8180685c0 tx=0x7fe8180978f0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 -- 172.21.15.60:0/778842476 shutdown_connections 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 --2- 172.21.15.60:0/778842476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81009ddc0 0x7fe8100a45c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 --2- 172.21.15.60:0/778842476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009d440 0x7fe81009d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 --2- 172.21.15.60:0/778842476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810097a30 0x7fe810097e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.260+0000 7fe81e34a640 1 -- 172.21.15.60:0/778842476 >> 172.21.15.60:0/778842476 conn(0x7fe810093460 msgr2=0x7fe810095880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.262+0000 7fe81e34a640 1 -- 172.21.15.60:0/778842476 shutdown_connections 2024-07-24T22:02:25.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.262+0000 7fe81e34a640 1 -- 172.21.15.60:0/778842476 wait complete. 2024-07-24T22:02:25.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 Processor -- start 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 -- start start 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 0x7fe8100a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 0x7fe810147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8101482d0 0x7fe810148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe810009d50 con 0x7fe8100a38a0 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe810009bd0 con 0x7fe8101482d0 2024-07-24T22:02:25.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.263+0000 7fe81e34a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe810009ed0 con 0x7fe81009ddc0 2024-07-24T22:02:25.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.264+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 0x7fe810147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 0x7fe810147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44158/0 (socket says 172.21.15.60:44158) 2024-07-24T22:02:25.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81cb47640 1 -- 172.21.15.60:0/1563196694 learned_addr learned my addr 172.21.15.60:0/1563196694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:25.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 0x7fe8100a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81db49640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8101482d0 0x7fe810148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 -- 172.21.15.60:0/1563196694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8101482d0 msgr2=0x7fe810148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8101482d0 0x7fe810148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 -- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 msgr2=0x7fe810147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 0x7fe810147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.265+0000 7fe81d348640 1 -- 172.21.15.60:0/1563196694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8100a6830 con 0x7fe81009ddc0 2024-07-24T22:02:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.266+0000 7fe81d348640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 0x7fe8100a3360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fd0 tx=0x7fe81400b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.267+0000 7fe80e7fc640 1 -- 172.21.15.60:0/1563196694 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814015020 con 0x7fe81009ddc0 2024-07-24T22:02:25.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.267+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8100a6b20 con 0x7fe81009ddc0 2024-07-24T22:02:25.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.267+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8100a7060 con 0x7fe81009ddc0 2024-07-24T22:02:25.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.267+0000 7fe80e7fc640 1 -- 172.21.15.60:0/1563196694 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe814011ce0 con 0x7fe81009ddc0 2024-07-24T22:02:25.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.267+0000 7fe80e7fc640 1 -- 172.21.15.60:0/1563196694 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814010d10 con 0x7fe81009ddc0 2024-07-24T22:02:25.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 Processor -- start 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 -- start start 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf886c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5585cf7fd990 con 0x5585cf8930e0 2024-07-24T22:02:25.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585cf7ece90 con 0x5585cf887150 2024-07-24T22:02:25.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.272+0000 7fe82195d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585cf7f0ed0 con 0x5585cf886830 2024-07-24T22:02:25.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.273+0000 7fe81db49640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf886c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.273+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.273+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44168/0 (socket says 172.21.15.60:44168) 2024-07-24T22:02:25.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.273+0000 7fe81cb47640 1 -- 172.21.15.60:0/3954665348 learned_addr learned my addr 172.21.15.60:0/3954665348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:25.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.275+0000 7fe81d348640 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.275+0000 7fe80e7fc640 1 -- 172.21.15.60:0/1563196694 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe814018020 con 0x7fe81009ddc0 2024-07-24T22:02:25.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.276+0000 7fe81cb47640 1 -- 172.21.15.60:0/3954665348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 msgr2=0x5585cf886c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:25.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.276+0000 7fe81cb47640 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf886c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.276+0000 7fe81cb47640 1 -- 172.21.15.60:0/3954665348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 msgr2=0x5585cf892ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:25.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.276+0000 7fe81cb47640 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.276+0000 7fe81cb47640 1 -- 172.21.15.60:0/3954665348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585cf776300 con 0x5585cf8930e0 2024-07-24T22:02:25.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.278+0000 7fe80e7fc640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 0x7fe8040b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.279+0000 7fe80e7fc640 1 -- 172.21.15.60:0/1563196694 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fe8140e3320 con 0x7fe81009ddc0 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.279+0000 7fe81cb47640 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe808009a60 tx=0x7fe808009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6a0deed1c9c245 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.280+0000 7fe8037fe640 1 -- 172.21.15.60:0/3954665348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808017070 con 0x5585cf8930e0 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.280+0000 7fe8037fe640 1 -- 172.21.15.60:0/3954665348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe808013470 con 0x5585cf8930e0 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.281+0000 7fe81db49640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 0x7fe8040b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.281+0000 7fe82195d740 1 -- 172.21.15.60:0/3954665348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 msgr2=0x5585cf8954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.281+0000 7fe82195d740 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe808009a60 tx=0x7fe808009f20 comp rx=0 tx=0).stop 2024-07-24T22:02:25.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.284+0000 7fe82195d740 1 -- 172.21.15.60:0/3954665348 shutdown_connections 2024-07-24T22:02:25.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.284+0000 7fe82195d740 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf8930e0 0x5585cf8954d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.284+0000 7fe82195d740 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.284+0000 7fe82195d740 1 --2- 172.21.15.60:0/3954665348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf886c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.284+0000 7fe82195d740 1 -- 172.21.15.60:0/3954665348 >> 172.21.15.60:0/3954665348 conn(0x5585cf884660 msgr2=0x5585cf885800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.285+0000 7fe82195d740 1 -- 172.21.15.60:0/3954665348 shutdown_connections 2024-07-24T22:02:25.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.285+0000 7fe82195d740 1 -- 172.21.15.60:0/3954665348 wait complete. 2024-07-24T22:02:25.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.285+0000 7fe81db49640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 0x7fe8040b63d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe818097ee0 tx=0x7fe8180a0040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.285+0000 7fe82195d740 1 -- start start 2024-07-24T22:02:25.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.285+0000 7fe82195d740 1 Processor -- start 2024-07-24T22:02:25.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf895230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 0x5585cf893040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5585cf650700 con 0x5585cf892bc0 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585cf78f5d0 con 0x5585cf887150 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585cf895db0 con 0x5585cf886830 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe82195d740 10 client.0 _ll_register_callbacks cb 0x5585cf7af940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.286+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:50018/0 (socket says 172.21.15.60:50018) 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.287+0000 7fe81cb47640 1 -- 172.21.15.60:0/2903583120 learned_addr learned my addr 172.21.15.60:0/2903583120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.287+0000 7fe81db49640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 0x5585cf893040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.287+0000 7fe81d348640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf895230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 -- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 msgr2=0x5585cf895230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf895230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 -- 172.21.15.60:0/2903583120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 msgr2=0x5585cf892680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 -- 172.21.15.60:0/2903583120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585cf782660 con 0x5585cf892bc0 2024-07-24T22:02:25.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.288+0000 7fe81db49640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 0x5585cf893040 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe818052280 tx=0x7fe818068960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3e0a3ed52e9660 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.294+0000 7fe800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.294+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8180a3070 con 0x5585cf892bc0 2024-07-24T22:02:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.294+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fe81809e660 con 0x5585cf892bc0 2024-07-24T22:02:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.294+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5585cf7ff8e0 con 0x5585cf892bc0 2024-07-24T22:02:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.295+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe81806b280 con 0x5585cf892bc0 2024-07-24T22:02:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.295+0000 7fe82195d740 10 client.18702 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.295+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5585cf54d650 con 0x5585cf892bc0 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5585cf7ff8e0 con 0x5585cf892bc0 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fe8180bd8a0 con 0x5585cf892bc0 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 10 client.18702 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 10 client.18702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 20 client.18702 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf8a3300 0x5585cf8a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 4 client.18702 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.296+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585cf7fb380 con 0x5585cf8a3300 2024-07-24T22:02:25.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.297+0000 7fe81d348640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf8a3300 0x5585cf8a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.303+0000 7fe81d348640 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf8a3300 0x5585cf8a5780 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365f5e2d04c2fb14 server_cookie=658bc63460d5976f in_seq=0 out_seq=0 2024-07-24T22:02:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.303+0000 7fe800ff9640 10 client.18702 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.307+0000 7fe800ff9640 1 -- 172.21.15.60:0/2903583120 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585cf7fb380 con 0x5585cf8a3300 2024-07-24T22:02:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.307+0000 7fe800ff9640 10 client.18702 handle_command_reply: tid=0 2024-07-24T22:02:25.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.308+0000 7fe82195d740 1 client.18702 shutdown 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.308+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 msgr2=0x5585cf893040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.308+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 0x5585cf893040 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe818052280 tx=0x7fe818068960 comp rx=0 tx=0).stop 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 shutdown_connections 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf8a3300 0x5585cf8a5780 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf892bc0 0x5585cf893040 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf887150 0x5585cf892680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 --2- 172.21.15.60:0/2903583120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf886830 0x5585cf895230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 >> 172.21.15.60:0/2903583120 conn(0x5585cf884660 msgr2=0x5585cf88a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe800ff9640 0 client.18702 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 shutdown_connections 2024-07-24T22:02:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.309+0000 7fe82195d740 1 -- 172.21.15.60:0/2903583120 wait complete. 2024-07-24T22:02:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.310+0000 7fe82195d740 20 client.18702 trim_cache size 0 max 16384 2024-07-24T22:02:25.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 Processor -- start 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 -- start start 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf8ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf8ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc011430 con 0x5585cf886830 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00ec40 con 0x5585cf892bc0 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe82195d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00cab0 con 0x5585cf887150 2024-07-24T22:02:25.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe81d348640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe81d348640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44190/0 (socket says 172.21.15.60:44190) 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.349+0000 7fe81d348640 1 -- 172.21.15.60:0/3427386658 learned_addr learned my addr 172.21.15.60:0/3427386658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 -- 172.21.15.60:0/3427386658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 msgr2=0x5585cf8ab050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf8ab050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 -- 172.21.15.60:0/3427386658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 msgr2=0x5585cf8ab590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf8ab590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 -- 172.21.15.60:0/3427386658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585cf884490 con 0x5585cf886830 2024-07-24T22:02:25.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.350+0000 7fe81d348640 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe81400bd20 tx=0x7fe814010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b085ec81918786 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.351+0000 7fe802ffd640 1 -- 172.21.15.60:0/3427386658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140e3040 con 0x5585cf886830 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.351+0000 7fe802ffd640 1 -- 172.21.15.60:0/3427386658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8140a9f80 con 0x5585cf886830 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.351+0000 7fe802ffd640 1 -- 172.21.15.60:0/3427386658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140e3040 con 0x5585cf886830 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.351+0000 7fe82195d740 1 -- 172.21.15.60:0/3427386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 msgr2=0x5585cf8a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.351+0000 7fe82195d740 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe81400bd20 tx=0x7fe814010850 comp rx=0 tx=0).stop 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.352+0000 7fe82195d740 1 -- 172.21.15.60:0/3427386658 shutdown_connections 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.352+0000 7fe82195d740 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf8ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.352+0000 7fe82195d740 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf8ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.352+0000 7fe82195d740 1 --2- 172.21.15.60:0/3427386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8a5770 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.352+0000 7fe82195d740 1 -- 172.21.15.60:0/3427386658 >> 172.21.15.60:0/3427386658 conn(0x5585cf884660 msgr2=0x5585cf890f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.353+0000 7fe82195d740 1 -- 172.21.15.60:0/3427386658 shutdown_connections 2024-07-24T22:02:25.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.355+0000 7fe82195d740 1 -- 172.21.15.60:0/3427386658 wait complete. 2024-07-24T22:02:25.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.355+0000 7fe82195d740 1 -- start start 2024-07-24T22:02:25.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.355+0000 7fe82195d740 1 Processor -- start 2024-07-24T22:02:25.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf896e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf90c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00d100 con 0x5585cf886830 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00e570 con 0x5585cf892bc0 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00d7d0 con 0x5585cf887150 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.356+0000 7fe82195d740 10 client.0 _ll_register_callbacks cb 0x7fe818068dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.357+0000 7fe81cb47640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf896e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.357+0000 7fe81d348640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.357+0000 7fe81d348640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44196/0 (socket says 172.21.15.60:44196) 2024-07-24T22:02:25.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.357+0000 7fe81d348640 1 -- 172.21.15.60:0/4257658792 learned_addr learned my addr 172.21.15.60:0/4257658792 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.359+0000 7fe81d348640 1 -- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 msgr2=0x5585cf896e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.359+0000 7fe81d348640 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf896e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.359+0000 7fe81d348640 1 -- 172.21.15.60:0/4257658792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 msgr2=0x5585cf90c010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.359+0000 7fe81d348640 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf90c010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.359+0000 7fe81d348640 1 -- 172.21.15.60:0/4257658792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585cf90c750 con 0x5585cf886830 2024-07-24T22:02:25.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.360+0000 7fe81d348640 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe8140e3290 tx=0x7fe8140bf3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be42ffd4484a8147 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:25.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.362+0000 7fe800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:25.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.362+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140c4070 con 0x5585cf886830 2024-07-24T22:02:25.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.362+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fe814010380 con 0x5585cf886830 2024-07-24T22:02:25.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.362+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8140a9f80 con 0x5585cf886830 2024-07-24T22:02:25.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.366+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe8140e3040 con 0x5585cf886830 2024-07-24T22:02:25.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.366+0000 7fe82195d740 10 client.18708 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:25.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.366+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5585cf90da60 con 0x5585cf886830 2024-07-24T22:02:25.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8140a9f80 con 0x5585cf886830 2024-07-24T22:02:25.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fe8140be030 con 0x5585cf886830 2024-07-24T22:02:25.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 10 client.18708 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:25.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 10 client.18708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:25.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 20 client.18708 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:25.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf90e0f0 0x5585cf910510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:25.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 4 client.18708 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:02:25.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.371+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585cf891ee0 con 0x5585cf90e0f0 2024-07-24T22:02:25.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.373+0000 7fe81cb47640 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf90e0f0 0x5585cf910510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:25.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.378+0000 7fe81cb47640 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf90e0f0 0x5585cf910510 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976daf99e692f763 server_cookie=ea57adca1b4a5ce7 in_seq=0 out_seq=0 2024-07-24T22:02:25.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.380+0000 7fe800ff9640 10 client.18708 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:25.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.380+0000 7fe800ff9640 1 -- 172.21.15.60:0/4257658792 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585cf891ee0 con 0x5585cf90e0f0 2024-07-24T22:02:25.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.380+0000 7fe800ff9640 10 client.18708 handle_command_reply: tid=0 2024-07-24T22:02:25.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.380+0000 7fe82195d740 1 client.18708 shutdown 2024-07-24T22:02:25.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.382+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 msgr2=0x5585cf8968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.382+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe8140e3290 tx=0x7fe8140bf3f0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 shutdown_connections 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5585cf90e0f0 0x5585cf910510 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cf892bc0 0x5585cf90c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585cf887150 0x5585cf896e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 --2- 172.21.15.60:0/4257658792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5585cf886830 0x5585cf8968b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 >> 172.21.15.60:0/4257658792 conn(0x5585cf884660 msgr2=0x5585cf8944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe800ff9640 0 client.18708 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.383+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 shutdown_connections 2024-07-24T22:02:25.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.384+0000 7fe82195d740 1 -- 172.21.15.60:0/4257658792 wait complete. 2024-07-24T22:02:25.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.384+0000 7fe82195d740 20 client.18708 trim_cache size 0 max 16384 2024-07-24T22:02:25.387 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:02:25.387 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:02:25.387 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:02:25.387 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:02:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 msgr2=0x7fe8040b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 0x7fe8040b63d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe818097ee0 tx=0x7fe8180a0040 comp rx=0 tx=0).stop 2024-07-24T22:02:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 msgr2=0x7fe8100a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 0x7fe8100a3360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fd0 tx=0x7fe81400b820 comp rx=0 tx=0).stop 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 shutdown_connections 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.385+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe8040b3f10 0x7fe8040b63d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8101482d0 0x7fe810148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8100a38a0 0x7fe810147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 --2- 172.21.15.60:0/1563196694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe81009ddc0 0x7fe8100a3360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 >> 172.21.15.60:0/1563196694 conn(0x7fe810093460 msgr2=0x7fe810095850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 shutdown_connections 2024-07-24T22:02:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:25.386+0000 7fe81e34a640 1 -- 172.21.15.60:0/1563196694 wait complete. 2024-07-24T22:02:26.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:25 smithi060 ceph-mon[33000]: pgmap v871: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-07-24T22:02:26.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:25 smithi123 ceph-mon[37143]: pgmap v871: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-07-24T22:02:26.286 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:02:26.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:02:26.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:25 smithi086 ceph-mon[37128]: pgmap v871: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-07-24T22:02:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:28.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:27 smithi123 ceph-mon[37143]: pgmap v872: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-07-24T22:02:28.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:27 smithi086 ceph-mon[37128]: pgmap v872: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-07-24T22:02:28.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:27 smithi060 ceph-mon[33000]: pgmap v872: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-07-24T22:02:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.899+0000 7f21b3fff640 1 -- 172.21.15.60:0/3108121835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b411ba90 msgr2=0x7f21b411de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.900+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3108121835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b411ba90 0x7f21b411de80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009c10 tx=0x7f21ac02a720 comp rx=0 tx=0).stop 2024-07-24T22:02:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.902+0000 7f21b3fff640 1 -- 172.21.15.60:0/3108121835 shutdown_connections 2024-07-24T22:02:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.902+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3108121835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b411ba90 0x7f21b411de80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.902+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3108121835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 0x7f21b410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.902+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3108121835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4109b40 0x7f21b4110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.902+0000 7f21b3fff640 1 -- 172.21.15.60:0/3108121835 >> 172.21.15.60:0/3108121835 conn(0x7f21b406c4f0 msgr2=0x7f21b406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.904+0000 7f21b3fff640 1 -- 172.21.15.60:0/3108121835 shutdown_connections 2024-07-24T22:02:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.904+0000 7f21b3fff640 1 -- 172.21.15.60:0/3108121835 wait complete. 2024-07-24T22:02:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.905+0000 7f21b3fff640 1 Processor -- start 2024-07-24T22:02:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 -- start start 2024-07-24T22:02:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4109b40 0x7f21b4114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 0x7f21b4115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4111740 con 0x7f21b411ba90 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41115c0 con 0x7f21b4109b40 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.908+0000 7f21b3fff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41118c0 con 0x7f21b4109f20 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.910+0000 7f21b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.910+0000 7f21b37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55048/0 (socket says 172.21.15.60:55048) 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.910+0000 7f21b37fe640 1 -- 172.21.15.60:0/3320282584 learned_addr learned my addr 172.21.15.60:0/3320282584 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.911+0000 7f21b27fc640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 0x7f21b4115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.912+0000 7f21b37fe640 1 -- 172.21.15.60:0/3320282584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 msgr2=0x7f21b4115420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.912+0000 7f21b37fe640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 0x7f21b4115420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.912+0000 7f21b37fe640 1 -- 172.21.15.60:0/3320282584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4109b40 msgr2=0x7f21b4114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.912+0000 7f21b37fe640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4109b40 0x7f21b4114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.912+0000 7f21b37fe640 1 -- 172.21.15.60:0/3320282584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b411ad90 con 0x7f21b411ba90 2024-07-24T22:02:29.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.913+0000 7f21b37fe640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f21ac02ac90 tx=0x7f21ac030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:29.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.914+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac040020 con 0x7f21b411ba90 2024-07-24T22:02:29.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.914+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b411b020 con 0x7f21b411ba90 2024-07-24T22:02:29.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.914+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b411b550 con 0x7f21b411ba90 2024-07-24T22:02:29.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.914+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ac037040 con 0x7f21b411ba90 2024-07-24T22:02:29.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.914+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac036030 con 0x7f21b411ba90 2024-07-24T22:02:29.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.916+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a00052c0 con 0x7f21b411ba90 2024-07-24T22:02:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.918+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f21ac043020 con 0x7f21b411ba90 2024-07-24T22:02:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.920+0000 7f219bfff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 0x7f219c0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:29.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.920+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f21ac104790 con 0x7f21b411ba90 2024-07-24T22:02:29.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.921+0000 7f21b2ffd640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 0x7f219c0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:29.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.921+0000 7f21b2ffd640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 0x7f219c0b6400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005fd0 tx=0x7f21a4005a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:29.926+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f21ac04a020 con 0x7f21b411ba90 2024-07-24T22:02:30.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:29 smithi060 ceph-mon[33000]: pgmap v873: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-07-24T22:02:30.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:29 smithi123 ceph-mon[37143]: pgmap v873: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-07-24T22:02:30.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:29 smithi086 ceph-mon[37128]: pgmap v873: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-07-24T22:02:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.375+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21a0005880 con 0x7f21b411ba90 2024-07-24T22:02:30.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.377+0000 7f219bfff640 1 -- 172.21.15.60:0/3320282584 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f21ac0cb9e0 con 0x7f21b411ba90 2024-07-24T22:02:30.379 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:02:30.379 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:02:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:02:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.381+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 msgr2=0x7f219c0b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.381+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 0x7f219c0b6400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005fd0 tx=0x7f21a4005a00 comp rx=0 tx=0).stop 2024-07-24T22:02:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.381+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 msgr2=0x7f21b411a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.381+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f21ac02ac90 tx=0x7f21ac030620 comp rx=0 tx=0).stop 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 shutdown_connections 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f219c0b3f40 0x7f219c0b6400 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b411ba90 0x7f21b411a750 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21b4109f20 0x7f21b4115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 --2- 172.21.15.60:0/3320282584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4109b40 0x7f21b4114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.386+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 >> 172.21.15.60:0/3320282584 conn(0x7f21b406c4f0 msgr2=0x7f21b4072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:30.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.387+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 shutdown_connections 2024-07-24T22:02:30.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:30.389+0000 7f21b3fff640 1 -- 172.21.15.60:0/3320282584 wait complete. 2024-07-24T22:02:31.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:30 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3320282584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:02:31.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:30 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3320282584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:02:31.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:02:31.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:30 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3320282584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:02:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 -- 172.21.15.60:0/901210186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109f20 msgr2=0x7f842010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 --2- 172.21.15.60:0/901210186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109f20 0x7f842010a360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8414009900 tx=0x7f841402a400 comp rx=0 tx=0).stop 2024-07-24T22:02:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 -- 172.21.15.60:0/901210186 shutdown_connections 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 --2- 172.21.15.60:0/901210186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f842011ba90 0x7f842011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 --2- 172.21.15.60:0/901210186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109f20 0x7f842010a360 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8414009900 tx=0x7f841402a400 comp rx=0 tx=0).stop 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 --2- 172.21.15.60:0/901210186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8420109b40 0x7f8420110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 -- 172.21.15.60:0/901210186 >> 172.21.15.60:0/901210186 conn(0x7f842006d760 msgr2=0x7f842006db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 -- 172.21.15.60:0/901210186 shutdown_connections 2024-07-24T22:02:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.509+0000 7f8424ec6640 1 -- 172.21.15.60:0/901210186 wait complete. 2024-07-24T22:02:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.510+0000 7f8424ec6640 1 Processor -- start 2024-07-24T22:02:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.511+0000 7f8424ec6640 1 -- start start 2024-07-24T22:02:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.511+0000 7f8424ec6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109b40 0x7f842011b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.511+0000 7f8424ec6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.511+0000 7f8424ec6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201159b0 0x7f8420115e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f8424ec6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8420111b70 con 0x7f842011ba90 2024-07-24T22:02:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55070/0 (socket says 172.21.15.60:55070) 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 -- 172.21.15.60:0/959420920 learned_addr learned my addr 172.21.15.60:0/959420920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84201119f0 con 0x7f8420109b40 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8420111cf0 con 0x7f84201159b0 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 -- 172.21.15.60:0/959420920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201159b0 msgr2=0x7f8420115e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201159b0 0x7f8420115e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 -- 172.21.15.60:0/959420920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109b40 msgr2=0x7f842011b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109b40 0x7f842011b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.512+0000 7f841dd9b640 1 -- 172.21.15.60:0/959420920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201bcb20 con 0x7f842011ba90 2024-07-24T22:02:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.513+0000 7f841dd9b640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f841402a9f0 tx=0x7f8414003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.513+0000 7f840f7fe640 1 -- 172.21.15.60:0/959420920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841403f020 con 0x7f842011ba90 2024-07-24T22:02:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.513+0000 7f840f7fe640 1 -- 172.21.15.60:0/959420920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8414031d60 con 0x7f842011ba90 2024-07-24T22:02:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.513+0000 7f840f7fe640 1 -- 172.21.15.60:0/959420920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414035070 con 0x7f842011ba90 2024-07-24T22:02:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.514+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201bcdb0 con 0x7f842011ba90 2024-07-24T22:02:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.514+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201bd1c0 con 0x7f842011ba90 2024-07-24T22:02:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.517+0000 7f8427c16740 1 Processor -- start 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.518+0000 7f8427c16740 1 -- start start 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.518+0000 7f8427c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f757ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.519+0000 7f8427c16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f763c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.519+0000 7f8427c16740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.519+0000 7f8427c16740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f6cd280 con 0x55bf5f764170 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.519+0000 7f8427c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f6bc780 con 0x55bf5f7578c0 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.519+0000 7f8427c16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f6c07c0 con 0x55bf5f7581e0 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.520+0000 7f841ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f757ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.521+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.521+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55074/0 (socket says 172.21.15.60:55074) 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.521+0000 7f841dd9b640 1 -- 172.21.15.60:0/892248374 learned_addr learned my addr 172.21.15.60:0/892248374 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f840f7fe640 1 -- 172.21.15.60:0/959420920 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8414042020 con 0x7f842011ba90 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f841dd9b640 1 -- 172.21.15.60:0/892248374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 msgr2=0x55bf5f763c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f841dd9b640 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f763c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f841dd9b640 1 -- 172.21.15.60:0/892248374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 msgr2=0x55bf5f757ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f841dd9b640 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f757ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.522+0000 7f841dd9b640 1 -- 172.21.15.60:0/892248374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5f420790 con 0x55bf5f764170 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.525+0000 7f841ed9d640 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f757ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.527+0000 7f840f7fe640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 0x7f84080b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.527+0000 7f841ed9d640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 0x7f84080b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.531+0000 7f840f7fe640 1 -- 172.21.15.60:0/959420920 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8414104a50 con 0x7f842011ba90 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.533+0000 7f841dd9b640 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8414102020 tx=0x7f8414108da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1b2c27dd76fedc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.533+0000 7f840cff9640 1 -- 172.21.15.60:0/892248374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140c7460 con 0x55bf5f764170 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.534+0000 7f840cff9640 1 -- 172.21.15.60:0/892248374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841410e040 con 0x55bf5f764170 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 -- 172.21.15.60:0/892248374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 msgr2=0x55bf5f766560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8414102020 tx=0x7f8414108da0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 -- 172.21.15.60:0/892248374 shutdown_connections 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f766560 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f763c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 --2- 172.21.15.60:0/892248374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f757ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.535+0000 7f8427c16740 1 -- 172.21.15.60:0/892248374 >> 172.21.15.60:0/892248374 conn(0x55bf5f7556f0 msgr2=0x55bf5f756890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.536+0000 7f841ed9d640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 0x7f84080b6410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f842006bc30 tx=0x7f84180093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.539+0000 7f8427c16740 1 -- 172.21.15.60:0/892248374 shutdown_connections 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.540+0000 7f8427c16740 1 -- 172.21.15.60:0/892248374 wait complete. 2024-07-24T22:02:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.541+0000 7f8427c16740 1 -- start start 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.541+0000 7f8427c16740 1 Processor -- start 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.541+0000 7f8427c16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7578c0 0x55bf5f7638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.541+0000 7f8427c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f8427c16740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f7612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f8427c16740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f5f6ab0 con 0x55bf5f764170 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f8427c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f6449e0 con 0x55bf5f7581e0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49972/0 (socket says 172.21.15.60:49972) 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841dd9b640 1 -- 172.21.15.60:0/2739726707 learned_addr learned my addr 172.21.15.60:0/2739726707 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf5f6cd280 con 0x55bf5f7578c0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841e59c640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7578c0 0x55bf5f7638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f7612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f8427c16740 10 client.0 _ll_register_callbacks cb 0x55bf5f3cca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 -- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7578c0 msgr2=0x55bf5f7638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7578c0 0x55bf5f7638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 -- 172.21.15.60:0/2739726707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 msgr2=0x55bf5f760da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.542+0000 7f841ed9d640 1 -- 172.21.15.60:0/2739726707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5f499620 con 0x55bf5f764170 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.543+0000 7f841ed9d640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f7612e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a572d2f29525c87f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.543+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf5f66bce0 con 0x55bf5f764170 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.543+0000 7f841dd9b640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.543+0000 7f8405ffb640 10 client.18723 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.543+0000 7f8405ffb640 1 -- 172.21.15.60:0/2739726707 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418022070 con 0x55bf5f764170 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.544+0000 7f8405ffb640 1 -- 172.21.15.60:0/2739726707 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8418003b70 con 0x55bf5f764170 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.544+0000 7f8405ffb640 1 -- 172.21.15.60:0/2739726707 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8418003d70 con 0x55bf5f764170 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.548+0000 7f8427c16740 10 client.18723 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.548+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55bf5f772260 con 0x55bf5f764170 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8405ffb640 1 -- 172.21.15.60:0/2739726707 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f841802e020 con 0x55bf5f764170 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 10 client.18723 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 10 client.18723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 20 client.18723 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f773bb0 0x55bf5f776070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 4 client.18723 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.551+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5f66bce0 con 0x55bf5f773bb0 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.553+0000 7f841e59c640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f773bb0 0x55bf5f776070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.557+0000 7f841e59c640 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f773bb0 0x55bf5f776070 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a79b9fd5189ea5 server_cookie=8d8047098b70c1ef in_seq=0 out_seq=0 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.557+0000 7f8405ffb640 10 client.18723 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.559+0000 7f8405ffb640 1 -- 172.21.15.60:0/2739726707 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf5f66bce0 con 0x55bf5f773bb0 2024-07-24T22:02:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.559+0000 7f8405ffb640 10 client.18723 handle_command_reply: tid=0 2024-07-24T22:02:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.559+0000 7f8427c16740 1 client.18723 shutdown 2024-07-24T22:02:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.559+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 msgr2=0x55bf5f7612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.559+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f7612e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418004a40 comp rx=0 tx=0).stop 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 shutdown_connections 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f773bb0 0x55bf5f776070 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f764170 0x55bf5f7612e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7581e0 0x55bf5f760da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 --2- 172.21.15.60:0/2739726707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7578c0 0x55bf5f7638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.560+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 >> 172.21.15.60:0/2739726707 conn(0x55bf5f7556f0 msgr2=0x55bf5f75b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.564+0000 7f8405ffb640 0 client.18723 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.567+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 shutdown_connections 2024-07-24T22:02:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.567+0000 7f8427c16740 1 -- 172.21.15.60:0/2739726707 wait complete. 2024-07-24T22:02:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.567+0000 7f8427c16740 20 client.18723 trim_cache size 0 max 16384 2024-07-24T22:02:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.604+0000 7f8427c16740 1 Processor -- start 2024-07-24T22:02:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.604+0000 7f8427c16740 1 -- start start 2024-07-24T22:02:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f8427c16740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 0x55bf5f756e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f8427c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f77b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f8427c16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f77be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f8427c16740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c87c0 con 0x55bf5f7556f0 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f841e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 0x55bf5f756e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f841e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 0x55bf5f756e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55092/0 (socket says 172.21.15.60:55092) 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f841e59c640 1 -- 172.21.15.60:0/4052720639 learned_addr learned my addr 172.21.15.60:0/4052720639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c5fd0 con 0x55bf5f7578c0 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f841ed9d640 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f77be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.605+0000 7f841dd9b640 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f77b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c3e40 con 0x55bf5f7581e0 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f841ed9d640 1 -- 172.21.15.60:0/4052720639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 msgr2=0x55bf5f77b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f841ed9d640 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f77b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f841ed9d640 1 -- 172.21.15.60:0/4052720639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 msgr2=0x55bf5f756e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f841ed9d640 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 0x55bf5f756e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.606+0000 7f841ed9d640 1 -- 172.21.15.60:0/4052720639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5f758010 con 0x55bf5f7581e0 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.607+0000 7f841ed9d640 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f77be80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8418011fd0 tx=0x7f841801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155bb3e59990c866 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.607+0000 7f8407fff640 1 -- 172.21.15.60:0/4052720639 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418022020 con 0x55bf5f7581e0 2024-07-24T22:02:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.607+0000 7f8407fff640 1 -- 172.21.15.60:0/4052720639 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8418005030 con 0x55bf5f7581e0 2024-07-24T22:02:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.608+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 msgr2=0x55bf5f77be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.608+0000 7f8427c16740 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f77be80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8418011fd0 tx=0x7f841801f000 comp rx=0 tx=0).stop 2024-07-24T22:02:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.612+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 shutdown_connections 2024-07-24T22:02:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.612+0000 7f8427c16740 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7581e0 0x55bf5f77be80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.612+0000 7f8427c16740 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f77b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.612+0000 7f8427c16740 1 --2- 172.21.15.60:0/4052720639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7556f0 0x55bf5f756e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.612+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 >> 172.21.15.60:0/4052720639 conn(0x55bf5f760910 msgr2=0x55bf5f756740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.614+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 shutdown_connections 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.614+0000 7f8427c16740 1 -- 172.21.15.60:0/4052720639 wait complete. 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.614+0000 7f8427c16740 1 -- start start 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.614+0000 7f8427c16740 1 Processor -- start 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f7676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7581e0 0x55bf5f7dc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c4490 con 0x55bf5f7581e0 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c5900 con 0x55bf5f7578c0 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84201c4b60 con 0x55bf5f7556f0 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f8427c16740 10 client.0 _ll_register_callbacks cb 0x7f8418015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f841e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f841e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43600/0 (socket says 172.21.15.60:43600) 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f841e59c640 1 -- 172.21.15.60:0/4051007569 learned_addr learned my addr 172.21.15.60:0/4051007569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f841dd9b640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f7676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.615+0000 7f841ed9d640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7581e0 0x55bf5f7dc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 -- 172.21.15.60:0/4051007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 msgr2=0x55bf5f7676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f7676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 -- 172.21.15.60:0/4051007569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7581e0 msgr2=0x55bf5f7dc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7581e0 0x55bf5f7dc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 -- 172.21.15.60:0/4051007569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5f7dce40 con 0x55bf5f7556f0 2024-07-24T22:02:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.616+0000 7f841e59c640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8410004880 tx=0x7f8410009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f465e6ea4aa28c1d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.618+0000 7f8405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.618+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410022020 con 0x55bf5f7556f0 2024-07-24T22:02:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.618+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8418005030 con 0x55bf5f7556f0 2024-07-24T22:02:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.618+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8410017bd0 con 0x55bf5f7556f0 2024-07-24T22:02:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.619+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f841002d050 con 0x55bf5f7556f0 2024-07-24T22:02:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.619+0000 7f8427c16740 10 client.27982 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.619+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bf5f7de0c0 con 0x55bf5f7556f0 2024-07-24T22:02:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.620+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8418005030 con 0x55bf5f7556f0 2024-07-24T22:02:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.620+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f841001d030 con 0x55bf5f7556f0 2024-07-24T22:02:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.620+0000 7f8427c16740 10 client.27982 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.620+0000 7f8427c16740 10 client.27982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.620+0000 7f8427c16740 20 client.27982 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.621+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f764170 0x55bf5f7e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.621+0000 7f8427c16740 4 client.27982 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:02:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.621+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf5f6cf1d0 con 0x55bf5f764170 2024-07-24T22:02:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.622+0000 7f841dd9b640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f764170 0x55bf5f7e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.623+0000 7f841dd9b640 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f764170 0x55bf5f7e08b0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1e5876720da90e server_cookie=76af37c5f8323338 in_seq=0 out_seq=0 2024-07-24T22:02:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.623+0000 7f8405ffb640 10 client.27982 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.719+0000 7f8405ffb640 1 -- 172.21.15.60:0/4051007569 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf5f6cf1d0 con 0x55bf5f764170 2024-07-24T22:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.719+0000 7f8405ffb640 10 client.27982 handle_command_reply: tid=0 2024-07-24T22:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.719+0000 7f8427c16740 1 client.27982 shutdown 2024-07-24T22:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.720+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 msgr2=0x55bf5f767160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.720+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8410004880 tx=0x7f8410009e70 comp rx=0 tx=0).stop 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 shutdown_connections 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bf5f764170 0x55bf5f7e08b0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bf5f7581e0 0x55bf5f7dc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf5f7578c0 0x55bf5f7676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 --2- 172.21.15.60:0/4051007569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf5f7556f0 0x55bf5f767160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 >> 172.21.15.60:0/4051007569 conn(0x55bf5f760910 msgr2=0x55bf5f75f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8405ffb640 0 client.27982 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 shutdown_connections 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 1 -- 172.21.15.60:0/4051007569 wait complete. 2024-07-24T22:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.721+0000 7f8427c16740 20 client.27982 trim_cache size 0 max 16384 2024-07-24T22:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 msgr2=0x7f84080b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 0x7f84080b6410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f842006bc30 tx=0x7f84180093e0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 msgr2=0x7f8420115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f841402a9f0 tx=0x7f8414003cd0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 shutdown_connections 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f84080b3f50 0x7f84080b6410 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201159b0 0x7f8420115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f842011ba90 0x7f8420115470 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 --2- 172.21.15.60:0/959420920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8420109b40 0x7f842011b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.724+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 >> 172.21.15.60:0/959420920 conn(0x7f842006d760 msgr2=0x7f8420071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.726+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 shutdown_connections 2024-07-24T22:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:31.728+0000 7f8424ec6640 1 -- 172.21.15.60:0/959420920 wait complete. 2024-07-24T22:02:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:32.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:31 smithi060 ceph-mon[33000]: pgmap v874: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-24T22:02:32.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:31 smithi123 ceph-mon[37143]: pgmap v874: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-24T22:02:32.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:31 smithi086 ceph-mon[37128]: pgmap v874: 97 pgs: 34 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 46 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-24T22:02:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:33 smithi060 ceph-mon[33000]: pgmap v875: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-07-24T22:02:34.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:33 smithi123 ceph-mon[37143]: pgmap v875: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-07-24T22:02:34.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:33 smithi086 ceph-mon[37128]: pgmap v875: 97 pgs: 34 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 47 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 -- 172.21.15.60:0/1065367288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b0109f20 msgr2=0x7fa2b010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/1065367288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b0109f20 0x7fa2b010a360 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009900 tx=0x7fa2a002a480 comp rx=0 tx=0).stop 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 -- 172.21.15.60:0/1065367288 shutdown_connections 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/1065367288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b011ba90 0x7fa2b011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/1065367288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b0109f20 0x7fa2b010a360 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/1065367288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b0110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.081+0000 7fa2b59ef640 1 -- 172.21.15.60:0/1065367288 >> 172.21.15.60:0/1065367288 conn(0x7fa2b006d980 msgr2=0x7fa2b006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.082+0000 7fa2b59ef640 1 -- 172.21.15.60:0/1065367288 shutdown_connections 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.082+0000 7fa2b59ef640 1 -- 172.21.15.60:0/1065367288 wait complete. 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.083+0000 7fa2b59ef640 1 Processor -- start 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.083+0000 7fa2b59ef640 1 -- start start 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.083+0000 7fa2b59ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b01a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2b59ef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b0109f20 0x7fa2b01a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2b59ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2b59ef640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0121550 con 0x7fa2b011ba90 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2b59ef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01213d0 con 0x7fa2b0109b40 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2b59ef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01216d0 con 0x7fa2b0109f20 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55116/0 (socket says 172.21.15.60:55116) 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 -- 172.21.15.60:0/3422294831 learned_addr learned my addr 172.21.15.60:0/3422294831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b01a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b0109f20 0x7fa2b01a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 -- 172.21.15.60:0/3422294831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b0109f20 msgr2=0x7fa2b01a7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b0109f20 0x7fa2b01a7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 -- 172.21.15.60:0/3422294831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 msgr2=0x7fa2b01a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b01a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2af7fe640 1 -- 172.21.15.60:0/3422294831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b01adad0 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.084+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b01a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d950 tx=0x7fa2a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2b49ed640 1 -- 172.21.15.60:0/3422294831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4017030 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b01addc0 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2b49ed640 1 -- 172.21.15.60:0/3422294831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a4004020 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01b4480 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.085+0000 7fa2b49ed640 1 -- 172.21.15.60:0/3422294831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4003b30 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.089+0000 7fa2b49ed640 1 -- 172.21.15.60:0/3422294831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fa2a401a070 con 0x7fa2b011ba90 2024-07-24T22:02:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.090+0000 7fa2b7da0740 1 Processor -- start 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.092+0000 7fa2b49ed640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 0x7fa2980b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.092+0000 7fa2b49ed640 1 -- 172.21.15.60:0/3422294831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fa2a40e3370 con 0x7fa2b011ba90 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.093+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 0x7fa2980b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.094+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 0x7fa2980b6340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2b011e0e0 tx=0x7fa2a0031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.094+0000 7fa2b7da0740 1 -- start start 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a77ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a83f80 0x55f815a86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f8159ee9b0 con 0x55f815a776d0 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8159ddeb0 con 0x55f815a83f80 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f8159e1ef0 con 0x55f815a77ff0 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a83a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:43632/0 (socket says 172.21.15.60:43632) 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.095+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/400652128 learned_addr learned my addr 172.21.15.60:0/400652128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a77ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 -- 172.21.15.60:0/400652128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 msgr2=0x55f815a83a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a83a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 -- 172.21.15.60:0/400652128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a83f80 msgr2=0x55f815a86370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a83f80 0x55f815a86370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.096+0000 7fa2aeffd640 1 -- 172.21.15.60:0/400652128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f815967320 con 0x55f815a776d0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.097+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a77ab0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a8011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11b9a7e920ccb0c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.098+0000 7fa295ffb640 1 -- 172.21.15.60:0/400652128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8003c30 con 0x55f815a776d0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.098+0000 7fa295ffb640 1 -- 172.21.15.60:0/400652128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a800fe50 con 0x55f815a776d0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.099+0000 7fa295ffb640 1 -- 172.21.15.60:0/400652128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80193e0 con 0x55f815a776d0 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.100+0000 7fa2b7da0740 1 -- 172.21.15.60:0/400652128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 msgr2=0x55f815a77ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.100+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a77ab0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a8011c70 comp rx=0 tx=0).stop 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.104+0000 7fa2b7da0740 1 -- 172.21.15.60:0/400652128 shutdown_connections 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.104+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a83f80 0x55f815a86370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.104+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.104+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/400652128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a77ab0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.104+0000 7fa2b7da0740 1 -- 172.21.15.60:0/400652128 >> 172.21.15.60:0/400652128 conn(0x55f815a75500 msgr2=0x55f815a766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.107+0000 7fa2b7da0740 1 -- 172.21.15.60:0/400652128 shutdown_connections 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.107+0000 7fa2b7da0740 1 -- 172.21.15.60:0/400652128 wait complete. 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.107+0000 7fa2b7da0740 1 -- start start 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.108+0000 7fa2b7da0740 1 Processor -- start 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a80f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a81480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a83f80 0x55f815a819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f815841700 con 0x55f815a776d0 2024-07-24T22:02:34.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8159805f0 con 0x55f815a77ff0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f815a86c00 con 0x55f815a83f80 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2b7da0740 10 client.0 _ll_register_callbacks cb 0x55f8159a0960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a81480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a81480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:50030/0 (socket says 172.21.15.60:50030) 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/2096649369 learned_addr learned my addr 172.21.15.60:0/2096649369 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a80f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.109+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a83f80 0x55f815a819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.110+0000 7fa2aeffd640 1 -- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a83f80 msgr2=0x55f815a819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.110+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a83f80 0x55f815a819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.110+0000 7fa2aeffd640 1 -- 172.21.15.60:0/2096649369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 msgr2=0x55f815a81480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.110+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a81480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.110+0000 7fa2aeffd640 1 -- 172.21.15.60:0/2096649369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f815973680 con 0x55f815a776d0 2024-07-24T22:02:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.111+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a80f40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a8004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e02ee398cc2a655 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.111+0000 7fa28effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.111+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a801f040 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.111+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a800fe50 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.112+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fa2a8011240 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.112+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa2a800fc60 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.112+0000 7fa2b7da0740 10 client.18741 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.112+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f81573e650 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.112+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a800fe50 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fa2a8018070 con 0x55f815a776d0 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 10 client.18741 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 10 client.18741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 20 client.18741 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815a93df0 0x55f815a96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 4 client.18741 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8159f0900 con 0x55f815a93df0 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.113+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815a93df0 0x55f815a96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.114+0000 7fa28effd640 10 client.18741 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.114+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815a93df0 0x55f815a96270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8312684e4edfb92 server_cookie=7a4474d45f944896 in_seq=0 out_seq=0 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.115+0000 7fa28effd640 1 -- 172.21.15.60:0/2096649369 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8159f0900 con 0x55f815a93df0 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.115+0000 7fa28effd640 10 client.18741 handle_command_reply: tid=0 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.115+0000 7fa2b7da0740 1 client.18741 shutdown 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.115+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 msgr2=0x55f815a80f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.115+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a80f40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a8004300 comp rx=0 tx=0).stop 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 shutdown_connections 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815a93df0 0x55f815a96270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a83f80 0x55f815a819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a81480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/2096649369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a776d0 0x55f815a80f40 secure :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8005fd0 tx=0x7fa2a8004300 comp rx=0 tx=0).stop 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 >> 172.21.15.60:0/2096649369 conn(0x55f815a75500 msgr2=0x55f815a7b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa28effd640 0 client.18741 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 shutdown_connections 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 1 -- 172.21.15.60:0/2096649369 wait complete. 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.116+0000 7fa2b7da0740 20 client.18741 trim_cache size 0 max 16384 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.143+0000 7fa2b7da0740 1 Processor -- start 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.143+0000 7fa2b7da0740 1 -- start start 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a831c0 0x55f815a9c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55128/0 (socket says 172.21.15.60:55128) 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.144+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3038224651 learned_addr learned my addr 172.21.15.60:0/3038224651 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a831c0 0x55f815a9c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bf980 con 0x55f815a83f80 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bd190 con 0x55f815a831c0 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bb000 con 0x55f815a77ff0 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3038224651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 msgr2=0x55f815a82c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a82c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3038224651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a831c0 msgr2=0x55f815a9c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a831c0 0x55f815a9c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.145+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3038224651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f815a75f00 con 0x55f815a83f80 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.146+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006010 tx=0x7fa2a0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc16295255a1366 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.146+0000 7fa2957fa640 1 -- 172.21.15.60:0/3038224651 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035040 con 0x55f815a83f80 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.146+0000 7fa2957fa640 1 -- 172.21.15.60:0/3038224651 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0005500 con 0x55f815a83f80 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.147+0000 7fa2957fa640 1 -- 172.21.15.60:0/3038224651 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004df0 con 0x55f815a83f80 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 msgr2=0x55f815a9bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006010 tx=0x7fa2a0040000 comp rx=0 tx=0).stop 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 shutdown_connections 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a831c0 0x55f815a9c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a9bb40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3038224651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a77ff0 0x55f815a82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 >> 172.21.15.60:0/3038224651 conn(0x55f815a75500 msgr2=0x55f815a82570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 shutdown_connections 2024-07-24T22:02:34.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.148+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3038224651 wait complete. 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.149+0000 7fa2b7da0740 1 -- start start 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.149+0000 7fa2b7da0740 1 Processor -- start 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.150+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.150+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 0x55f815a87990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.150+0000 7fa2b7da0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a87ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.150+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a8c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:50046/0 (socket says 172.21.15.60:50046) 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 0x55f815a87990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2aeffd640 1 -- 172.21.15.60:0/3835653729 learned_addr learned my addr 172.21.15.60:0/3835653729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bb650 con 0x55f815a83f80 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bcac0 con 0x55f815a77ff0 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01bbd20 con 0x55f815a831c0 2024-07-24T22:02:34.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2af7fe640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a87ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2b7da0740 10 client.0 _ll_register_callbacks cb 0x7fa2a8011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3835653729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 msgr2=0x55f815a8c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a8c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3835653729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 msgr2=0x55f815a87ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a87ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.151+0000 7fa2ae7fc640 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f815a88660 con 0x55f815a831c0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.152+0000 7fa2ae7fc640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 0x55f815a87990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55f815a75e50 tx=0x7fa2a0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2f2768d0ceee99 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.152+0000 7fa28effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.152+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0061060 con 0x55f815a831c0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.152+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a0005500 con 0x55f815a831c0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.152+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fa2a0004a70 con 0x55f815a831c0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.153+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa2a0035040 con 0x55f815a831c0 2024-07-24T22:02:34.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.153+0000 7fa2b7da0740 10 client.28000 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.153+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f815afeaa0 con 0x55f815a831c0 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.153+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0005500 con 0x55f815a831c0 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fa2a0052020 con 0x55f815a831c0 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 10 client.28000 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 10 client.28000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 20 client.28000 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815aff180 0x55f815b015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 4 client.28000 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.154+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8159ec3a0 con 0x55f815aff180 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.155+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815aff180 0x55f815b015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.155+0000 7fa2aeffd640 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815aff180 0x55f815b015a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51b050a0d63b0bee server_cookie=249b48da0b2b45dd in_seq=0 out_seq=0 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.155+0000 7fa28effd640 10 client.28000 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:02:34.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.158+0000 7fa28effd640 1 -- 172.21.15.60:0/3835653729 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8159ec3a0 con 0x55f815aff180 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.158+0000 7fa28effd640 10 client.28000 handle_command_reply: tid=0 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.158+0000 7fa2b7da0740 1 client.28000 shutdown 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.158+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 msgr2=0x55f815a87990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.158+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 0x55f815a87990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55f815a75e50 tx=0x7fa2a0005740 comp rx=0 tx=0).stop 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 shutdown_connections 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55f815aff180 0x55f815b015a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa28effd640 0 client.28000 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f815a83f80 0x55f815a87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f815a831c0 0x55f815a87990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 --2- 172.21.15.60:0/3835653729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f815a77ff0 0x55f815a8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 >> 172.21.15.60:0/3835653729 conn(0x55f815a75500 msgr2=0x55f815a7a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 shutdown_connections 2024-07-24T22:02:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.159+0000 7fa2b7da0740 1 -- 172.21.15.60:0/3835653729 wait complete. 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.160+0000 7fa2b7da0740 20 client.28000 trim_cache size 0 max 16384 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.161+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 msgr2=0x7fa2980b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.161+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 0x7fa2980b6340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2b011e0e0 tx=0x7fa2a0031400 comp rx=0 tx=0).stop 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.161+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 msgr2=0x7fa2b01ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.161+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d950 tx=0x7fa2a400de10 comp rx=0 tx=0).stop 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 shutdown_connections 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fa2980b3e80 0x7fa2980b6340 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b011ba90 0x7fa2b01ad300 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b0109f20 0x7fa2b01a7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 --2- 172.21.15.60:0/3422294831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2b0109b40 0x7fa2b01a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 >> 172.21.15.60:0/3422294831 conn(0x7fa2b006d980 msgr2=0x7fa2b011c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 shutdown_connections 2024-07-24T22:02:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:34.162+0000 7fa2b59ef640 1 -- 172.21.15.60:0/3422294831 wait complete. 2024-07-24T22:02:35.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:02:35.979 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:36.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:35 smithi123 ceph-mon[37143]: pgmap v876: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-07-24T22:02:36.292 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:35 smithi060 ceph-mon[33000]: pgmap v876: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-07-24T22:02:36.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:35 smithi086 ceph-mon[37128]: pgmap v876: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 2.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-07-24T22:02:38.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:37 smithi060 ceph-mon[33000]: pgmap v877: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.238+0000 7fef94e68640 1 -- 172.21.15.60:0/684255511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef90107850 msgr2=0x7fef9010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.238+0000 7fef94e68640 1 --2- 172.21.15.60:0/684255511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef90107850 0x7fef9010e050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fef8801acb0 tx=0x7fef8803b920 comp rx=0 tx=0).stop 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.239+0000 7fef94e68640 1 -- 172.21.15.60:0/684255511 shutdown_connections 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.239+0000 7fef94e68640 1 --2- 172.21.15.60:0/684255511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef90107850 0x7fef9010e050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.239+0000 7fef94e68640 1 --2- 172.21.15.60:0/684255511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef90107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.239+0000 7fef94e68640 1 --2- 172.21.15.60:0/684255511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef901014c0 0x7fef901018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.239+0000 7fef94e68640 1 -- 172.21.15.60:0/684255511 >> 172.21.15.60:0/684255511 conn(0x7fef900fcf90 msgr2=0x7fef900ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:37 smithi123 ceph-mon[37143]: pgmap v877: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-07-24T22:02:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.242+0000 7fef94e68640 1 -- 172.21.15.60:0/684255511 shutdown_connections 2024-07-24T22:02:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.242+0000 7fef94e68640 1 -- 172.21.15.60:0/684255511 wait complete. 2024-07-24T22:02:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.243+0000 7fef94e68640 1 Processor -- start 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 -- start start 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef901014c0 0x7fef9010dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef9006f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef9006eda0 con 0x7fef90107850 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9006ec20 con 0x7fef901014c0 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef94e68640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef9006ef20 con 0x7fef90106ed0 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.244+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37598/0 (socket says 172.21.15.60:37598) 2024-07-24T22:02:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 -- 172.21.15.60:0/3406051105 learned_addr learned my addr 172.21.15.60:0/3406051105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8effd640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef9006f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 -- 172.21.15.60:0/3406051105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 msgr2=0x7fef9006f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef9006f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 -- 172.21.15.60:0/3406051105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef901014c0 msgr2=0x7fef9010dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef901014c0 0x7fef9010dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8ffff640 1 -- 172.21.15.60:0/3406051105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9010e370 con 0x7fef90107850 2024-07-24T22:02:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.245+0000 7fef8effd640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef9006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:02:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.251+0000 7fef8ffff640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fef880409c0 tx=0x7fef880409f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.252+0000 7fef8cff9640 1 -- 172.21.15.60:0/3406051105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88045040 con 0x7fef90107850 2024-07-24T22:02:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.252+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90070170 con 0x7fef90107850 2024-07-24T22:02:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.252+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef90196bd0 con 0x7fef90107850 2024-07-24T22:02:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.252+0000 7fef8cff9640 1 -- 172.21.15.60:0/3406051105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef88041dc0 con 0x7fef90107850 2024-07-24T22:02:38.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.253+0000 7fef8cff9640 1 -- 172.21.15.60:0/3406051105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8804b690 con 0x7fef90107850 2024-07-24T22:02:38.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.257+0000 7fef9847b740 1 Processor -- start 2024-07-24T22:02:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.259+0000 7fef8cff9640 1 -- 172.21.15.60:0/3406051105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fef88048020 con 0x7fef90107850 2024-07-24T22:02:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.261+0000 7fef9847b740 1 -- start start 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512219170 0x557512224bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x5575122274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55751218f9b0 con 0x557512219170 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55751217eeb0 con 0x557512225100 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.262+0000 7fef9847b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557512182ef0 con 0x557512218850 2024-07-24T22:02:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.263+0000 7fef8cff9640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 0x7fef780b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.263+0000 7fef8cff9640 1 -- 172.21.15.60:0/3406051105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fef88116190 con 0x7fef90107850 2024-07-24T22:02:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54858/0 (socket says 172.21.15.60:54858) 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 -- 172.21.15.60:0/3055930033 learned_addr learned my addr 172.21.15.60:0/3055930033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x5575122274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 -- 172.21.15.60:0/3055930033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 msgr2=0x5575122274f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x5575122274f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 -- 172.21.15.60:0/3055930033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512219170 msgr2=0x557512224bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512219170 0x557512224bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.265+0000 7fef8effd640 1 -- 172.21.15.60:0/3055930033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557512108320 con 0x557512218850 2024-07-24T22:02:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.266+0000 7fef8effd640 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef80016570 tx=0x7fef8000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44591c6b5b8c539 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.267+0000 7fef75ffb640 1 -- 172.21.15.60:0/3055930033 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80018430 con 0x557512218850 2024-07-24T22:02:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.267+0000 7fef75ffb640 1 -- 172.21.15.60:0/3055930033 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef80018a80 con 0x557512218850 2024-07-24T22:02:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.267+0000 7fef75ffb640 1 -- 172.21.15.60:0/3055930033 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800178a0 con 0x557512218850 2024-07-24T22:02:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.268+0000 7fef9847b740 1 -- 172.21.15.60:0/3055930033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 msgr2=0x557512218c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.268+0000 7fef9847b740 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef80016570 tx=0x7fef8000a330 comp rx=0 tx=0).stop 2024-07-24T22:02:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.269+0000 7fef8ffff640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 0x7fef780b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef9847b740 1 -- 172.21.15.60:0/3055930033 shutdown_connections 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef9847b740 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x5575122274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef9847b740 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512219170 0x557512224bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef9847b740 1 --2- 172.21.15.60:0/3055930033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512218c30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef9847b740 1 -- 172.21.15.60:0/3055930033 >> 172.21.15.60:0/3055930033 conn(0x557512216680 msgr2=0x557512217820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.272+0000 7fef8ffff640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 0x7fef780b6370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fef88041040 tx=0x7fef880c4040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.275+0000 7fef9847b740 1 -- 172.21.15.60:0/3055930033 shutdown_connections 2024-07-24T22:02:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.275+0000 7fef9847b740 1 -- 172.21.15.60:0/3055930033 wait complete. 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.275+0000 7fef9847b740 1 -- start start 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 Processor -- start 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 0x557512222090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x5575122225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 0x557512222090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x557512222b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557511fe2700 con 0x557512218850 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575121215f0 con 0x557512225100 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.276+0000 7fef9847b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557512227d20 con 0x557512219170 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.277+0000 7fef9847b740 10 client.0 _ll_register_callbacks cb 0x557512141960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.277+0000 7fef8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 0x557512222090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37612/0 (socket says 172.21.15.60:37612) 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.277+0000 7fef8effd640 1 -- 172.21.15.60:0/2460159302 learned_addr learned my addr 172.21.15.60:0/2460159302 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.277+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x5575122225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.277+0000 7fef8effd640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x557512222b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 -- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 msgr2=0x557512222b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x557512222b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 -- 172.21.15.60:0/2460159302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 msgr2=0x557512222090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 0x557512222090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 -- 172.21.15.60:0/2460159302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557512114680 con 0x557512219170 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x5575122225d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e600 tx=0x7fef8400eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcea1d33a8a0fce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef6effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004220 con 0x557512219170 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fef840043c0 con 0x557512219170 2024-07-24T22:02:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.278+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef80018a80 con 0x557512219170 2024-07-24T22:02:38.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.284+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fef84005020 con 0x557512219170 2024-07-24T22:02:38.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.284+0000 7fef9847b740 10 client.28009 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.284+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557511edf650 con 0x557512219170 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef840047a0 con 0x557512219170 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fef84010850 con 0x557512219170 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 10 client.28009 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 10 client.28009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 20 client.28009 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557512234d10 0x557512237180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 4 client.28009 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.288+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef80018a80 con 0x557512234d10 2024-07-24T22:02:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.289+0000 7fef8effd640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557512234d10 0x557512237180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:37 smithi086 ceph-mon[37128]: pgmap v877: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-07-24T22:02:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.296+0000 7fef8effd640 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557512234d10 0x557512237180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7206083d0bf755d6 server_cookie=c4ea85446dffedc7 in_seq=0 out_seq=0 2024-07-24T22:02:38.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.296+0000 7fef6effd640 10 client.28009 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:02:38.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef6effd640 1 -- 172.21.15.60:0/2460159302 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fef80018a80 con 0x557512234d10 2024-07-24T22:02:38.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef6effd640 10 client.28009 handle_command_reply: tid=0 2024-07-24T22:02:38.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 client.28009 shutdown 2024-07-24T22:02:38.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 msgr2=0x5575122225d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x5575122225d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e600 tx=0x7fef8400eac0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 shutdown_connections 2024-07-24T22:02:38.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557512234d10 0x557512237180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512225100 0x557512222b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x5575122225d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 --2- 172.21.15.60:0/2460159302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512218850 0x557512222090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef6effd640 0 client.28009 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.302+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 >> 172.21.15.60:0/2460159302 conn(0x557512216680 msgr2=0x55751221c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.303+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 shutdown_connections 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.303+0000 7fef9847b740 1 -- 172.21.15.60:0/2460159302 wait complete. 2024-07-24T22:02:38.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.304+0000 7fef9847b740 20 client.28009 trim_cache size 0 max 16384 2024-07-24T22:02:38.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.330+0000 7fef9847b740 1 Processor -- start 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 -- start start 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 0x55751221d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x557512223dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x55751223ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c011310 con 0x557512225100 2024-07-24T22:02:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00eb20 con 0x557512218850 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.335+0000 7fef9847b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00c990 con 0x557512219170 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.336+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x557512223dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.336+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x557512223dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54872/0 (socket says 172.21.15.60:54872) 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.336+0000 7fef8f7fe640 1 -- 172.21.15.60:0/1609712517 learned_addr learned my addr 172.21.15.60:0/1609712517 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.336+0000 7fef8effd640 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 0x55751221d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 -- 172.21.15.60:0/1609712517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 msgr2=0x557512223dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x557512223dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 -- 172.21.15.60:0/1609712517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 msgr2=0x55751223ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x55751223ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 -- 172.21.15.60:0/1609712517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557512220590 con 0x557512218850 2024-07-24T22:02:38.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.337+0000 7fef8effd640 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 0x55751221d340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fef80017420 tx=0x7fef80006d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b430f191a1c60e94 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.338+0000 7fef757fa640 1 -- 172.21.15.60:0/1609712517 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800187e0 con 0x557512218850 2024-07-24T22:02:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.338+0000 7fef757fa640 1 -- 172.21.15.60:0/1609712517 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef800044d0 con 0x557512218850 2024-07-24T22:02:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.338+0000 7fef757fa640 1 -- 172.21.15.60:0/1609712517 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80003c00 con 0x557512218850 2024-07-24T22:02:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.339+0000 7fef9847b740 1 -- 172.21.15.60:0/1609712517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 msgr2=0x55751221d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.339+0000 7fef9847b740 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 0x55751221d340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fef80017420 tx=0x7fef80006d50 comp rx=0 tx=0).stop 2024-07-24T22:02:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.340+0000 7fef9847b740 1 -- 172.21.15.60:0/1609712517 shutdown_connections 2024-07-24T22:02:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.340+0000 7fef9847b740 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x55751223ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.340+0000 7fef9847b740 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512219170 0x557512223dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.340+0000 7fef9847b740 1 --2- 172.21.15.60:0/1609712517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512218850 0x55751221d340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.340+0000 7fef9847b740 1 -- 172.21.15.60:0/1609712517 >> 172.21.15.60:0/1609712517 conn(0x557512216680 msgr2=0x5575122236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.343+0000 7fef9847b740 1 -- 172.21.15.60:0/1609712517 shutdown_connections 2024-07-24T22:02:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.343+0000 7fef9847b740 1 -- 172.21.15.60:0/1609712517 wait complete. 2024-07-24T22:02:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.343+0000 7fef9847b740 1 -- start start 2024-07-24T22:02:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.343+0000 7fef9847b740 1 Processor -- start 2024-07-24T22:02:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512219170 0x5575122294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x557512229a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00cfe0 con 0x557512225100 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00e450 con 0x557512219170 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00d6b0 con 0x557512218850 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef9847b740 10 client.0 _ll_register_callbacks cb 0x7fef84009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512219170 0x5575122294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef8effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef8effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54876/0 (socket says 172.21.15.60:54876) 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.344+0000 7fef8effd640 1 -- 172.21.15.60:0/3158279271 learned_addr learned my addr 172.21.15.60:0/3158279271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 -- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512219170 msgr2=0x5575122294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512219170 0x5575122294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 -- 172.21.15.60:0/3158279271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 msgr2=0x557512229a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x557512229a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 -- 172.21.15.60:0/3158279271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751229eb80 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.345+0000 7fef8effd640 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef8000a260 tx=0x7fef8000a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eeb66ea65b7273f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef6effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8003e060 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef800044d0 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7fef80003690 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fef800185e0 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef9847b740 10 client.28015 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.346+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5575122c1e70 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef800044d0 con 0x557512218850 2024-07-24T22:02:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fef80024040 con 0x557512218850 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 10 client.28015 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 10 client.28015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 20 client.28015 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5575122c2470 0x5575122c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 4 client.28015 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.347+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557512191900 con 0x5575122c2470 2024-07-24T22:02:38.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.348+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5575122c2470 0x5575122c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.354+0000 7fef6effd640 10 client.28015 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:02:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.354+0000 7fef8f7fe640 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5575122c2470 0x5575122c4890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7eedbef1597090c0 server_cookie=a8ad5273f34debbe in_seq=0 out_seq=0 2024-07-24T22:02:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.359+0000 7fef6effd640 1 -- 172.21.15.60:0/3158279271 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557512191900 con 0x5575122c2470 2024-07-24T22:02:38.364 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:02:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.359+0000 7fef6effd640 10 client.28015 handle_command_reply: tid=0 2024-07-24T22:02:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.359+0000 7fef9847b740 1 client.28015 shutdown 2024-07-24T22:02:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 msgr2=0x557512228f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef8000a260 tx=0x7fef8000a290 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 shutdown_connections 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x5575122c2470 0x5575122c4890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557512225100 0x557512229a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557512219170 0x5575122294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 --2- 172.21.15.60:0/3158279271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557512218850 0x557512228f90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 >> 172.21.15.60:0/3158279271 conn(0x557512216680 msgr2=0x557512223ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef6effd640 0 client.28015 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.360+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 shutdown_connections 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.361+0000 7fef9847b740 1 -- 172.21.15.60:0/3158279271 wait complete. 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.361+0000 7fef9847b740 20 client.28015 trim_cache size 0 max 16384 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 msgr2=0x7fef780b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 0x7fef780b6370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fef88041040 tx=0x7fef880c4040 comp rx=0 tx=0).stop 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 msgr2=0x7fef9006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fef880409c0 tx=0x7fef880409f0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 shutdown_connections 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fef780b3eb0 0x7fef780b6370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef90107850 0x7fef9006fb70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef90106ed0 0x7fef9006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 --2- 172.21.15.60:0/3406051105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef901014c0 0x7fef9010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.362+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 >> 172.21.15.60:0/3406051105 conn(0x7fef900fcf90 msgr2=0x7fef90103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.363+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 shutdown_connections 2024-07-24T22:02:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:38.363+0000 7fef94e68640 1 -- 172.21.15.60:0/3406051105 wait complete. 2024-07-24T22:02:39.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:02:39.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:39 smithi123 ceph-mon[37143]: pgmap v878: 97 pgs: 27 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 59 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-07-24T22:02:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:39 smithi086 ceph-mon[37128]: pgmap v878: 97 pgs: 27 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 59 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-07-24T22:02:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:39 smithi060 ceph-mon[33000]: pgmap v878: 97 pgs: 27 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 59 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-07-24T22:02:39.779 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:41 smithi086 ceph-mon[37128]: pgmap v879: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-07-24T22:02:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:41 smithi060 ceph-mon[33000]: pgmap v879: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-07-24T22:02:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:41 smithi123 ceph-mon[37143]: pgmap v879: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-07-24T22:02:42.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 -- 172.21.15.60:0/2602094867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d010b570 msgr2=0x7f63d0113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 --2- 172.21.15.60:0/2602094867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d010b570 0x7f63d0113cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009860 tx=0x7f63b802a3f0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 -- 172.21.15.60:0/2602094867 shutdown_connections 2024-07-24T22:02:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 --2- 172.21.15.60:0/2602094867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d0114200 0x7f63d0116630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 --2- 172.21.15.60:0/2602094867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d010b570 0x7f63d0113cc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 --2- 172.21.15.60:0/2602094867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d010b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.344+0000 7f63d574b640 1 -- 172.21.15.60:0/2602094867 >> 172.21.15.60:0/2602094867 conn(0x7f63d006d980 msgr2=0x7f63d006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.345+0000 7f63d574b640 1 -- 172.21.15.60:0/2602094867 shutdown_connections 2024-07-24T22:02:42.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.345+0000 7f63d574b640 1 -- 172.21.15.60:0/2602094867 wait complete. 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 Processor -- start 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 -- start start 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d010b570 0x7f63d01175d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d0114200 0x7f63d0117b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63d011afa0 con 0x7f63d010ac50 2024-07-24T22:02:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63d011ae20 con 0x7f63d0114200 2024-07-24T22:02:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.346+0000 7f63d574b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63d011b120 con 0x7f63d010b570 2024-07-24T22:02:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.347+0000 7f63ce7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d010b570 0x7f63d01175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.347+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.347+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37638/0 (socket says 172.21.15.60:37638) 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.347+0000 7f63ceffd640 1 -- 172.21.15.60:0/1729849438 learned_addr learned my addr 172.21.15.60:0/1729849438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.347+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d0114200 0x7f63d0117b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.348+0000 7f63ceffd640 1 -- 172.21.15.60:0/1729849438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d010b570 msgr2=0x7f63d01175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.348+0000 7f63ceffd640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d010b570 0x7f63d01175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.348+0000 7f63ceffd640 1 -- 172.21.15.60:0/1729849438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d0114200 msgr2=0x7f63d0117b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.348+0000 7f63ceffd640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d0114200 0x7f63d0117b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.348+0000 7f63ceffd640 1 -- 172.21.15.60:0/1729849438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d01af930 con 0x7f63d010ac50 2024-07-24T22:02:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.349+0000 7f63ceffd640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f63c000efb0 tx=0x7f63c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.349+0000 7f63c7fff640 1 -- 172.21.15.60:0/1729849438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c001b070 con 0x7f63d010ac50 2024-07-24T22:02:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.349+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d01afc20 con 0x7f63d010ac50 2024-07-24T22:02:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.350+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d01b0130 con 0x7f63d010ac50 2024-07-24T22:02:42.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.350+0000 7f63c7fff640 1 -- 172.21.15.60:0/1729849438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63c000ba20 con 0x7f63d010ac50 2024-07-24T22:02:42.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.350+0000 7f63c7fff640 1 -- 172.21.15.60:0/1729849438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0016b30 con 0x7f63d010ac50 2024-07-24T22:02:42.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.353+0000 7f63d7afc740 1 Processor -- start 2024-07-24T22:02:42.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 -- start start 2024-07-24T22:02:42.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb38ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56519baaf9b0 con 0x56519bb386d0 2024-07-24T22:02:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56519ba9eeb0 con 0x56519bb4d790 2024-07-24T22:02:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63d7afc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56519baa2ef0 con 0x56519bb38ff0 2024-07-24T22:02:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb38ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb38ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37652/0 (socket says 172.21.15.60:37652) 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63cf7fe640 1 -- 172.21.15.60:0/163556376 learned_addr learned my addr 172.21.15.60:0/163556376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.354+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 -- 172.21.15.60:0/163556376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 msgr2=0x56519bb4d250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb4d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 -- 172.21.15.60:0/163556376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 msgr2=0x56519bb38ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb38ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 -- 172.21.15.60:0/163556376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519ba28320 con 0x56519bb4d790 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.355+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb4fb80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009830 tx=0x7f63b8030550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85722fb801a2d30 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.356+0000 7f63c7fff640 1 -- 172.21.15.60:0/1729849438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f63c001e070 con 0x7f63d010ac50 2024-07-24T22:02:42.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.357+0000 7f63c57fa640 1 -- 172.21.15.60:0/163556376 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80309c0 con 0x56519bb4d790 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.357+0000 7f63c57fa640 1 -- 172.21.15.60:0/163556376 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b8005030 con 0x56519bb4d790 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.357+0000 7f63c57fa640 1 -- 172.21.15.60:0/163556376 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8004b60 con 0x56519bb4d790 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.358+0000 7f63c7fff640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 0x7f63bc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.358+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 0x7f63bc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.358+0000 7f63d7afc740 1 -- 172.21.15.60:0/163556376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 msgr2=0x56519bb4fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.358+0000 7f63d7afc740 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb4fb80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009830 tx=0x7f63b8030550 comp rx=0 tx=0).stop 2024-07-24T22:02:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.358+0000 7f63c7fff640 1 -- 172.21.15.60:0/1729849438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f63c00e3d30 con 0x7f63d010ac50 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.360+0000 7f63d7afc740 1 -- 172.21.15.60:0/163556376 shutdown_connections 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.360+0000 7f63d7afc740 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb4fb80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.360+0000 7f63d7afc740 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb4d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.360+0000 7f63d7afc740 1 --2- 172.21.15.60:0/163556376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb38ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.360+0000 7f63d7afc740 1 -- 172.21.15.60:0/163556376 >> 172.21.15.60:0/163556376 conn(0x56519bb36500 msgr2=0x56519bb376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.361+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 0x7f63bc0b63d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005fd0 tx=0x7f63c800b590 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.362+0000 7f63d7afc740 1 -- 172.21.15.60:0/163556376 shutdown_connections 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.362+0000 7f63d7afc740 1 -- 172.21.15.60:0/163556376 wait complete. 2024-07-24T22:02:42.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.363+0000 7f63d7afc740 1 -- start start 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.363+0000 7f63d7afc740 1 Processor -- start 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.363+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54898/0 (socket says 172.21.15.60:54898) 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63ceffd640 1 -- 172.21.15.60:0/3600668506 learned_addr learned my addr 172.21.15.60:0/3600668506 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb4b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56519b902700 con 0x56519bb4d790 2024-07-24T22:02:42.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56519ba415f0 con 0x56519bb38ff0 2024-07-24T22:02:42.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56519bb503e0 con 0x56519bb386d0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63d7afc740 10 client.0 _ll_register_callbacks cb 0x56519ba61960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.364+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb4b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 -- 172.21.15.60:0/3600668506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 msgr2=0x56519bb4ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb4ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 -- 172.21.15.60:0/3600668506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 msgr2=0x56519bb4b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb4b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519ba34680 con 0x56519bb386d0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63c000cd00 tx=0x7f63c00a61f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607e05cfe16887b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00c4060 con 0x56519bb386d0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.365+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f63c00b34a0 con 0x56519bb386d0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b8005030 con 0x56519bb386d0 2024-07-24T22:02:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f63c00e1850 con 0x56519bb386d0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 10 client.28024 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56519b7ff650 con 0x56519bb386d0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b8005030 con 0x56519bb386d0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f63c000b040 con 0x56519bb386d0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 10 client.28024 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 10 client.28024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 20 client.28024 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bb5d260 0x56519bb5f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 4 client.28024 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.366+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56519bab1900 con 0x56519bb5d260 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.367+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bb5d260 0x56519bb5f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.368+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bb5d260 0x56519bb5f6a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99234607d10abef0 server_cookie=fe02378a0f13c439 in_seq=0 out_seq=0 2024-07-24T22:02:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.368+0000 7f63b67fc640 10 client.28024 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.368+0000 7f63b67fc640 1 -- 172.21.15.60:0/3600668506 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56519bab1900 con 0x56519bb5d260 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.368+0000 7f63b67fc640 10 client.28024 handle_command_reply: tid=0 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.368+0000 7f63d7afc740 1 client.28024 shutdown 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 msgr2=0x56519bb4a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63c000cd00 tx=0x7f63c00a61f0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 shutdown_connections 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bb5d260 0x56519bb5f6a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb4b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3600668506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb4a720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 >> 172.21.15.60:0/3600668506 conn(0x56519bb36500 msgr2=0x56519bb3c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63b67fc640 0 client.28024 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 shutdown_connections 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.369+0000 7f63d7afc740 1 -- 172.21.15.60:0/3600668506 wait complete. 2024-07-24T22:02:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.370+0000 7f63d7afc740 20 client.28024 trim_cache size 0 max 16384 2024-07-24T22:02:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.390+0000 7f63d7afc740 1 Processor -- start 2024-07-24T22:02:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.390+0000 7f63d7afc740 1 -- start start 2024-07-24T22:02:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.391+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.391+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.391+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54908/0 (socket says 172.21.15.60:54908) 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb48220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37662/0 (socket says 172.21.15.60:37662) 2024-07-24T22:02:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 -- 172.21.15.60:0/128864048 learned_addr learned my addr 172.21.15.60:0/128864048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0011340 con 0x56519bb386d0 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000eb50 con 0x56519bb4d790 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000c9c0 con 0x56519bb38ff0 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 -- 172.21.15.60:0/128864048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 msgr2=0x56519bb3b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb3b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 -- 172.21.15.60:0/128864048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 msgr2=0x56519bb48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb48220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.392+0000 7f63ce7fc640 1 -- 172.21.15.60:0/128864048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519bb30a90 con 0x56519bb38ff0 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.393+0000 7f63ce7fc640 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56519bb38680 tx=0x7f63b802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35016eb40e10d5b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.393+0000 7f63c4ff9640 1 -- 172.21.15.60:0/128864048 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8004030 con 0x56519bb38ff0 2024-07-24T22:02:42.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.393+0000 7f63c4ff9640 1 -- 172.21.15.60:0/128864048 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b80035f0 con 0x56519bb38ff0 2024-07-24T22:02:42.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.393+0000 7f63c4ff9640 1 -- 172.21.15.60:0/128864048 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80516a0 con 0x56519bb38ff0 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 msgr2=0x56519bb35d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56519bb38680 tx=0x7f63b802a3f0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 shutdown_connections 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb4d790 0x56519bb3b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb38ff0 0x56519bb35d60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 --2- 172.21.15.60:0/128864048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb386d0 0x56519bb48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.394+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 >> 172.21.15.60:0/128864048 conn(0x56519bb36500 msgr2=0x56519bb4c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.395+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 shutdown_connections 2024-07-24T22:02:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.395+0000 7f63d7afc740 1 -- 172.21.15.60:0/128864048 wait complete. 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.395+0000 7f63d7afc740 1 -- start start 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 Processor -- start 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb55e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000d010 con 0x56519bb4d790 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000e480 con 0x56519bb38ff0 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000d6e0 con 0x56519bb386d0 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.396+0000 7f63d7afc740 10 client.0 _ll_register_callbacks cb 0x7f63c00aa150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63ce7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb55e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37668/0 (socket says 172.21.15.60:37668) 2024-07-24T22:02:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 -- 172.21.15.60:0/3842223412 learned_addr learned my addr 172.21.15.60:0/3842223412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 -- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 msgr2=0x56519bb558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 -- 172.21.15.60:0/3842223412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 msgr2=0x56519bb55e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb55e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.397+0000 7f63cf7fe640 1 -- 172.21.15.60:0/3842223412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519bb3af80 con 0x56519bb4d790 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.398+0000 7f63cf7fe640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f63c80127d0 tx=0x7f63c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf317ce5373aae36 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.398+0000 7f63b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.398+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c802e060 con 0x56519bb4d790 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.398+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b80035f0 con 0x56519bb4d790 2024-07-24T22:02:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.398+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f63c8020b40 con 0x56519bb4d790 2024-07-24T22:02:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.399+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f63c8012e10 con 0x56519bb4d790 2024-07-24T22:02:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.400+0000 7f63d7afc740 10 client.18774 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.400+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56519bb652e0 con 0x56519bb4d790 2024-07-24T22:02:42.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.400+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b80035f0 con 0x56519bb4d790 2024-07-24T22:02:42.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.400+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f63c8029410 con 0x56519bb4d790 2024-07-24T22:02:42.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 10 client.18774 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 20 client.18774 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bbc84b0 0x56519bbca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 4 client.18774 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.401+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56519baad3a0 con 0x56519bbc84b0 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.403+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bbc84b0 0x56519bbca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.404+0000 7f63ceffd640 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bbc84b0 0x56519bbca8d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3761c24b08e8b81 server_cookie=bcbbfda241244742 in_seq=0 out_seq=0 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.404+0000 7f63b67fc640 10 client.18774 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.404+0000 7f63b67fc640 1 -- 172.21.15.60:0/3842223412 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56519baad3a0 con 0x56519bbc84b0 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.404+0000 7f63b67fc640 10 client.18774 handle_command_reply: tid=0 2024-07-24T22:02:42.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.404+0000 7f63d7afc740 1 client.18774 shutdown 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.405+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 msgr2=0x56519bb3a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.405+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f63c80127d0 tx=0x7f63c80046c0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 shutdown_connections 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56519bbc84b0 0x56519bbca8d0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56519bb4d790 0x56519bb3a750 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56519bb38ff0 0x56519bb55e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 --2- 172.21.15.60:0/3842223412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56519bb386d0 0x56519bb558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 >> 172.21.15.60:0/3842223412 conn(0x56519bb36500 msgr2=0x56519bb48fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63b67fc640 0 client.18774 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:42.409 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:02:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 shutdown_connections 2024-07-24T22:02:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 1 -- 172.21.15.60:0/3842223412 wait complete. 2024-07-24T22:02:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.406+0000 7f63d7afc740 20 client.18774 trim_cache size 0 max 16384 2024-07-24T22:02:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 msgr2=0x7f63bc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 0x7f63bc0b63d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005fd0 tx=0x7f63c800b590 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 msgr2=0x7f63d0117090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f63c000efb0 tx=0x7f63c000c800 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 shutdown_connections 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f63bc0b3f10 0x7f63bc0b63d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d0114200 0x7f63d0117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d010b570 0x7f63d01175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 --2- 172.21.15.60:0/1729849438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d010ac50 0x7f63d0117090 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.407+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 >> 172.21.15.60:0/1729849438 conn(0x7f63d006d980 msgr2=0x7f63d0071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.408+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 shutdown_connections 2024-07-24T22:02:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:42.408+0000 7f63d574b640 1 -- 172.21.15.60:0/1729849438 wait complete. 2024-07-24T22:02:43.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:43 smithi060 ceph-mon[33000]: pgmap v880: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-07-24T22:02:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:43 smithi086 ceph-mon[37128]: pgmap v880: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-07-24T22:02:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:43 smithi123 ceph-mon[37143]: pgmap v880: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-07-24T22:02:44.353 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T22:02:44.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T22:02:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 -- 172.21.15.60:0/2627908762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a4103e80 msgr2=0x7f30a410c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 --2- 172.21.15.60:0/2627908762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a4103e80 0x7f30a410c460 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f308c009900 tx=0x7f308c02a410 comp rx=0 tx=0).stop 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 -- 172.21.15.60:0/2627908762 shutdown_connections 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 --2- 172.21.15.60:0/2627908762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a410ca30 0x7f30a410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 --2- 172.21.15.60:0/2627908762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a4103e80 0x7f30a410c460 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 --2- 172.21.15.60:0/2627908762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.882+0000 7f30a9694640 1 -- 172.21.15.60:0/2627908762 >> 172.21.15.60:0/2627908762 conn(0x7f30a40fd010 msgr2=0x7f30a40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.883+0000 7f30a9694640 1 -- 172.21.15.60:0/2627908762 shutdown_connections 2024-07-24T22:02:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.883+0000 7f30a9694640 1 -- 172.21.15.60:0/2627908762 wait complete. 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.883+0000 7f30a9694640 1 Processor -- start 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.883+0000 7f30a9694640 1 -- start start 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a4103e80 0x7f30a4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 0x7f30a4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4112630 con 0x7f30a410ca30 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41124b0 con 0x7f30a4103e80 2024-07-24T22:02:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a9694640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41127b0 con 0x7f30a4103480 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 0x7f30a4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54934/0 (socket says 172.21.15.60:54934) 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 -- 172.21.15.60:0/3218009290 learned_addr learned my addr 172.21.15.60:0/3218009290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f309bfff640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a4103e80 0x7f30a4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 -- 172.21.15.60:0/3218009290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a4103e80 msgr2=0x7f30a4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a4103e80 0x7f30a4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 -- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 msgr2=0x7f30a4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 0x7f30a4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a2ffd640 1 -- 172.21.15.60:0/3218009290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41a6fc0 con 0x7f30a4103480 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.884+0000 7f30a37fe640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 0x7f30a4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a2ffd640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f309400e9f0 tx=0x7f309400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a0ff9640 1 -- 172.21.15.60:0/3218009290 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309400cd80 con 0x7f30a4103480 2024-07-24T22:02:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a0ff9640 1 -- 172.21.15.60:0/3218009290 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30940054d0 con 0x7f30a4103480 2024-07-24T22:02:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a72b0 con 0x7f30a4103480 2024-07-24T22:02:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a0ff9640 1 -- 172.21.15.60:0/3218009290 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094010690 con 0x7f30a4103480 2024-07-24T22:02:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.885+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a7780 con 0x7f30a4103480 2024-07-24T22:02:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.888+0000 7f30ac3f2740 1 Processor -- start 2024-07-24T22:02:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 -- start start 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31429ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31435c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b3139f280 con 0x558b31436170 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b3138e780 con 0x558b314298c0 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b313927c0 con 0x558b3142a1e0 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f309bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31429ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f309bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37700/0 (socket says 172.21.15.60:37700) 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f309bfff640 1 -- 172.21.15.60:0/4019217094 learned_addr learned my addr 172.21.15.60:0/4019217094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.889+0000 7f30a0ff9640 1 -- 172.21.15.60:0/3218009290 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3094004e90 con 0x7f30a4103480 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 -- 172.21.15.60:0/4019217094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 msgr2=0x558b31435c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31435c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 -- 172.21.15.60:0/4019217094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 msgr2=0x558b31438560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 -- 172.21.15.60:0/4019217094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b310f2790 con 0x558b314298c0 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f309bfff640 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30a37fe640 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31429ca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26dde811c9e6b4b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30997fa640 1 -- 172.21.15.60:0/4019217094 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004d20 con 0x558b314298c0 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30997fa640 1 -- 172.21.15.60:0/4019217094 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c00f9a0 con 0x558b314298c0 2024-07-24T22:02:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.890+0000 7f30997fa640 1 -- 172.21.15.60:0/4019217094 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004470 con 0x558b314298c0 2024-07-24T22:02:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30a0ff9640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 0x7f30900b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30a0ff9640 1 -- 172.21.15.60:0/3218009290 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f30940e4440 con 0x7f30a4103480 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30a2ffd640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 0x7f30900b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 -- 172.21.15.60:0/4019217094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 msgr2=0x558b31429ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31429ca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c011bf0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 -- 172.21.15.60:0/4019217094 shutdown_connections 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31438560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31435c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/4019217094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31429ca0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.891+0000 7f30ac3f2740 1 -- 172.21.15.60:0/4019217094 >> 172.21.15.60:0/4019217094 conn(0x558b314276f0 msgr2=0x558b31428890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.892+0000 7f30ac3f2740 1 -- 172.21.15.60:0/4019217094 shutdown_connections 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.892+0000 7f30ac3f2740 1 -- 172.21.15.60:0/4019217094 wait complete. 2024-07-24T22:02:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.892+0000 7f30a2ffd640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 0x7f30900b6200 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x558b3142ca40 tx=0x7f3094004990 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.892+0000 7f30ac3f2740 1 -- start start 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.892+0000 7f30ac3f2740 1 Processor -- start 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31435800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31433220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558b312c8ab0 con 0x558b31436170 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b313169e0 con 0x558b314298c0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b31438ef0 con 0x558b3142a1e0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30ac3f2740 10 client.0 _ll_register_callbacks cb 0x558b3109ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54938/0 (socket says 172.21.15.60:54938) 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31435800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 -- 172.21.15.60:0/512846098 learned_addr learned my addr 172.21.15.60:0/512846098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f30a37fe640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31433220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 -- 172.21.15.60:0/512846098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 msgr2=0x558b31435800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31435800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 -- 172.21.15.60:0/512846098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 msgr2=0x558b31433220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31433220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.893+0000 7f309bfff640 1 -- 172.21.15.60:0/512846098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3116b620 con 0x558b3142a1e0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f309bfff640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f308c032d80 tx=0x7f308c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f779ad91cd01b230 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f308a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c035040 con 0x558b3142a1e0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f308c0316a0 con 0x558b3142a1e0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b3133dce0 con 0x558b3142a1e0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f308c0318a0 con 0x558b3142a1e0 2024-07-24T22:02:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f30ac3f2740 10 client.28036 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.894+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558b31444600 con 0x558b3142a1e0 2024-07-24T22:02:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558b3133dce0 con 0x558b3142a1e0 2024-07-24T22:02:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f308c004750 con 0x558b3142a1e0 2024-07-24T22:02:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 10 client.28036 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 10 client.28036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 20 client.28036 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b31445b30 0x558b31447f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 4 client.28036 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b313a11d0 con 0x558b31445b30 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f30a37fe640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b31445b30 0x558b31447f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.895+0000 7f308a7fc640 10 client.28036 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.896+0000 7f30a37fe640 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b31445b30 0x558b31447f20 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a591e411841e53 server_cookie=5a6731fef06b540a in_seq=0 out_seq=0 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.897+0000 7f308a7fc640 1 -- 172.21.15.60:0/512846098 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b313a11d0 con 0x558b31445b30 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.897+0000 7f308a7fc640 10 client.28036 handle_command_reply: tid=0 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.897+0000 7f30ac3f2740 1 client.28036 shutdown 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 msgr2=0x558b31432ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f308c032d80 tx=0x7f308c009680 comp rx=0 tx=0).stop 2024-07-24T22:02:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 shutdown_connections 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b31445b30 0x558b31447f20 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b31433220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b3142a1e0 0x558b31432ce0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f308a7fc640 0 client.28036 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/512846098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b314298c0 0x558b31435800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 >> 172.21.15.60:0/512846098 conn(0x558b314276f0 msgr2=0x558b3142d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 shutdown_connections 2024-07-24T22:02:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 1 -- 172.21.15.60:0/512846098 wait complete. 2024-07-24T22:02:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.898+0000 7f30ac3f2740 20 client.28036 trim_cache size 0 max 16384 2024-07-24T22:02:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 Processor -- start 2024-07-24T22:02:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 -- start start 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b31436170 0x558b3144d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b2cf0 con 0x558b314298c0 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b0500 con 0x558b3142a1e0 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.917+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ae370 con 0x558b31436170 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37728/0 (socket says 172.21.15.60:37728) 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 -- 172.21.15.60:0/1774934601 learned_addr learned my addr 172.21.15.60:0/1774934601 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f309bfff640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b31436170 0x558b3144d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 -- 172.21.15.60:0/1774934601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b31436170 msgr2=0x558b3144d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b31436170 0x558b3144d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 -- 172.21.15.60:0/1774934601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 msgr2=0x558b3142e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 -- 172.21.15.60:0/1774934601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31448290 con 0x558b314298c0 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f309bfff640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f30a37fe640 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f309c005c40 tx=0x7f309c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d97f7dfb1fc358 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f3098ff9640 1 -- 172.21.15.60:0/1774934601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c01e040 con 0x558b314298c0 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f3098ff9640 1 -- 172.21.15.60:0/1774934601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c00fe50 con 0x558b314298c0 2024-07-24T22:02:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.918+0000 7f3098ff9640 1 -- 172.21.15.60:0/1774934601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004860 con 0x558b314298c0 2024-07-24T22:02:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 -- 172.21.15.60:0/1774934601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 msgr2=0x558b314331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f309c005c40 tx=0x7f309c009500 comp rx=0 tx=0).stop 2024-07-24T22:02:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 -- 172.21.15.60:0/1774934601 shutdown_connections 2024-07-24T22:02:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b31436170 0x558b3144d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/1774934601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b314298c0 0x558b314331c0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 -- 172.21.15.60:0/1774934601 >> 172.21.15.60:0/1774934601 conn(0x558b314276f0 msgr2=0x558b3142dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 -- 172.21.15.60:0/1774934601 shutdown_connections 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.919+0000 7f30ac3f2740 1 -- 172.21.15.60:0/1774934601 wait complete. 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.920+0000 7f30ac3f2740 1 -- start start 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.920+0000 7f30ac3f2740 1 Processor -- start 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.920+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.920+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30ac3f2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b3142f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ae9c0 con 0x558b31436170 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30ac3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41afe30 con 0x558b3142a1e0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54952/0 (socket says 172.21.15.60:54952) 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 -- 172.21.15.60:0/2928607499 learned_addr learned my addr 172.21.15.60:0/2928607499 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f309bfff640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41af090 con 0x558b314298c0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b3142f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30ac3f2740 10 client.0 _ll_register_callbacks cb 0x7f3094005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 -- 172.21.15.60:0/2928607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 msgr2=0x558b3142eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 -- 172.21.15.60:0/2928607499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 msgr2=0x558b3142f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b3142f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 -- 172.21.15.60:0/2928607499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3143d1b0 con 0x558b314298c0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f30a37fe640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f309c004f30 tx=0x7f309c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfc8ae94618c621 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.921+0000 7f308a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c02d070 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f309c00fe50 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f309c004130 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f309c0045e0 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 10 client.28042 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558b3143e440 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c00fe50 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f309c024040 con 0x558b314298c0 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 10 client.28042 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 10 client.28042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 20 client.28042 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b3143e8c0 0x558b314b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 4 client.28042 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:02:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.922+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b3139cc70 con 0x558b3143e8c0 2024-07-24T22:02:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.923+0000 7f309bfff640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b3143e8c0 0x558b314b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.923+0000 7f308a7fc640 10 client.28042 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.923+0000 7f309bfff640 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b3143e8c0 0x558b314b2810 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa1ac21ffd4efc4 server_cookie=9a182df2ea830ad9 in_seq=0 out_seq=0 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f308a7fc640 1 -- 172.21.15.60:0/2928607499 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b3139cc70 con 0x558b3143e8c0 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f308a7fc640 10 client.28042 handle_command_reply: tid=0 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 client.28042 shutdown 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 msgr2=0x558b31430810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f309c004f30 tx=0x7f309c005bd0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 shutdown_connections 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558b3143e8c0 0x558b314b2810 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f308a7fc640 0 client.28042 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b31436170 0x558b3142f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b3142a1e0 0x558b3142eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 --2- 172.21.15.60:0/2928607499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b314298c0 0x558b31430810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.933+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 >> 172.21.15.60:0/2928607499 conn(0x558b314276f0 msgr2=0x558b3142e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 shutdown_connections 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30ac3f2740 1 -- 172.21.15.60:0/2928607499 wait complete. 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30ac3f2740 20 client.28042 trim_cache size 0 max 16384 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:02:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:02:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 msgr2=0x7f30900b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 0x7f30900b6200 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x558b3142ca40 tx=0x7f3094004990 comp rx=0 tx=0).stop 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 msgr2=0x7f30a4101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.934+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f309400e9f0 tx=0x7f309400edf0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 shutdown_connections 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f30900b3d40 0x7f30900b6200 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30a410ca30 0x7f30a4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a4103e80 0x7f30a4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 --2- 172.21.15.60:0/3218009290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30a4103480 0x7f30a4101300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 >> 172.21.15.60:0/3218009290 conn(0x7f30a40fd010 msgr2=0x7f30a40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 shutdown_connections 2024-07-24T22:02:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:44.935+0000 7f30a9694640 1 -- 172.21.15.60:0/3218009290 wait complete. 2024-07-24T22:02:45.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:45 smithi060 ceph-mon[33000]: pgmap v881: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-07-24T22:02:45.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:45 smithi086 ceph-mon[37128]: pgmap v881: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-07-24T22:02:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:45 smithi123 ceph-mon[37143]: pgmap v881: 97 pgs: 18 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 -- 172.21.15.60:0/1275229582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108160 msgr2=0x7f8e101085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 --2- 172.21.15.60:0/1275229582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108160 0x7f8e101085a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009900 tx=0x7f8e0402a410 comp rx=0 tx=0).stop 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 -- 172.21.15.60:0/1275229582 shutdown_connections 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 --2- 172.21.15.60:0/1275229582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108ae0 0x7f8e1010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 --2- 172.21.15.60:0/1275229582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108160 0x7f8e101085a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 --2- 172.21.15.60:0/1275229582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 -- 172.21.15.60:0/1275229582 >> 172.21.15.60:0/1275229582 conn(0x7f8e100fe200 msgr2=0x7f8e10100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 -- 172.21.15.60:0/1275229582 shutdown_connections 2024-07-24T22:02:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.074+0000 7f8e14d00640 1 -- 172.21.15.60:0/1275229582 wait complete. 2024-07-24T22:02:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.075+0000 7f8e14d00640 1 Processor -- start 2024-07-24T22:02:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.075+0000 7f8e14d00640 1 -- start start 2024-07-24T22:02:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108160 0x7f8e101991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112730 con 0x7f8e10108ae0 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101125b0 con 0x7f8e10102750 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e14d00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101128b0 con 0x7f8e10108160 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108160 0x7f8e101991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e0ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e0ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49464/0 (socket says 172.21.15.60:49464) 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e0ed9d640 1 -- 172.21.15.60:0/3012663674 learned_addr learned my addr 172.21.15.60:0/3012663674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.076+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 -- 172.21.15.60:0/3012663674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108160 msgr2=0x7f8e101991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108160 0x7f8e101991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 -- 172.21.15.60:0/3012663674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 msgr2=0x7f8e10198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 -- 172.21.15.60:0/3012663674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1019ed10 con 0x7f8e10108ae0 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:02:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800eb10 tx=0x7f8e0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8dff7fe640 1 -- 172.21.15.60:0/3012663674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004b20 con 0x7f8e10108ae0 2024-07-24T22:02:47.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1019f000 con 0x7f8e10108ae0 2024-07-24T22:02:47.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8dff7fe640 1 -- 172.21.15.60:0/3012663674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e08004cc0 con 0x7f8e10108ae0 2024-07-24T22:02:47.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.077+0000 7f8dff7fe640 1 -- 172.21.15.60:0/3012663674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08010720 con 0x7f8e10108ae0 2024-07-24T22:02:47.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.078+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a56c0 con 0x7f8e10108ae0 2024-07-24T22:02:47.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.081+0000 7f8dff7fe640 1 -- 172.21.15.60:0/3012663674 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8e08005020 con 0x7f8e10108ae0 2024-07-24T22:02:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.082+0000 7f8e170b1740 1 Processor -- start 2024-07-24T22:02:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.082+0000 7f8e170b1740 1 -- start start 2024-07-24T22:02:47.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137757fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49478/0 (socket says 172.21.15.60:49478) 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41656/0 (socket says 172.21.15.60:41656) 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0e59c640 1 -- 172.21.15.60:0/475407043 learned_addr learned my addr 172.21.15.60:0/475407043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5641376bc9f0 con 0x564137745840 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641376abef0 con 0x564137744f20 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641376aff30 con 0x564137755be0 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.083+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137757fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 -- 172.21.15.60:0/475407043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 msgr2=0x564137757fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137757fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 -- 172.21.15.60:0/475407043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 msgr2=0x564137745300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 -- 172.21.15.60:0/475407043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56413740c930 con 0x564137745840 2024-07-24T22:02:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009d80 tx=0x7f8e000117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595f42dc14c149dd server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8dfcff9640 1 -- 172.21.15.60:0/475407043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0001b670 con 0x564137745840 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8dfcff9640 1 -- 172.21.15.60:0/475407043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00016070 con 0x564137745840 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.084+0000 7f8dfcff9640 1 -- 172.21.15.60:0/475407043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00012a20 con 0x564137745840 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.085+0000 7f8dff7fe640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 0x7f8df80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.085+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 0x7f8df80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.085+0000 7f8dff7fe640 1 -- 172.21.15.60:0/3012663674 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8e080e4620 con 0x7f8e10108ae0 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 0x7f8df80b6280 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005ab0 tx=0x7f8e04005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 msgr2=0x5641377556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009d80 tx=0x7f8e000117b0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 shutdown_connections 2024-07-24T22:02:47.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137757fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377556a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 --2- 172.21.15.60:0/475407043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137745300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 >> 172.21.15.60:0/475407043 conn(0x564137742d50 msgr2=0x564137743ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 shutdown_connections 2024-07-24T22:02:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.086+0000 7f8e170b1740 1 -- 172.21.15.60:0/475407043 wait complete. 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.087+0000 7f8e170b1740 1 -- start start 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.087+0000 7f8e170b1740 1 Processor -- start 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.087+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 0x564137752bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.087+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 0x564137753100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137755be0 0x564137753640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e0e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 0x564137752bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e0e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 0x564137752bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48190/0 (socket says 172.21.15.60:48190) 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e0e59c640 1 -- 172.21.15.60:0/3458663635 learned_addr learned my addr 172.21.15.60:0/3458663635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:47.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56413750f890 con 0x564137755be0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56413764e390 con 0x564137745840 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564137758890 con 0x564137744f20 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137755be0 0x564137753640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e170b1740 10 client.0 _ll_register_callbacks cb 0x564137658500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.088+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 0x564137753100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 msgr2=0x564137752bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 0x564137752bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3458663635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137755be0 msgr2=0x564137753640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137755be0 0x564137753640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564137648d60 con 0x564137745840 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 0x564137753100 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8e0402aa00 tx=0x7f8e04005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7f85898a32a523 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.089+0000 7f8df5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:02:47.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.090+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0404f020 con 0x564137745840 2024-07-24T22:02:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.090+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e00016070 con 0x564137745840 2024-07-24T22:02:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.090+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8e04004970 con 0x564137745840 2024-07-24T22:02:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.090+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8e04035040 con 0x564137745840 2024-07-24T22:02:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.094+0000 7f8e170b1740 10 client.28169 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.094+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564137764850 con 0x564137745840 2024-07-24T22:02:47.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.096+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e040035f0 con 0x564137745840 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f8e04004b70 con 0x564137745840 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 10 client.28169 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 10 client.28169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 20 client.28169 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564137765e00 0x564137768220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 4 client.28169 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.098+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8e00016070 con 0x564137765e00 2024-07-24T22:02:47.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.099+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564137765e00 0x564137768220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.099+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564137765e00 0x564137768220 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de22e94293782ab7 server_cookie=2169bf444571bb3a in_seq=0 out_seq=0 2024-07-24T22:02:47.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.100+0000 7f8df5ffb640 10 client.28169 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.100+0000 7f8df5ffb640 1 -- 172.21.15.60:0/3458663635 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8e00016070 con 0x564137765e00 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.100+0000 7f8df5ffb640 10 client.28169 handle_command_reply: tid=0 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.100+0000 7f8e170b1740 1 client.28169 shutdown 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 msgr2=0x564137753100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 0x564137753100 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8e0402aa00 tx=0x7f8e04005a20 comp rx=0 tx=0).stop 2024-07-24T22:02:47.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 shutdown_connections 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564137765e00 0x564137768220 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8df5ffb640 0 client.28169 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137755be0 0x564137753640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137745840 0x564137753100 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3458663635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137744f20 0x564137752bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.101+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 >> 172.21.15.60:0/3458663635 conn(0x564137742d50 msgr2=0x564137748880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.102+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 shutdown_connections 2024-07-24T22:02:47.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.102+0000 7f8e170b1740 1 -- 172.21.15.60:0/3458663635 wait complete. 2024-07-24T22:02:47.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.102+0000 7f8e170b1740 20 client.28169 trim_cache size 0 max 16384 2024-07-24T22:02:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 Processor -- start 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 -- start start 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137750670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137741b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101ae940 con 0x564137745840 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101ac150 con 0x564137744f20 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e170b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101a9fc0 con 0x564137755be0 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49490/0 (socket says 172.21.15.60:49490) 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3816527610 learned_addr learned my addr 172.21.15.60:0/3816527610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137750670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137741b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3816527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 msgr2=0x564137741b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.127+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137741b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3816527610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 msgr2=0x564137750670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137750670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/3816527610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564137755030 con 0x564137745840 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137750670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137741b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8e04004620 tx=0x7f8e0403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6457998dd8516d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8df7fff640 1 -- 172.21.15.60:0/3816527610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0404f070 con 0x564137745840 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8df7fff640 1 -- 172.21.15.60:0/3816527610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04061070 con 0x564137745840 2024-07-24T22:02:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.128+0000 7f8df7fff640 1 -- 172.21.15.60:0/3816527610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040035f0 con 0x564137745840 2024-07-24T22:02:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- 172.21.15.60:0/3816527610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 msgr2=0x5641377415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8e04004620 tx=0x7f8e0403a000 comp rx=0 tx=0).stop 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- 172.21.15.60:0/3816527610 shutdown_connections 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x564137741b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x5641377415f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 --2- 172.21.15.60:0/3816527610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x564137750670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- 172.21.15.60:0/3816527610 >> 172.21.15.60:0/3816527610 conn(0x564137742d50 msgr2=0x564137744250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- 172.21.15.60:0/3816527610 shutdown_connections 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- 172.21.15.60:0/3816527610 wait complete. 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 -- start start 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.129+0000 7f8e170b1740 1 Processor -- start 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x56413775db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x56413775e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x56413775aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x56413775e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101aa610 con 0x564137745840 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101aba80 con 0x564137744f20 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101aace0 con 0x564137755be0 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x56413775db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x56413775e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49498/0 (socket says 172.21.15.60:49498) 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0dd9b640 1 -- 172.21.15.60:0/1855934605 learned_addr learned my addr 172.21.15.60:0/1855934605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e170b1740 10 client.0 _ll_register_callbacks cb 0x7f8e04004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0ed9d640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x56413775aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 -- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 msgr2=0x56413775aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x56413775aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 -- 172.21.15.60:0/1855934605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 msgr2=0x56413775e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x56413775e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 -- 172.21.15.60:0/1855934605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56413776e670 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8e0e59c640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x56413775db20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x564137768a60 tx=0x7f8e000124e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489dd6dd429483f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8df5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.130+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000054d0 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e04061070 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7f8e000035f0 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8e00003810 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 10 client.28175 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5641377d1060 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04061070 con 0x564137744f20 2024-07-24T22:02:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f8e00012c50 con 0x564137744f20 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 10 client.28175 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 10 client.28175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 20 client.28175 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5641377d15f0 0x5641377d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 4 client.28175 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641376be940 con 0x5641377d15f0 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.131+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5641377d15f0 0x5641377d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8e0dd9b640 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5641377d15f0 0x5641377d3a10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6809c95894de300b server_cookie=92b7bf3a6ffa9873 in_seq=0 out_seq=0 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8df5ffb640 10 client.28175 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8df5ffb640 1 -- 172.21.15.60:0/1855934605 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641376be940 con 0x5641377d15f0 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8df5ffb640 10 client.28175 handle_command_reply: tid=0 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8e170b1740 1 client.28175 shutdown 2024-07-24T22:02:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 msgr2=0x56413775db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.132+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x56413775db20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x564137768a60 tx=0x7f8e000124e0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 shutdown_connections 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5641377d15f0 0x5641377d3a10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564137755be0 0x56413775aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8df5ffb640 0 client.28175 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564137745840 0x56413775e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 --2- 172.21.15.60:0/1855934605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564137744f20 0x56413775db20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 >> 172.21.15.60:0/1855934605 conn(0x564137742d50 msgr2=0x564137748060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 shutdown_connections 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 1 -- 172.21.15.60:0/1855934605 wait complete. 2024-07-24T22:02:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e170b1740 20 client.28175 trim_cache size 0 max 16384 2024-07-24T22:02:47.141 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:02:47.141 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T22:02:47.141 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "6208259a-9073-43bf-98a0-c5047ff832f5", 2024-07-24T22:02:47.141 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T22:02:47.141 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 msgr2=0x7f8df80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 0x7f8df80b6280 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005ab0 tx=0x7f8e04005fd0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.133+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 msgr2=0x7f8e1019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800eb10 tx=0x7f8e0800c2d0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 shutdown_connections 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8df80b3dc0 0x7f8df80b6280 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e10108ae0 0x7f8e1019e540 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e10108160 0x7f8e101991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 --2- 172.21.15.60:0/3012663674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10102750 0x7f8e10198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 >> 172.21.15.60:0/3012663674 conn(0x7f8e100fe200 msgr2=0x7f8e101005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 shutdown_connections 2024-07-24T22:02:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:47.134+0000 7f8e14d00640 1 -- 172.21.15.60:0/3012663674 wait complete. 2024-07-24T22:02:47.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:47 smithi060 ceph-mon[33000]: pgmap v882: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 185 op/s 2024-07-24T22:02:47.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:47 smithi086 ceph-mon[37128]: pgmap v882: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 185 op/s 2024-07-24T22:02:47.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:02:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:47 smithi123 ceph-mon[37143]: pgmap v882: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 185 op/s 2024-07-24T22:02:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:48.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:48 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T22:02:48.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:48 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:48.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:48 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:48.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:48 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:02:48.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:48 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:48 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T22:02:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:48 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:48 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:48 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:02:48.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:48 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:48 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T22:02:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:48 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:48 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:48 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:02:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:48 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:49.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:49 smithi060 ceph-mon[33000]: pgmap v883: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-07-24T22:02:49.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:49 smithi086 ceph-mon[37128]: pgmap v883: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-07-24T22:02:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:49 smithi123 ceph-mon[37143]: pgmap v883: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-07-24T22:02:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 -- 172.21.15.60:0/1359330246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4101650 msgr2=0x7ff1e4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1359330246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4101650 0x7ff1e4101a30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009900 tx=0x7ff1d402a410 comp rx=0 tx=0).stop 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 -- 172.21.15.60:0/1359330246 shutdown_connections 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1359330246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41079e0 0x7ff1e410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1359330246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4107060 0x7ff1e41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1359330246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4101650 0x7ff1e4101a30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 -- 172.21.15.60:0/1359330246 >> 172.21.15.60:0/1359330246 conn(0x7ff1e40fd0c0 msgr2=0x7ff1e40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 -- 172.21.15.60:0/1359330246 shutdown_connections 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.401+0000 7ff1e9118640 1 -- 172.21.15.60:0/1359330246 wait complete. 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 Processor -- start 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 -- start start 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4107060 0x7ff1e4199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 0x7ff1e419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4111610 con 0x7ff1e41079e0 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4111490 con 0x7ff1e4107060 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e9118640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4111790 con 0x7ff1e4101650 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e2d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 0x7ff1e419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e2d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48228/0 (socket says 172.21.15.60:48228) 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1e2d9d640 1 -- 172.21.15.60:0/1975275366 learned_addr learned my addr 172.21.15.60:0/1975275366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.402+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4107060 0x7ff1e4199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 -- 172.21.15.60:0/1975275366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4107060 msgr2=0x7ff1e4199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4107060 0x7ff1e4199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 -- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 msgr2=0x7ff1e419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 0x7ff1e419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 -- 172.21.15.60:0/1975275366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e41a4e20 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 0x7ff1e419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005940 tx=0x7ff1d40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e0d99640 1 -- 172.21.15.60:0/1975275366 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d403f030 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e41a50b0 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e0d99640 1 -- 172.21.15.60:0/1975275366 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1d40044c0 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e0d99640 1 -- 172.21.15.60:0/1975275366 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4035070 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.403+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a55e0 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1e0d99640 1 -- 172.21.15.60:0/1975275366 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff1d4042030 con 0x7ff1e4101650 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 Processor -- start 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 -- start start 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a902ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a9038f30 0x55a0a903b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8fa3960 con 0x55a0a902cfa0 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8f92e60 con 0x55a0a9038f30 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8f96ea0 con 0x55a0a902c680 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a902ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49526/0 (socket says 172.21.15.60:49526) 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1e2d9d640 1 -- 172.21.15.60:0/683626887 learned_addr learned my addr 172.21.15.60:0/683626887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.406+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a9038f30 0x55a0a903b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 -- 172.21.15.60:0/683626887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a9038f30 msgr2=0x55a0a903b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a9038f30 0x55a0a903b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 -- 172.21.15.60:0/683626887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 msgr2=0x55a0a90389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 -- 172.21.15.60:0/683626887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a8f1c2d0 con 0x55a0a902c680 2024-07-24T22:02:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e359e640 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a902ca60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff1e419aa20 tx=0x7ff1dc011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912b57b7246c6059 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e0d99640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 0x7ff1d00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1d97fa640 1 -- 172.21.15.60:0/683626887 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004d20 con 0x55a0a902c680 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1d97fa640 1 -- 172.21.15.60:0/683626887 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc003970 con 0x55a0a902c680 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1d97fa640 1 -- 172.21.15.60:0/683626887 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004380 con 0x55a0a902c680 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 0x7ff1d00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.407+0000 7ff1e0d99640 1 -- 172.21.15.60:0/1975275366 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7ff1d4104380 con 0x7ff1e4101650 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1e2d9d640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 0x7ff1d00b6200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006010 tx=0x7ff1d4107ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/683626887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 msgr2=0x55a0a902ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a902ca60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff1e419aa20 tx=0x7ff1dc011bf0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/683626887 shutdown_connections 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a9038f30 0x55a0a903b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902cfa0 0x55a0a90389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/683626887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a902ca60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/683626887 >> 172.21.15.60:0/683626887 conn(0x55a0a902a4b0 msgr2=0x55a0a902b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/683626887 shutdown_connections 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/683626887 wait complete. 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.408+0000 7ff1eb4c9740 1 -- start start 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 Processor -- start 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a90363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a90368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8df6700 con 0x55a0a902c680 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8f355a0 con 0x55a0a902cfa0 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a0a903bbb0 con 0x55a0a9038f30 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1eb4c9740 10 client.0 _ll_register_callbacks cb 0x55a0a8f55910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49538/0 (socket says 172.21.15.60:49538) 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a90363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1e359e640 1 -- 172.21.15.60:0/1362252098 learned_addr learned my addr 172.21.15.60:0/1362252098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a90368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 -- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 msgr2=0x55a0a90368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a90368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 -- 172.21.15.60:0/1362252098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 msgr2=0x55a0a9035e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1dbfff640 1 -- 172.21.15.60:0/1362252098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a8f28630 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.409+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a90363a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0126d0 tx=0x7ff1cc012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b71335bdc014a65 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004020 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1dc003970 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7ff1cc0041c0 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc0043c0 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 10 client.28187 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a0a8cf3650 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc003970 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff1cc003ae0 con 0x55a0a902cfa0 2024-07-24T22:02:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 10 client.28187 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 10 client.28187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 20 client.28187 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.410+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a9048a70 0x55a0a904aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.411+0000 7ff1eb4c9740 4 client.28187 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.411+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a8fa58b0 con 0x55a0a9048a70 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.411+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a9048a70 0x55a0a904aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.411+0000 7ff1ca7fc640 10 client.28187 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.411+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a9048a70 0x55a0a904aec0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b99c0aac822c26 server_cookie=766d1bcd10c5b1fa in_seq=0 out_seq=0 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1362252098 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0a8fa58b0 con 0x55a0a9048a70 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1ca7fc640 10 client.28187 handle_command_reply: tid=0 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 client.28187 shutdown 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 msgr2=0x55a0a90363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a90363a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0126d0 tx=0x7ff1cc012b90 comp rx=0 tx=0).stop 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 shutdown_connections 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a9048a70 0x55a0a904aec0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a90368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a90363a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1ca7fc640 0 client.28187 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1362252098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a9035e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 >> 172.21.15.60:0/1362252098 conn(0x55a0a902a4b0 msgr2=0x55a0a9030050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 shutdown_connections 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1362252098 wait complete. 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.412+0000 7ff1eb4c9740 20 client.28187 trim_cache size 0 max 16384 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 Processor -- start 2024-07-24T22:02:50.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 -- start start 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9050d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a9033070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b0b60 con 0x55a0a902c680 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41ae370 con 0x55a0a902cfa0 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41ac1e0 con 0x55a0a9038f30 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49554/0 (socket says 172.21.15.60:49554) 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "6208259a-9073-43bf-98a0-c5047ff832f5": { 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "6208259a-9073-43bf-98a0-c5047ff832f5", 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:02:50.585 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 -- 172.21.15.60:0/4003117313 learned_addr learned my addr 172.21.15.60:0/4003117313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9050d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a9033070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 -- 172.21.15.60:0/4003117313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 msgr2=0x55a0a9033070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a9033070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 -- 172.21.15.60:0/4003117313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 msgr2=0x55a0a9050d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9050d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.432+0000 7ff1e359e640 1 -- 172.21.15.60:0/4003117313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a902cdd0 con 0x55a0a902c680 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1e359e640 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc011730 tx=0x7ff1dc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa591984f5d44260 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1d8ff9640 1 -- 172.21.15.60:0/4003117313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc01d070 con 0x55a0a902c680 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1d8ff9640 1 -- 172.21.15.60:0/4003117313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc01e040 con 0x55a0a902c680 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1d8ff9640 1 -- 172.21.15.60:0/4003117313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc019690 con 0x55a0a902c680 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/4003117313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 msgr2=0x55a0a90507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.433+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc011730 tx=0x7ff1dc011440 comp rx=0 tx=0).stop 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/4003117313 shutdown_connections 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a9038f30 0x55a0a9033070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9050d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/4003117313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a902c680 0x55a0a90507d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/4003117313 >> 172.21.15.60:0/4003117313 conn(0x55a0a902a4b0 msgr2=0x55a0a90346e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/4003117313 shutdown_connections 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/4003117313 wait complete. 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- start start 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 Processor -- start 2024-07-24T22:02:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a9040e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9041380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41ac830 con 0x55a0a9038f30 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41adca0 con 0x55a0a902cfa0 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41acf00 con 0x55a0a902c680 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1eb4c9740 10 client.0 _ll_register_callbacks cb 0x7ff1cc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49562/0 (socket says 172.21.15.60:49562) 2024-07-24T22:02:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1e359e640 1 -- 172.21.15.60:0/1636859566 learned_addr learned my addr 172.21.15.60:0/1636859566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9041380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.434+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a9040e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 -- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 msgr2=0x55a0a9040e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a9040e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 -- 172.21.15.60:0/1636859566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 msgr2=0x55a0a9041380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9041380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 -- 172.21.15.60:0/1636859566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a90ad050 con 0x55a0a9038f30 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1e359e640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003ef0 tx=0x7ff1dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50891b7d70b483fc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc01d040 con 0x55a0a9038f30 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8609+0+0 (secure 0 0 0) 0x7ff1dc0044c0 con 0x55a0a9038f30 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1dc01e040 con 0x55a0a9038f30 2024-07-24T22:02:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff1dc037450 con 0x55a0a9038f30 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1eb4c9740 10 client.18828 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a0a90b8400 con 0x55a0a9038f30 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.435+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc01e040 con 0x55a0a9038f30 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff1dc036670 con 0x55a0a9038f30 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 10 client.18828 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 20 client.18828 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a90b8880 0x55a0a90baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 4 client.18828 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0a8fa1350 con 0x55a0a90b8880 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a90b8880 0x55a0a90baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1dbfff640 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a90b8880 0x55a0a90baca0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aae98c02ea31694 server_cookie=d218ea2bfb0b9cb7 in_seq=0 out_seq=0 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.436+0000 7ff1ca7fc640 10 client.18828 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1ca7fc640 1 -- 172.21.15.60:0/1636859566 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a0a8fa1350 con 0x55a0a90b8880 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1ca7fc640 10 client.18828 handle_command_reply: tid=0 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 client.18828 shutdown 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 msgr2=0x55a0a90acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003ef0 tx=0x7ff1dc004770 comp rx=0 tx=0).stop 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 shutdown_connections 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55a0a90b8880 0x55a0a90baca0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a9038f30 0x55a0a90acb10 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a902cfa0 0x55a0a9041380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1ca7fc640 0 client.18828 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 --2- 172.21.15.60:0/1636859566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a0a902c680 0x55a0a9040e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 >> 172.21.15.60:0/1636859566 conn(0x55a0a902a4b0 msgr2=0x55a0a9034b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 shutdown_connections 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 1 -- 172.21.15.60:0/1636859566 wait complete. 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1eb4c9740 20 client.18828 trim_cache size 0 max 16384 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 msgr2=0x7ff1d00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 0x7ff1d00b6200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006010 tx=0x7ff1d4107ec0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 msgr2=0x7ff1e4199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.437+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005940 tx=0x7ff1d40040c0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 shutdown_connections 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff1d00b3d40 0x7ff1d00b6200 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e41079e0 0x7ff1e419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4107060 0x7ff1e4199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 --2- 172.21.15.60:0/1975275366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e4101650 0x7ff1e4199a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 >> 172.21.15.60:0/1975275366 conn(0x7ff1e40fd0c0 msgr2=0x7ff1e40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 shutdown_connections 2024-07-24T22:02:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:02:50.438+0000 7ff1e9118640 1 -- 172.21.15.60:0/1975275366 wait complete. 2024-07-24T22:02:51.488 INFO:tasks.cephfs.filesystem:scrub status for tag:6208259a-9073-43bf-98a0-c5047ff832f5 - {'path': '/', 'tag': '6208259a-9073-43bf-98a0-c5047ff832f5', 'options': 'recursive,force'} 2024-07-24T22:02:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:51 smithi086 ceph-mon[37128]: pgmap v884: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-07-24T22:02:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:51 smithi060 ceph-mon[33000]: pgmap v884: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-07-24T22:02:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:51 smithi123 ceph-mon[37143]: pgmap v884: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-07-24T22:02:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:53 smithi086 ceph-mon[37128]: pgmap v885: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 563 op/s 2024-07-24T22:02:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:53 smithi086 ceph-mon[37128]: osdmap e110: 12 total, 12 up, 12 in 2024-07-24T22:02:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:53 smithi060 ceph-mon[33000]: pgmap v885: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 563 op/s 2024-07-24T22:02:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:53 smithi060 ceph-mon[33000]: osdmap e110: 12 total, 12 up, 12 in 2024-07-24T22:02:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:53 smithi123 ceph-mon[37143]: pgmap v885: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 563 op/s 2024-07-24T22:02:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:53 smithi123 ceph-mon[37143]: osdmap e110: 12 total, 12 up, 12 in 2024-07-24T22:02:54.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:02:55.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:55 smithi086 ceph-mon[37128]: pgmap v887: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 1.10k op/s 2024-07-24T22:02:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:55 smithi060 ceph-mon[33000]: pgmap v887: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 1.10k op/s 2024-07-24T22:02:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:55 smithi123 ceph-mon[37143]: pgmap v887: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 1.10k op/s 2024-07-24T22:02:57.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:57 smithi086 ceph-mon[37128]: pgmap v888: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-07-24T22:02:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:57 smithi060 ceph-mon[33000]: pgmap v888: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-07-24T22:02:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:57 smithi123 ceph-mon[37143]: pgmap v888: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-07-24T22:02:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 -- 172.21.15.60:0/2882412884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 msgr2=0x7f29b81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 --2- 172.21.15.60:0/2882412884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b81039c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009900 tx=0x7f29a802a410 comp rx=0 tx=0).stop 2024-07-24T22:02:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 -- 172.21.15.60:0/2882412884 shutdown_connections 2024-07-24T22:02:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 --2- 172.21.15.60:0/2882412884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 0x7f29b810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 --2- 172.21.15.60:0/2882412884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 0x7f29b810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 --2- 172.21.15.60:0/2882412884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b81039c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 -- 172.21.15.60:0/2882412884 >> 172.21.15.60:0/2882412884 conn(0x7f29b80fd030 msgr2=0x7f29b80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.032+0000 7f29bd534640 1 -- 172.21.15.60:0/2882412884 shutdown_connections 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 -- 172.21.15.60:0/2882412884 wait complete. 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 Processor -- start 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 -- start start 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 0x7f29b8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 0x7f29b819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81126c0 con 0x7f29b81035e0 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8112540 con 0x7f29b8103f00 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.033+0000 7f29bd534640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8112840 con 0x7f29b810ca30 2024-07-24T22:02:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33724/0 (socket says 172.21.15.60:33724) 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 -- 172.21.15.60:0/3489011704 learned_addr learned my addr 172.21.15.60:0/3489011704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 0x7f29b8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 0x7f29b819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 -- 172.21.15.60:0/3489011704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 msgr2=0x7f29b819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 0x7f29b819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 -- 172.21.15.60:0/3489011704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 msgr2=0x7f29b8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 0x7f29b8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 -- 172.21.15.60:0/3489011704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b819eca0 con 0x7f29b81035e0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005940 tx=0x7f29a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b4ff9640 1 -- 172.21.15.60:0/3489011704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a803f030 con 0x7f29b81035e0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b4ff9640 1 -- 172.21.15.60:0/3489011704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a80044c0 con 0x7f29b81035e0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b819ef30 con 0x7f29b81035e0 2024-07-24T22:02:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29b4ff9640 1 -- 172.21.15.60:0/3489011704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8035070 con 0x7f29b81035e0 2024-07-24T22:02:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.034+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a5650 con 0x7f29b81035e0 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29b4ff9640 1 -- 172.21.15.60:0/3489011704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f29a8042030 con 0x7f29b81035e0 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 Processor -- start 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 -- start start 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cbf755f280 con 0x55cbf75f6170 2024-07-24T22:02:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbf754e780 con 0x55cbf75ea1e0 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29c0285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbf75527c0 con 0x55cbf75e98c0 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33738/0 (socket says 172.21.15.60:33738) 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:43148/0 (socket says 172.21.15.60:43148) 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29ae7fc640 1 -- 172.21.15.60:0/1083766807 learned_addr learned my addr 172.21.15.60:0/1083766807 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.037+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75e9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53656/0 (socket says 172.21.15.60:53656) 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 -- 172.21.15.60:0/1083766807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 msgr2=0x55cbf75e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 -- 172.21.15.60:0/1083766807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 msgr2=0x55cbf75f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 -- 172.21.15.60:0/1083766807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf72b2790 con 0x55cbf75f6170 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29b4ff9640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 0x7f29a40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ae7fc640 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009c50 tx=0x7f29a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ec4384d0da35f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 0x7f29a40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29b4ff9640 1 -- 172.21.15.60:0/3489011704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f29a8104350 con 0x7f29b81035e0 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ad7fa640 1 -- 172.21.15.60:0/1083766807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0004220 con 0x55cbf75f6170 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ad7fa640 1 -- 172.21.15.60:0/1083766807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a00043c0 con 0x55cbf75f6170 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.038+0000 7f29ad7fa640 1 -- 172.21.15.60:0/1083766807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0004c90 con 0x55cbf75f6170 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- 172.21.15.60:0/1083766807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 msgr2=0x55cbf75f8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 0x7f29a40b6200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fd0 tx=0x7f29b0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009c50 tx=0x7f29a0011810 comp rx=0 tx=0).stop 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- 172.21.15.60:0/1083766807 shutdown_connections 2024-07-24T22:02:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f8560 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 --2- 172.21.15.60:0/1083766807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- 172.21.15.60:0/1083766807 >> 172.21.15.60:0/1083766807 conn(0x55cbf75e76f0 msgr2=0x55cbf75e8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- 172.21.15.60:0/1083766807 shutdown_connections 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- 172.21.15.60:0/1083766807 wait complete. 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 -- start start 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.039+0000 7f29c0285740 1 Processor -- start 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55cbf7488ab0 con 0x55cbf75f6170 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cbf74d69e0 con 0x55cbf75ea1e0 2024-07-24T22:02:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbf75f8e50 con 0x55cbf75e98c0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 10 client.0 _ll_register_callbacks cb 0x55cbf725ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33742/0 (socket says 172.21.15.60:33742) 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 -- 172.21.15.60:0/3456355134 learned_addr learned my addr 172.21.15.60:0/3456355134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29ae7fc640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 -- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 msgr2=0x55cbf75f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 -- 172.21.15.60:0/3456355134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 msgr2=0x55cbf75f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 -- 172.21.15.60:0/3456355134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf732b620 con 0x55cbf75f6170 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29b77fe640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f29b0003fd0 tx=0x7f29b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471f322385b4619 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f299e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b000f8b0 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f29b000fa50 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.040+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbf74fdce0 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f29b000fc50 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 10 client.18840 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cbf7604590 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbf74fdce0 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f29b0026070 con 0x55cbf75f6170 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 10 client.18840 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 10 client.18840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 20 client.18840 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf7605b60 0x55cbf7607fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 4 client.18840 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf75611d0 con 0x55cbf7605b60 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.041+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf7605b60 0x55cbf7607fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f299e7fc640 10 client.18840 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf7605b60 0x55cbf7607fd0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9169305ddc2a75df server_cookie=bc3e9a7df329f0b4 in_seq=0 out_seq=0 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f299e7fc640 1 -- 172.21.15.60:0/3456355134 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbf75611d0 con 0x55cbf7605b60 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f299e7fc640 10 client.18840 handle_command_reply: tid=0 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 client.18840 shutdown 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 msgr2=0x55cbf75f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f29b0003fd0 tx=0x7f29b0004ab0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 shutdown_connections 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf7605b60 0x55cbf7607fd0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf75f3220 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75ea1e0 0x55cbf75f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f299e7fc640 0 client.18840 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 --2- 172.21.15.60:0/3456355134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75e98c0 0x55cbf75f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 >> 172.21.15.60:0/3456355134 conn(0x55cbf75e76f0 msgr2=0x55cbf75ed220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 shutdown_connections 2024-07-24T22:02:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.042+0000 7f29c0285740 1 -- 172.21.15.60:0/3456355134 wait complete. 2024-07-24T22:02:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.043+0000 7f29c0285740 20 client.18840 trim_cache size 0 max 16384 2024-07-24T22:02:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 Processor -- start 2024-07-24T22:02:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 -- start start 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 0x55cbf760d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b0ad0 con 0x55cbf75ea1e0 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81ae2e0 con 0x55cbf75e98c0 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29c0285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81ac150 con 0x55cbf75f6170 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 0x55cbf760d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 0x55cbf760d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33744/0 (socket says 172.21.15.60:33744) 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53674/0 (socket says 172.21.15.60:53674) 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29ae7fc640 1 -- 172.21.15.60:0/2438073842 learned_addr learned my addr 172.21.15.60:0/2438073842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 -- 172.21.15.60:0/2438073842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 msgr2=0x55cbf75e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 -- 172.21.15.60:0/2438073842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 msgr2=0x55cbf760d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 0x55cbf760d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.058+0000 7f29b77fe640 1 -- 172.21.15.60:0/2438073842 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf75ea5c0 con 0x55cbf75f6170 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29b0028590 tx=0x7f29b0028440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18cb9c8a5bf67a9f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29acff9640 1 -- 172.21.15.60:0/2438073842 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b000fdf0 con 0x55cbf75f6170 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29acff9640 1 -- 172.21.15.60:0/2438073842 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b0028800 con 0x55cbf75f6170 2024-07-24T22:02:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29acff9640 1 -- 172.21.15.60:0/2438073842 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b001d6d0 con 0x55cbf75f6170 2024-07-24T22:02:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.059+0000 7f29c0285740 1 -- 172.21.15.60:0/2438073842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 msgr2=0x55cbf760dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29b0028590 tx=0x7f29b0028440 comp rx=0 tx=0).stop 2024-07-24T22:02:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- 172.21.15.60:0/2438073842 shutdown_connections 2024-07-24T22:02:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75f6170 0x55cbf760dde0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75ea1e0 0x55cbf760d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- 172.21.15.60:0/2438073842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- 172.21.15.60:0/2438073842 >> 172.21.15.60:0/2438073842 conn(0x55cbf75e76f0 msgr2=0x55cbf75e8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- 172.21.15.60:0/2438073842 shutdown_connections 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- 172.21.15.60:0/2438073842 wait complete. 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- start start 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 Processor -- start 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 0x55cbf75fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81ac7a0 con 0x55cbf75f6170 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81adc10 con 0x55cbf75e98c0 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81ace70 con 0x55cbf75ea1e0 2024-07-24T22:02:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29c0285740 10 client.0 _ll_register_callbacks cb 0x7f29b0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 0x55cbf75fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33750/0 (socket says 172.21.15.60:33750) 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:43172/0 (socket says 172.21.15.60:43172) 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29ae7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 0x55cbf75fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53686/0 (socket says 172.21.15.60:53686) 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.060+0000 7f29b77fe640 1 -- 172.21.15.60:0/2928393129 learned_addr learned my addr 172.21.15.60:0/2928393129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 -- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 msgr2=0x55cbf75fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 0x55cbf75fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 -- 172.21.15.60:0/2928393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 msgr2=0x55cbf75fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 -- 172.21.15.60:0/2928393129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf76731c0 con 0x55cbf75f6170 2024-07-24T22:02:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29b77fe640 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fa0 tx=0x7f29b0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671088c040c37976 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f299e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00289b0 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f29b000f8b0 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b0028800 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f29b000fab0 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29c0285740 10 client.18846 fetch_fsmap learned FSMap version 46 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cbf7674320 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.061+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b0028800 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f29b0047070 con 0x55cbf75f6170 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 10 client.18846 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 10 client.18846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 20 client.18846 populate_metadata read hostname 'smithi060' 2024-07-24T22:02:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf76749c0 0x55cbf7676de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:02:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 4 client.18846 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:02:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbf755cc70 con 0x55cbf76749c0 2024-07-24T22:02:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf76749c0 0x55cbf7676de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:02:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f299e7fc640 10 client.18846 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.062+0000 7f29b6ffd640 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf76749c0 0x55cbf7676de0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562da7dfc61b85bf server_cookie=220b63c7ab5a9463 in_seq=0 out_seq=0 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.070+0000 7f299e7fc640 1 -- 172.21.15.60:0/2928393129 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbf755cc70 con 0x55cbf76749c0 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.070+0000 7f299e7fc640 10 client.18846 handle_command_reply: tid=0 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.070+0000 7f29c0285740 1 client.18846 shutdown 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.070+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 msgr2=0x55cbf7672c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.070+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fa0 tx=0x7f29b0009620 comp rx=0 tx=0).stop 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 shutdown_connections 2024-07-24T22:02:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55cbf76749c0 0x55cbf7676de0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cbf75f6170 0x55cbf7672c80 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbf75ea1e0 0x55cbf75fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f299e7fc640 0 client.18846 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 --2- 172.21.15.60:0/2928393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cbf75e98c0 0x55cbf75fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 >> 172.21.15.60:0/2928393129 conn(0x55cbf75e76f0 msgr2=0x55cbf75e5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 shutdown_connections 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 1 -- 172.21.15.60:0/2928393129 wait complete. 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29c0285740 20 client.18846 trim_cache size 0 max 16384 2024-07-24T22:02:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:02:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:02:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:02:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:02:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 msgr2=0x7f29a40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 0x7f29a40b6200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fd0 tx=0x7f29b0009500 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 msgr2=0x7f29b8198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005940 tx=0x7f29a80040c0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 shutdown_connections 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f29a40b3d40 0x7f29a40b6200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b810ca30 0x7f29b819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29b8103f00 0x7f29b8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 --2- 172.21.15.60:0/3489011704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29b81035e0 0x7f29b8198c00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 >> 172.21.15.60:0/3489011704 conn(0x7f29b80fd030 msgr2=0x7f29b80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:02:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 shutdown_connections 2024-07-24T22:02:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:02:58.071+0000 7f29bd534640 1 -- 172.21.15.60:0/3489011704 wait complete. 2024-07-24T22:02:59.606 INFO:tasks.quiescer.fs.[cephfs]:Total duration (54.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-24T22:02:59.607 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.3 2024-07-24T22:02:59.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:02:59.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:02:59 smithi086 ceph-mon[37128]: pgmap v889: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 1.38k op/s 2024-07-24T22:02:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:02:59 smithi060 ceph-mon[33000]: pgmap v889: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 1.38k op/s 2024-07-24T22:02:59.951 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:02:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:02:59 smithi123 ceph-mon[37143]: pgmap v889: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 1.38k op/s 2024-07-24T22:03:01.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:01 smithi086 ceph-mon[37128]: pgmap v890: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-07-24T22:03:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:01 smithi060 ceph-mon[33000]: pgmap v890: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- 172.21.15.60:0/1838814037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 msgr2=0x7fcdd810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 --2- 172.21.15.60:0/1838814037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd810e880 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a480 comp rx=0 tx=0).stop 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- 172.21.15.60:0/1838814037 shutdown_connections 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 --2- 172.21.15.60:0/1838814037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd810edc0 0x7fcdd81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 --2- 172.21.15.60:0/1838814037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd810e880 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 --2- 172.21.15.60:0/1838814037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd8101210 0x7fcdd81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- 172.21.15.60:0/1838814037 >> 172.21.15.60:0/1838814037 conn(0x7fcdd80fd0c0 msgr2=0x7fcdd80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- 172.21.15.60:0/1838814037 shutdown_connections 2024-07-24T22:03:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- 172.21.15.60:0/1838814037 wait complete. 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 Processor -- start 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.860+0000 7fcdde346640 1 -- start start 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd8101210 0x7fcdd819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd810edc0 0x7fcdd81a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8114900 con 0x7fcdd8101b30 2024-07-24T22:03:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8114780 con 0x7fcdd810edc0 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdde346640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8114a80 con 0x7fcdd8101210 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33760/0 (socket says 172.21.15.60:33760) 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 -- 172.21.15.60:0/3473165646 learned_addr learned my addr 172.21.15.60:0/3473165646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcddc8e3640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd810edc0 0x7fcdd81a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd8101210 0x7fcdd819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 -- 172.21.15.60:0/3473165646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd8101210 msgr2=0x7fcdd819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd8101210 0x7fcdd819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 -- 172.21.15.60:0/3473165646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd810edc0 msgr2=0x7fcdd81a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.861+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd810edc0 0x7fcdd81a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdd77fe640 1 -- 172.21.15.60:0/3473165646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd81a0ec0 con 0x7fcdd8101b30 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005940 tx=0x7fcdcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc03f030 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc003cb0 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd81a1150 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd81a7870 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.862+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.863+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc40052c0 con 0x7fcdd8101b30 2024-07-24T22:03:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.864+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fcdcc042030 con 0x7fcdd8101b30 2024-07-24T22:03:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.865+0000 7fcdd57fa640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 0x7fcdc00b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.865+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 0x7fcdc00b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.865+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7fcdcc104770 con 0x7fcdd8101b30 2024-07-24T22:03:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.866+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 0x7fcdc00b6350 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005d00 tx=0x7fcdc800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:01.868+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fcdcc0cb480 con 0x7fcdd8101b30 2024-07-24T22:03:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:01 smithi123 ceph-mon[37143]: pgmap v890: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 1.33k op/s 2024-07-24T22:03:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.102+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdc40055b0 con 0x7fcdd8101b30 2024-07-24T22:03:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.103+0000 7fcdd57fa640 1 -- 172.21.15.60:0/3473165646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7fcdcc0cf9d0 con 0x7fcdd8101b30 2024-07-24T22:03:02.105 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:03:02.105 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:03:02.105 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 msgr2=0x7fcdc00b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 0x7fcdc00b6350 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005d00 tx=0x7fcdc800b410 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 msgr2=0x7fcdd819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005940 tx=0x7fcdcc0042f0 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 shutdown_connections 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcdc00b3e90 0x7fcdc00b6350 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd810edc0 0x7fcdd81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd8101b30 0x7fcdd819b350 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 --2- 172.21.15.60:0/3473165646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdd8101210 0x7fcdd819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 >> 172.21.15.60:0/3473165646 conn(0x7fcdd80fd0c0 msgr2=0x7fcdd80ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 shutdown_connections 2024-07-24T22:03:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:02.105+0000 7fcdde346640 1 -- 172.21.15.60:0/3473165646 wait complete. 2024-07-24T22:03:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:03:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:03:02.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:02 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3473165646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:02.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:03:02.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:03:02.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:02 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3473165646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:03:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:03:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:02 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3473165646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:03.345 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0' 2024-07-24T22:03:03.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0 2024-07-24T22:03:03.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:03 smithi060 ceph-mon[33000]: pgmap v891: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-07-24T22:03:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:03.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:03 smithi086 ceph-mon[37128]: pgmap v891: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-07-24T22:03:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:03 smithi123 ceph-mon[37143]: pgmap v891: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 -- 172.21.15.60:0/1305232539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c4410ed60 msgr2=0x7f2c44111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/1305232539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c4410ed60 0x7f2c44111150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009ca0 tx=0x7f2c3c02a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 -- 172.21.15.60:0/1305232539 shutdown_connections 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/1305232539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c4410ed60 0x7f2c44111150 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/1305232539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c440757f0 0x7f2c4410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/1305232539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 0x7f2c440752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 -- 172.21.15.60:0/1305232539 >> 172.21.15.60:0/1305232539 conn(0x7f2c440fe1e0 msgr2=0x7f2c44100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 -- 172.21.15.60:0/1305232539 shutdown_connections 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.389+0000 7f2c4ac40640 1 -- 172.21.15.60:0/1305232539 wait complete. 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 Processor -- start 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 -- start start 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 0x7f2c4419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 0x7f2c441a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44114980 con 0x7f2c440757f0 2024-07-24T22:03:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44114800 con 0x7f2c4410ed60 2024-07-24T22:03:05.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c4ac40640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44114b00 con 0x7f2c44074ed0 2024-07-24T22:03:05.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33786/0 (socket says 172.21.15.60:33786) 2024-07-24T22:03:05.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c43fff640 1 -- 172.21.15.60:0/2894093027 learned_addr learned my addr 172.21.15.60:0/2894093027 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:05.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c489dc640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 0x7f2c4419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c491dd640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 0x7f2c441a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c43fff640 1 -- 172.21.15.60:0/2894093027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 msgr2=0x7f2c4419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.390+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 0x7f2c4419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c43fff640 1 -- 172.21.15.60:0/2894093027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 msgr2=0x7f2c441a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 0x7f2c441a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c43fff640 1 -- 172.21.15.60:0/2894093027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441a0ee0 con 0x7f2c440757f0 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c491dd640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 0x7f2c441a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b4a0 tx=0x7f2c3800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c41ffb640 1 -- 172.21.15.60:0/2894093027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38015020 con 0x7f2c440757f0 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c41ffb640 1 -- 172.21.15.60:0/2894093027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c38011ce0 con 0x7f2c440757f0 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441a11d0 con 0x7f2c440757f0 2024-07-24T22:03:05.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c41ffb640 1 -- 172.21.15.60:0/2894093027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38010dc0 con 0x7f2c440757f0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.391+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441a7910 con 0x7f2c440757f0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.393+0000 7f2c41ffb640 1 -- 172.21.15.60:0/2894093027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f2c38018020 con 0x7f2c440757f0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 Processor -- start 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 -- start start 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 0x55fccc7a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fccc71f670 con 0x55fccc7a9770 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fccc70eb70 con 0x55fccc7aacd0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fccc712bb0 con 0x55fccc7b55a0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c489dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c489dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33790/0 (socket says 172.21.15.60:33790) 2024-07-24T22:03:05.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c489dc640 1 -- 172.21.15.60:0/4213012168 learned_addr learned my addr 172.21.15.60:0/4213012168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c43fff640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c41ffb640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 0x7f2c2c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c491dd640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 0x55fccc7a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c489dc640 1 -- 172.21.15.60:0/4213012168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 msgr2=0x55fccc7b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.394+0000 7f2c489dc640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c489dc640 1 -- 172.21.15.60:0/4213012168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 msgr2=0x55fccc7a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c489dc640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 0x55fccc7a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c489dc640 1 -- 172.21.15.60:0/4213012168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccc6a6d30 con 0x55fccc7a9770 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c491dd640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 0x55fccc7a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:05.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c41ffb640 1 -- 172.21.15.60:0/2894093027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f2c380e3880 con 0x7f2c440757f0 2024-07-24T22:03:05.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c489dc640 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400acf0 tx=0x7f2c3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8dd48b4ac499ce server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c491dd640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 0x7f2c2c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c2affd640 1 -- 172.21.15.60:0/4213012168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401d070 con 0x55fccc7a9770 2024-07-24T22:03:05.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c2affd640 1 -- 172.21.15.60:0/4213012168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34018af0 con 0x55fccc7a9770 2024-07-24T22:03:05.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.395+0000 7f2c2affd640 1 -- 172.21.15.60:0/4213012168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34010c20 con 0x55fccc7a9770 2024-07-24T22:03:05.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- 172.21.15.60:0/4213012168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 msgr2=0x55fccc7a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c491dd640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 0x7f2c2c0b6200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005e30 tx=0x7f2c3c0095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400acf0 tx=0x7f2c3400f790 comp rx=0 tx=0).stop 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- 172.21.15.60:0/4213012168 shutdown_connections 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7a9770 0x55fccc7a9b90 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/4213012168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7aacd0 0x55fccc7a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- 172.21.15.60:0/4213012168 >> 172.21.15.60:0/4213012168 conn(0x55fccc773720 msgr2=0x55fccc7a7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- 172.21.15.60:0/4213012168 shutdown_connections 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- 172.21.15.60:0/4213012168 wait complete. 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 -- start start 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.396+0000 7f2c4cff1740 1 Processor -- start 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55fccc5727a0 con 0x55fccc7aacd0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fccc6b0ff0 con 0x55fccc7a9770 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fccc7b8460 con 0x55fccc7b55a0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c4cff1740 10 client.0 _ll_register_callbacks cb 0x55fccc41c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33802/0 (socket says 172.21.15.60:33802) 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 -- 172.21.15.60:0/2858111863 learned_addr learned my addr 172.21.15.60:0/2858111863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c489dc640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c491dd640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 -- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 msgr2=0x55fccc7b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 -- 172.21.15.60:0/2858111863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 msgr2=0x55fccc7b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.397+0000 7f2c43fff640 1 -- 172.21.15.60:0/2858111863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccc6b50a0 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c43fff640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2c380aa4a0 tx=0x7f2c380a6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98ad1a1cb699c67 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380e6e60 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f2c380b3330 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34018af0 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c380a9c80 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 10 client.18864 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fccc47eb90 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34018af0 con 0x55fccc7aacd0 2024-07-24T22:03:05.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f2c380a9770 con 0x55fccc7aacd0 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 10 client.18864 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 10 client.18864 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 20 client.18864 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc7c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 4 client.18864 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.398+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fccc7215c0 con 0x55fccc7c5480 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.399+0000 7f2c489dc640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc7c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.399+0000 7f2c489dc640 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc7c78b0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c309f42a78dc159 server_cookie=60bf3009ccda6aba in_seq=0 out_seq=0 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.399+0000 7f2c23fff640 10 client.18864 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c23fff640 1 -- 172.21.15.60:0/2858111863 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fccc7215c0 con 0x55fccc7c5480 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c23fff640 10 client.18864 handle_command_reply: tid=0 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 client.18864 shutdown 2024-07-24T22:03:05.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 msgr2=0x55fccc7b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2c380aa4a0 tx=0x7f2c380a6a20 comp rx=0 tx=0).stop 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 shutdown_connections 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc7c78b0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c23fff640 0 client.18864 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b2580 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/2858111863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 >> 172.21.15.60:0/2858111863 conn(0x55fccc773720 msgr2=0x55fccc7acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 shutdown_connections 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 1 -- 172.21.15.60:0/2858111863 wait complete. 2024-07-24T22:03:05.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.400+0000 7f2c4cff1740 20 client.18864 trim_cache size 0 max 16384 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 Processor -- start 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 -- start start 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7a9770 0x55fccc7b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 0x55fccc7b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441b2d60 con 0x55fccc7aacd0 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441b0570 con 0x55fccc7b55a0 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ae3e0 con 0x55fccc7a9770 2024-07-24T22:03:05.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33812/0 (socket says 172.21.15.60:33812) 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.418+0000 7f2c43fff640 1 -- 172.21.15.60:0/1192945315 learned_addr learned my addr 172.21.15.60:0/1192945315 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c489dc640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7a9770 0x55fccc7b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c491dd640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 0x55fccc7b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 -- 172.21.15.60:0/1192945315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7a9770 msgr2=0x55fccc7b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7a9770 0x55fccc7b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 -- 172.21.15.60:0/1192945315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 msgr2=0x55fccc7b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 0x55fccc7b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 -- 172.21.15.60:0/1192945315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccc7b0a40 con 0x55fccc7aacd0 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c491dd640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 0x55fccc7b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c43fff640 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a9b40 tx=0x7f2c38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b703bbe8503f13a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c2a7fc640 1 -- 172.21.15.60:0/1192945315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380e13d0 con 0x55fccc7aacd0 2024-07-24T22:03:05.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c2a7fc640 1 -- 172.21.15.60:0/1192945315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c380aac10 con 0x55fccc7aacd0 2024-07-24T22:03:05.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.419+0000 7f2c2a7fc640 1 -- 172.21.15.60:0/1192945315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380092b0 con 0x55fccc7aacd0 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- 172.21.15.60:0/1192945315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 msgr2=0x55fccc7a8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a9b40 tx=0x7f2c38000bc0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- 172.21.15.60:0/1192945315 shutdown_connections 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7b55a0 0x55fccc7b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7a8d50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/1192945315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7a9770 0x55fccc7b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- 172.21.15.60:0/1192945315 >> 172.21.15.60:0/1192945315 conn(0x55fccc773720 msgr2=0x55fccc7b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- 172.21.15.60:0/1192945315 shutdown_connections 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- 172.21.15.60:0/1192945315 wait complete. 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 -- start start 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.420+0000 7f2c4cff1740 1 Processor -- start 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441aea30 con 0x55fccc7aacd0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441afea0 con 0x55fccc7a9770 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441af100 con 0x55fccc7b55a0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c4cff1740 10 client.0 _ll_register_callbacks cb 0x7f2c3800b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33828/0 (socket says 172.21.15.60:33828) 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 -- 172.21.15.60:0/3872718367 learned_addr learned my addr 172.21.15.60:0/3872718367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c491dd640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c489dc640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 -- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 msgr2=0x55fccc7b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 -- 172.21.15.60:0/3872718367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 msgr2=0x55fccc7bdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7bdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 -- 172.21.15.60:0/3872718367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccc7b9380 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c43fff640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a6590 tx=0x7f2c380a6c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168fd3dc9efd6c08 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.421+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380e13d0 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7f2c380bfbb0 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c380aac10 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c380aa510 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 10 client.18870 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fccc7b9380 con 0x55fccc7aacd0 2024-07-24T22:03:05.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c380aac10 con 0x55fccc7aacd0 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f2c380cb020 con 0x55fccc7aacd0 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 10 client.18870 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 10 client.18870 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 20 client.18870 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc833490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 4 client.18870 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}] 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}) -- 0x55fccc71d060 con 0x55fccc7c5480 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.422+0000 7f2c489dc640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc833490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.423+0000 7f2c489dc640 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc833490 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34009f58d9b9535e server_cookie=9806022c84dbcdd8 in_seq=0 out_seq=0 2024-07-24T22:03:05.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:05.423+0000 7f2c23fff640 10 client.18870 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:05.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:05 smithi060 ceph-mon[33000]: pgmap v892: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 660 op/s 2024-07-24T22:03:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:05 smithi123 ceph-mon[37143]: pgmap v892: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 660 op/s 2024-07-24T22:03:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:05 smithi086 ceph-mon[37128]: pgmap v892: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 660 op/s 2024-07-24T22:03:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:07 smithi060 ceph-mon[33000]: pgmap v893: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-07-24T22:03:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:07 smithi123 ceph-mon[37143]: pgmap v893: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-07-24T22:03:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:07 smithi086 ceph-mon[37128]: pgmap v893: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-07-24T22:03:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:09 smithi060 ceph-mon[33000]: pgmap v894: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2024-07-24T22:03:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:09 smithi123 ceph-mon[37143]: pgmap v894: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2024-07-24T22:03:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:09 smithi086 ceph-mon[37128]: pgmap v894: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2024-07-24T22:03:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 -- 172.21.15.60:0/465231606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 msgr2=0x7fe3c010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 --2- 172.21.15.60:0/465231606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c010eeb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009ca0 tx=0x7fe3b802a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 -- 172.21.15.60:0/465231606 shutdown_connections 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 --2- 172.21.15.60:0/465231606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c010eeb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 --2- 172.21.15.60:0/465231606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 0x7fe3c010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 --2- 172.21.15.60:0/465231606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 -- 172.21.15.60:0/465231606 >> 172.21.15.60:0/465231606 conn(0x7fe3c00fd030 msgr2=0x7fe3c00ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 -- 172.21.15.60:0/465231606 shutdown_connections 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.172+0000 7fe3c6552640 1 -- 172.21.15.60:0/465231606 wait complete. 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 Processor -- start 2024-07-24T22:03:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 -- start start 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 0x7fe3c01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01126b0 con 0x7fe3c010ca30 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0112530 con 0x7fe3c0103f00 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c6552640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0112830 con 0x7fe3c01035e0 2024-07-24T22:03:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 0x7fe3c01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44806/0 (socket says 172.21.15.60:44806) 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39092/0 (socket says 172.21.15.60:39092) 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.173+0000 7fe3bffff640 1 -- 172.21.15.60:0/2176068422 learned_addr learned my addr 172.21.15.60:0/2176068422 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 -- 172.21.15.60:0/2176068422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 msgr2=0x7fe3c01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 0x7fe3c01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 -- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 msgr2=0x7fe3c019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 -- 172.21.15.60:0/2176068422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c019ec50 con 0x7fe3c01035e0 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bffff640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e9f0 tx=0x7fe3b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bd7fa640 1 -- 172.21.15.60:0/2176068422 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000cd80 con 0x7fe3c01035e0 2024-07-24T22:03:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bd7fa640 1 -- 172.21.15.60:0/2176068422 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b00054d0 con 0x7fe3c01035e0 2024-07-24T22:03:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3bd7fa640 1 -- 172.21.15.60:0/2176068422 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0010690 con 0x7fe3c01035e0 2024-07-24T22:03:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c019ef40 con 0x7fe3c01035e0 2024-07-24T22:03:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.174+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01a5600 con 0x7fe3c01035e0 2024-07-24T22:03:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3bd7fa640 1 -- 172.21.15.60:0/2176068422 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe3b0004e90 con 0x7fe3c01035e0 2024-07-24T22:03:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 Processor -- start 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 -- start start 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cc0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e707c23280 con 0x55e707cae1e0 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e707c12780 con 0x55e707cad8c0 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c9383740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e707c167c0 con 0x55e707cbe580 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52850/0 (socket says 172.21.15.60:52850) 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 -- 172.21.15.60:0/2017645185 learned_addr learned my addr 172.21.15.60:0/2017645185 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cc0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 -- 172.21.15.60:0/2017645185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 msgr2=0x55e707cc0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cc0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 -- 172.21.15.60:0/2017645185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 msgr2=0x55e707cbe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cbe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.177+0000 7fe3c4aef640 1 -- 172.21.15.60:0/2017645185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e707976790 con 0x55e707cad8c0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3bd7fa640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 0x7fe3a80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8033040 tx=0x7fe3b802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5e80a8af2c357 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3bffff640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 0x7fe3a80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3a67fc640 1 -- 172.21.15.60:0/2017645185 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8031450 con 0x55e707cad8c0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3bd7fa640 1 -- 172.21.15.60:0/2176068422 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7fe3b00e4410 con 0x7fe3c01035e0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3a67fc640 1 -- 172.21.15.60:0/2017645185 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b8036040 con 0x55e707cad8c0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.178+0000 7fe3a67fc640 1 -- 172.21.15.60:0/2017645185 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80043a0 con 0x55e707cad8c0 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- 172.21.15.60:0/2017645185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 msgr2=0x55e707cadca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8033040 tx=0x7fe3b802f310 comp rx=0 tx=0).stop 2024-07-24T22:03:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3bffff640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 0x7fe3a80b6200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00e1be0 tx=0x7fe3b00e7f50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- 172.21.15.60:0/2017645185 shutdown_connections 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cc0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cbe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2017645185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cadca0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- 172.21.15.60:0/2017645185 >> 172.21.15.60:0/2017645185 conn(0x55e707cab6f0 msgr2=0x55e707cac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- 172.21.15.60:0/2017645185 shutdown_connections 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- 172.21.15.60:0/2017645185 wait complete. 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 -- start start 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.179+0000 7fe3c9383740 1 Processor -- start 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707cbb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e707b4cab0 con 0x55e707cbe580 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e707b9a9e0 con 0x55e707cad8c0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e707cc1080 con 0x55e707cae1e0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 10 client.0 _ll_register_callbacks cb 0x55e707922a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44830/0 (socket says 172.21.15.60:44830) 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/2630411094 learned_addr learned my addr 172.21.15.60:0/2630411094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707cbb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/2630411094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 msgr2=0x55e707cbdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cbdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/2630411094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 msgr2=0x55e707cbb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707cbb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/2630411094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7079ef620 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4012770 tx=0x7fe3b4012c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e006535e015a7d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe39f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4004050 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e707bc1ce0 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.180+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7fe3b40041f0 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe3b40043f0 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 10 client.28126 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e707ccc890 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e707bc1ce0 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b4023070 con 0x55e707cae1e0 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 10 client.28126 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 10 client.28126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 20 client.28126 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707ccddc0 0x55e707cd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 4 client.28126 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e707c251d0 con 0x55e707ccddc0 2024-07-24T22:03:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.181+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707ccddc0 0x55e707cd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe39f7fe640 10 client.28126 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707ccddc0 0x55e707cd01e0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dfc8ebee84a275 server_cookie=476d26f2807ca976 in_seq=0 out_seq=0 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe39f7fe640 1 -- 172.21.15.60:0/2630411094 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e707c251d0 con 0x55e707ccddc0 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe39f7fe640 10 client.28126 handle_command_reply: tid=0 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 client.28126 shutdown 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 msgr2=0x55e707cbb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4012770 tx=0x7fe3b4012c30 comp rx=0 tx=0).stop 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 shutdown_connections 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707ccddc0 0x55e707cd01e0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.182+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707cbb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cae1e0 0x55e707cbb1d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 1 --2- 172.21.15.60:0/2630411094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe39f7fe640 0 client.28126 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 >> 172.21.15.60:0/2630411094 conn(0x55e707cab6f0 msgr2=0x55e707cb1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 shutdown_connections 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 1 -- 172.21.15.60:0/2630411094 wait complete. 2024-07-24T22:03:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.183+0000 7fe3c9383740 20 client.28126 trim_cache size 0 max 16384 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 Processor -- start 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 -- start start 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cd6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01ae830 con 0x55e707cae1e0 2024-07-24T22:03:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01ac040 con 0x55e707cad8c0 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c9383740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01a9eb0 con 0x55e707cbe580 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.198+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39120/0 (socket says 172.21.15.60:39120) 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52856/0 (socket says 172.21.15.60:52856) 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/1928488809 learned_addr learned my addr 172.21.15.60:0/1928488809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cd6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/1928488809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 msgr2=0x55e707cd6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cd6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/1928488809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 msgr2=0x55e707cca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/1928488809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e707cbd620 con 0x55e707cae1e0 2024-07-24T22:03:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003cc0 tx=0x7fe3b4009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2b6ceea74e36d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3a5ffb640 1 -- 172.21.15.60:0/1928488809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4029070 con 0x55e707cae1e0 2024-07-24T22:03:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3a5ffb640 1 -- 172.21.15.60:0/1928488809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b40035f0 con 0x55e707cae1e0 2024-07-24T22:03:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.199+0000 7fe3a5ffb640 1 -- 172.21.15.60:0/1928488809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400f790 con 0x55e707cae1e0 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- 172.21.15.60:0/1928488809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 msgr2=0x55e707cd5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003cc0 tx=0x7fe3b4009530 comp rx=0 tx=0).stop 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- 172.21.15.60:0/1928488809 shutdown_connections 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cbe580 0x55e707cd6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cae1e0 0x55e707cd5b00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 --2- 172.21.15.60:0/1928488809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cad8c0 0x55e707cca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- 172.21.15.60:0/1928488809 >> 172.21.15.60:0/1928488809 conn(0x55e707cab6f0 msgr2=0x55e707cbcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- 172.21.15.60:0/1928488809 shutdown_connections 2024-07-24T22:03:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- 172.21.15.60:0/1928488809 wait complete. 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 -- start start 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.200+0000 7fe3c9383740 1 Processor -- start 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cad8c0 0x55e707cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 0x55e707cc40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01aa500 con 0x55e707cbe580 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01ab970 con 0x55e707cae1e0 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01aabd0 con 0x55e707cad8c0 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c9383740 10 client.0 _ll_register_callbacks cb 0x7fe3b4009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 0x55e707cc40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c4aef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39136/0 (socket says 172.21.15.60:39136) 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c4aef640 1 -- 172.21.15.60:0/682701533 learned_addr learned my addr 172.21.15.60:0/682701533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cad8c0 0x55e707cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cad8c0 msgr2=0x55e707cc3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cad8c0 0x55e707cc3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/682701533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 msgr2=0x55e707d30aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 -- 172.21.15.60:0/682701533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e707d311e0 con 0x55e707cae1e0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe3bf7fe640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 0x55e707cc40e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003d60 tx=0x7fe3b4012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=facc88ee293573d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.201+0000 7fe39f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4032020 con 0x55e707cae1e0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8565+0+0 (secure 0 0 0) 0x7fe3b4004100 con 0x55e707cae1e0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b40035f0 con 0x55e707cae1e0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe3b4029040 con 0x55e707cae1e0 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 10 client.28241 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e707d38650 con 0x55e707cae1e0 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b40035f0 con 0x55e707cae1e0 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b4023030 con 0x55e707cae1e0 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 10 client.28241 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 10 client.28241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 20 client.28241 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707d38d90 0x55e707d3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 4 client.28241 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e707c20c70 con 0x55e707d38d90 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.202+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707d38d90 0x55e707d3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.203+0000 7fe39f7fe640 10 client.28241 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.203+0000 7fe3c4aef640 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707d38d90 0x55e707d3b1b0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa25416e3f4c2ae server_cookie=a07226f4caf111f1 in_seq=0 out_seq=0 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.349+0000 7f2c23fff640 1 -- 172.21.15.60:0/3872718367 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fccc71d060 con 0x55fccc7c5480 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.349+0000 7f2c23fff640 10 client.18870 handle_command_reply: tid=0 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.349+0000 7f2c4cff1740 1 client.18870 shutdown 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 msgr2=0x55fccc7b86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a6590 tx=0x7f2c380a6c90 comp rx=0 tx=0).stop 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 shutdown_connections 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55fccc7c5480 0x55fccc833490 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fccc7b55a0 0x55fccc7b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fccc7aacd0 0x55fccc7b86b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 --2- 172.21.15.60:0/3872718367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fccc7a9770 0x55fccc7bdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 >> 172.21.15.60:0/3872718367 conn(0x55fccc773720 msgr2=0x55fccc7acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c23fff640 0 client.18870 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 shutdown_connections 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 1 -- 172.21.15.60:0/3872718367 wait complete. 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.350+0000 7f2c4cff1740 20 client.18870 trim_cache size 0 max 16384 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 46, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 50, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "1bb2ce4a": { 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 50, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:03:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.3, 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:11.355 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 msgr2=0x7f2c2c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 0x7f2c2c0b6200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005e30 tx=0x7f2c3c0095c0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 msgr2=0x7f2c4419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b4a0 tx=0x7f2c3800b8a0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 shutdown_connections 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f2c2c0b3d40 0x7f2c2c0b6200 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c4410ed60 0x7f2c441a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c440757f0 0x7f2c4419b370 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 --2- 172.21.15.60:0/2894093027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c44074ed0 0x7f2c4419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 >> 172.21.15.60:0/2894093027 conn(0x7f2c440fe1e0 msgr2=0x7f2c44110ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 shutdown_connections 2024-07-24T22:03:11.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:11.351+0000 7f2c4ac40640 1 -- 172.21.15.60:0/2894093027 wait complete. 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe39f7fe640 1 -- 172.21.15.60:0/682701533 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e707c20c70 con 0x55e707d38d90 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe39f7fe640 10 client.28241 handle_command_reply: tid=0 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 client.28241 shutdown 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 msgr2=0x55e707cc40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 0x55e707cc40e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003d60 tx=0x7fe3b4012450 comp rx=0 tx=0).stop 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 shutdown_connections 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e707d38d90 0x55e707d3b1b0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e707cbe580 0x55e707d30aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e707cae1e0 0x55e707cc40e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 --2- 172.21.15.60:0/682701533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e707cad8c0 0x55e707cc3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe39f7fe640 0 client.28241 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 >> 172.21.15.60:0/682701533 conn(0x55e707cab6f0 msgr2=0x55e707cb0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 shutdown_connections 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 1 -- 172.21.15.60:0/682701533 wait complete. 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.361+0000 7fe3c9383740 20 client.28241 trim_cache size 0 max 16384 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:03:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:03:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:03:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 msgr2=0x7fe3a80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 0x7fe3a80b6200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00e1be0 tx=0x7fe3b00e7f50 comp rx=0 tx=0).stop 2024-07-24T22:03:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 msgr2=0x7fe3c0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e9f0 tx=0x7fe3b000edf0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 shutdown_connections 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe3a80b3d40 0x7fe3a80b6200 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c010ca30 0x7fe3c019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0103f00 0x7fe3c01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 --2- 172.21.15.60:0/2176068422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c01035e0 0x7fe3c0198bb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 >> 172.21.15.60:0/2176068422 conn(0x7fe3c00fd030 msgr2=0x7fe3c00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 shutdown_connections 2024-07-24T22:03:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:11.362+0000 7fe3c6552640 1 -- 172.21.15.60:0/2176068422 wait complete. 2024-07-24T22:03:11.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:11 smithi060 ceph-mon[33000]: pgmap v895: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 759 KiB/s wr, 124 op/s 2024-07-24T22:03:11.847 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:11 smithi086 ceph-mon[37128]: pgmap v895: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 759 KiB/s wr, 124 op/s 2024-07-24T22:03:11.967 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1bb2ce4a', quiesce took 12.4 seconds. Will release after: 14.9 2024-07-24T22:03:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:11 smithi123 ceph-mon[37143]: pgmap v895: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 759 KiB/s wr, 124 op/s 2024-07-24T22:03:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: pgmap v896: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 761 KiB/s wr, 115 op/s 2024-07-24T22:03:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: osdmap e111: 12 total, 12 up, 12 in 2024-07-24T22:03:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:03:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:13 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:03:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: pgmap v896: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 761 KiB/s wr, 115 op/s 2024-07-24T22:03:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: osdmap e111: 12 total, 12 up, 12 in 2024-07-24T22:03:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:03:13.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:13 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:03:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: pgmap v896: 97 pgs: 97 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 761 KiB/s wr, 115 op/s 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: osdmap e111: 12 total, 12 up, 12 in 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-24T22:03:14.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:03:14.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:03:14.047 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:13 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:03:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:14 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T22:03:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:14 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:14 smithi123 ceph-mon[37143]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T22:03:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:14 smithi123 ceph-mon[37143]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:14 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T22:03:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:14 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:14 smithi086 ceph-mon[37128]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T22:03:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:14 smithi086 ceph-mon[37128]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:15.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:14 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-07-24T22:03:15.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:14 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:15.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:14 smithi060 ceph-mon[33000]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-07-24T22:03:15.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:14 smithi060 ceph-mon[33000]: Unable to set osd_memory_target on smithi086 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-07-24T22:03:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:15 smithi123 ceph-mon[37143]: pgmap v898: 97 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-07-24T22:03:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:15 smithi086 ceph-mon[37128]: pgmap v898: 97 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-07-24T22:03:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:15 smithi060 ceph-mon[33000]: pgmap v898: 97 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-07-24T22:03:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:17 smithi123 ceph-mon[37143]: pgmap v899: 97 pgs: 31 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-07-24T22:03:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:17 smithi086 ceph-mon[37128]: pgmap v899: 97 pgs: 31 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-07-24T22:03:18.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:17 smithi060 ceph-mon[33000]: pgmap v899: 97 pgs: 31 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 48 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 32 op/s 2024-07-24T22:03:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:19 smithi123 ceph-mon[37143]: pgmap v900: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-24T22:03:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:19 smithi086 ceph-mon[37128]: pgmap v900: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-24T22:03:20.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:19 smithi060 ceph-mon[33000]: pgmap v900: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-24T22:03:21.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:03:21.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:21 smithi060 ceph-mon[33000]: pgmap v901: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-24T22:03:21.972 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:21 smithi123 ceph-mon[37143]: pgmap v901: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-24T22:03:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:21 smithi086 ceph-mon[37128]: pgmap v901: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-24T22:03:23.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:23 smithi060 ceph-mon[33000]: pgmap v902: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-24T22:03:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:23 smithi123 ceph-mon[37143]: pgmap v902: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-24T22:03:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:23 smithi086 ceph-mon[37128]: pgmap v902: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-24T22:03:24.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 -- 172.21.15.60:0/2990494630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec107870 msgr2=0x7f9dec10e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 --2- 172.21.15.60:0/2990494630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec107870 0x7f9dec10e070 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009ca0 tx=0x7f9de402a8f0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 -- 172.21.15.60:0/2990494630 shutdown_connections 2024-07-24T22:03:24.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 --2- 172.21.15.60:0/2990494630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec107870 0x7f9dec10e070 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 --2- 172.21.15.60:0/2990494630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec106ef0 0x7f9dec107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 --2- 172.21.15.60:0/2990494630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 0x7f9dec1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.466+0000 7f9df243e640 1 -- 172.21.15.60:0/2990494630 >> 172.21.15.60:0/2990494630 conn(0x7f9dec0fcf90 msgr2=0x7f9dec0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.472+0000 7f9df243e640 1 -- 172.21.15.60:0/2990494630 shutdown_connections 2024-07-24T22:03:24.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.479+0000 7f9df243e640 1 -- 172.21.15.60:0/2990494630 wait complete. 2024-07-24T22:03:24.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.480+0000 7f9df243e640 1 Processor -- start 2024-07-24T22:03:24.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.480+0000 7f9df243e640 1 -- start start 2024-07-24T22:03:24.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 0x7f9dec196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 0x7f9dec196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec110430 con 0x7f9dec106ef0 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1102b0 con 0x7f9dec107870 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df243e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1105b0 con 0x7f9dec1014e0 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.481+0000 7f9df09db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.482+0000 7f9df09db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41888/0 (socket says 172.21.15.60:41888) 2024-07-24T22:03:24.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.482+0000 7f9df09db640 1 -- 172.21.15.60:0/1815036366 learned_addr learned my addr 172.21.15.60:0/1815036366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.484+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 0x7f9dec196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9debfff640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 0x7f9dec196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9df09db640 1 -- 172.21.15.60:0/1815036366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 msgr2=0x7f9dec196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9df09db640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 0x7f9dec196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9df09db640 1 -- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 msgr2=0x7f9dec196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9df09db640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 0x7f9dec196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.485+0000 7f9df09db640 1 -- 172.21.15.60:0/1815036366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec19c8f0 con 0x7f9dec107870 2024-07-24T22:03:24.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.486+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 0x7f9dec196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:24.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.487+0000 7f9df09db640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9de402aee0 tx=0x7f9de4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.488+0000 7f9de97fa640 1 -- 172.21.15.60:0/1815036366 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4003c70 con 0x7f9dec107870 2024-07-24T22:03:24.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.488+0000 7f9de97fa640 1 -- 172.21.15.60:0/1815036366 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de402d070 con 0x7f9dec107870 2024-07-24T22:03:24.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.488+0000 7f9de97fa640 1 -- 172.21.15.60:0/1815036366 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de402fb40 con 0x7f9dec107870 2024-07-24T22:03:24.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.489+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec19cb80 con 0x7f9dec107870 2024-07-24T22:03:24.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.489+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec1a32a0 con 0x7f9dec107870 2024-07-24T22:03:24.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.496+0000 7f9df47ef740 1 Processor -- start 2024-07-24T22:03:24.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.496+0000 7f9de97fa640 1 -- 172.21.15.60:0/1815036366 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9de402fce0 con 0x7f9dec107870 2024-07-24T22:03:24.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.498+0000 7f9de97fa640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 0x7f9dd40b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.498+0000 7f9de97fa640 1 -- 172.21.15.60:0/1815036366 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f9de41052a0 con 0x7f9dec107870 2024-07-24T22:03:24.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.498+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 0x7f9dd40b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.499+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 0x7f9dd40b8660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9dec105a40 tx=0x7f9de000a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 -- start start 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f0db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555613e78960 con 0x555613f01800 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555613e67e60 con 0x555613f0e0b0 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9df47ef740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555613e6bea0 con 0x555613f02120 2024-07-24T22:03:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.503+0000 7f9debfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.504+0000 7f9debfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50358/0 (socket says 172.21.15.60:50358) 2024-07-24T22:03:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.504+0000 7f9debfff640 1 -- 172.21.15.60:0/1766139898 learned_addr learned my addr 172.21.15.60:0/1766139898 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.506+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f0db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.506+0000 7f9df09db640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9debfff640 1 -- 172.21.15.60:0/1766139898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 msgr2=0x555613f0db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9debfff640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f0db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9debfff640 1 -- 172.21.15.60:0/1766139898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 msgr2=0x555613f104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9debfff640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9debfff640 1 -- 172.21.15.60:0/1766139898 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555613df12d0 con 0x555613f01800 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.508+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f0db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.510+0000 7f9debfff640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00a8d0 tx=0x7f9ddc00ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aca6c8454338a2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.510+0000 7f9df09db640 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.510+0000 7f9dd27fc640 1 -- 172.21.15.60:0/1766139898 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc015070 con 0x555613f01800 2024-07-24T22:03:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.511+0000 7f9dd27fc640 1 -- 172.21.15.60:0/1766139898 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc0113f0 con 0x555613f01800 2024-07-24T22:03:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.512+0000 7f9df47ef740 1 -- 172.21.15.60:0/1766139898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 msgr2=0x555613f01be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.512+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00a8d0 tx=0x7f9ddc00ad90 comp rx=0 tx=0).stop 2024-07-24T22:03:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.513+0000 7f9df47ef740 1 -- 172.21.15.60:0/1766139898 shutdown_connections 2024-07-24T22:03:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.513+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.514+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f0db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.514+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1766139898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f01be0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.514+0000 7f9df47ef740 1 -- 172.21.15.60:0/1766139898 >> 172.21.15.60:0/1766139898 conn(0x555613eff630 msgr2=0x555613f007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.514+0000 7f9df47ef740 1 -- 172.21.15.60:0/1766139898 shutdown_connections 2024-07-24T22:03:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.518+0000 7f9df47ef740 1 -- 172.21.15.60:0/1766139898 wait complete. 2024-07-24T22:03:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f42dfe86640 1 -- 172.21.15.60:0/2705147601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 msgr2=0x7f42d8110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2705147601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d8110ba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f42d401a910 tx=0x7f42d403b470 comp rx=0 tx=0).stop 2024-07-24T22:03:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.518+0000 7f9df47ef740 1 -- start start 2024-07-24T22:03:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 Processor -- start 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f07c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x555613ccb700 con 0x555613f01800 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555613e0a5a0 con 0x555613f0e0b0 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555613f10c70 con 0x555613f02120 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.519+0000 7f9df47ef740 10 client.0 _ll_register_callbacks cb 0x555613e2a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.520+0000 7f9debfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.520+0000 7f9debfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50378/0 (socket says 172.21.15.60:50378) 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.520+0000 7f9debfff640 1 -- 172.21.15.60:0/2905952023 learned_addr learned my addr 172.21.15.60:0/2905952023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.520+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.520+0000 7f9df09db640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f07c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f42dfe86640 1 -- 172.21.15.60:0/2705147601 shutdown_connections 2024-07-24T22:03:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2705147601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42d811ba90 0x7f42d811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2705147601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 0x7f42d810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2705147601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d8110ba0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f42dfe86640 1 -- 172.21.15.60:0/2705147601 >> 172.21.15.60:0/2705147601 conn(0x7f42d806d760 msgr2=0x7f42d806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f9debfff640 1 -- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 msgr2=0x555613f076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f9debfff640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f9debfff640 1 -- 172.21.15.60:0/2905952023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 msgr2=0x555613f07c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f9debfff640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f07c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.521+0000 7f9debfff640 1 -- 172.21.15.60:0/2905952023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555613dfd630 con 0x555613f01800 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.523+0000 7f9debfff640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006a90 tx=0x7f9ddc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5a0699c16935ed server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.523+0000 7f9dcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.523+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc027060 con 0x555613f01800 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.523+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f9ddc004240 con 0x555613f01800 2024-07-24T22:03:24.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.523+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ddc0113f0 con 0x555613f01800 2024-07-24T22:03:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.526+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9ddc015070 con 0x555613f01800 2024-07-24T22:03:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.527+0000 7f42dfe86640 1 -- 172.21.15.60:0/2705147601 shutdown_connections 2024-07-24T22:03:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.527+0000 7f9df47ef740 10 client.18903 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.527+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555613bc8650 con 0x555613f01800 2024-07-24T22:03:24.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.527+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc0113f0 con 0x555613f01800 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.527+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9ddc021030 con 0x555613f01800 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 10 client.18903 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 20 client.18903 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613f37690 0x555613f39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 4 client.18903 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:24.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555613e7a8b0 con 0x555613f37690 2024-07-24T22:03:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f42dfe86640 1 -- 172.21.15.60:0/2705147601 wait complete. 2024-07-24T22:03:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.528+0000 7f42dfe86640 1 Processor -- start 2024-07-24T22:03:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 -- start start 2024-07-24T22:03:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 0x7f42d811b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42d811ba90 0x7f42d8117440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8113b60 con 0x7f42d8109b40 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81139e0 con 0x7f42d8109f20 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dfe86640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8113ce0 con 0x7f42d811ba90 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42ddc22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42ddc22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50390/0 (socket says 172.21.15.60:50390) 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42ddc22640 1 -- 172.21.15.60:0/2373171844 learned_addr learned my addr 172.21.15.60:0/2373171844 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.529+0000 7f42dd421640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 0x7f42d811b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 -- 172.21.15.60:0/2373171844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42d811ba90 msgr2=0x7f42d8117440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42d811ba90 0x7f42d8117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 -- 172.21.15.60:0/2373171844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 msgr2=0x7f42d811b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 0x7f42d811b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 -- 172.21.15.60:0/2373171844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d8117d00 con 0x7f42d8109b40 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ddc22640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f42d401aa30 tx=0x7f42d4004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ceffd640 1 -- 172.21.15.60:0/2373171844 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d404f070 con 0x7f42d8109b40 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ceffd640 1 -- 172.21.15.60:0/2373171844 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d4040bd0 con 0x7f42d8109b40 2024-07-24T22:03:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42ceffd640 1 -- 172.21.15.60:0/2373171844 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4046030 con 0x7f42d8109b40 2024-07-24T22:03:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d8117f90 con 0x7f42d8109b40 2024-07-24T22:03:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.530+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81abcb0 con 0x7f42d8109b40 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.534+0000 7f42e2bd8740 1 Processor -- start 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.535+0000 7f42e2bd8740 1 -- start start 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.535+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.535+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06311e0 0x559de063cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41946/0 (socket says 172.21.15.60:41946) 2024-07-24T22:03:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.532+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613f37690 0x555613f39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 -- 172.21.15.60:0/363421311 learned_addr learned my addr 172.21.15.60:0/363421311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de063f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559de05a6280 con 0x559de06311e0 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559de0595780 con 0x559de06308c0 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42dd421640 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de063f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559de05997c0 con 0x559de063d170 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 -- 172.21.15.60:0/363421311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 msgr2=0x559de063f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de063f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 -- 172.21.15.60:0/363421311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06311e0 msgr2=0x559de063cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.536+0000 7f42de423640 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06311e0 0x559de063cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.538+0000 7f42de423640 1 -- 172.21.15.60:0/363421311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de02f9790 con 0x559de06308c0 2024-07-24T22:03:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.538+0000 7f42de423640 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009d20 tx=0x7f42d000b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548c981d627c6e5a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.540+0000 7f42ceffd640 1 -- 172.21.15.60:0/2373171844 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f42d4052070 con 0x7f42d8109b40 2024-07-24T22:03:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.540+0000 7f42c3fff640 1 -- 172.21.15.60:0/363421311 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0018070 con 0x559de06308c0 2024-07-24T22:03:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.540+0000 7f42c3fff640 1 -- 172.21.15.60:0/363421311 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d00138c0 con 0x559de06308c0 2024-07-24T22:03:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.540+0000 7f42c3fff640 1 -- 172.21.15.60:0/363421311 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0012890 con 0x559de06308c0 2024-07-24T22:03:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42ceffd640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 0x7f42c40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42ceffd640 1 -- 172.21.15.60:0/2373171844 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f42d4117620 con 0x7f42d8109b40 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 msgr2=0x559de0630ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009d20 tx=0x7f42d000b820 comp rx=0 tx=0).stop 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 shutdown_connections 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.541+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de063f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42ddc22640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 0x7f42c40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06311e0 0x559de063cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/363421311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06308c0 0x559de0630ca0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 >> 172.21.15.60:0/363421311 conn(0x559de062e6f0 msgr2=0x559de062f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613f37690 0x555613f39af0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2562967a41e080 server_cookie=a9bec230ae85aed3 in_seq=0 out_seq=0 2024-07-24T22:03:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f9dcb7fe640 10 client.18903 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 shutdown_connections 2024-07-24T22:03:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42e2bd8740 1 -- 172.21.15.60:0/363421311 wait complete. 2024-07-24T22:03:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.542+0000 7f42ddc22640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 0x7f42c40b62b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x559de0633a40 tx=0x7f42d403b470 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.543+0000 7f42e2bd8740 1 -- start start 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.543+0000 7f42e2bd8740 1 Processor -- start 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de063a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42de423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42de423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48150/0 (socket says 172.21.15.60:48150) 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42dd421640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.544+0000 7f42dd421640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41960/0 (socket says 172.21.15.60:41960) 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42de423640 1 -- 172.21.15.60:0/3224358909 learned_addr learned my addr 172.21.15.60:0/3224358909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559de04cfab0 con 0x559de063d170 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559de051d9e0 con 0x559de06311e0 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559de063fe50 con 0x559de06308c0 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42e2bd8740 10 client.0 _ll_register_callbacks cb 0x559de02a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.545+0000 7f42dd421640 1 -- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 msgr2=0x559de063c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.546+0000 7f42dd421640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.546+0000 7f42dd421640 1 -- 172.21.15.60:0/3224358909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 msgr2=0x559de063a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.546+0000 7f42dd421640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de063a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.546+0000 7f42dd421640 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de0372620 con 0x559de06311e0 2024-07-24T22:03:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.546+0000 7f42de423640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f42dd421640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f42c80058c0 tx=0x7f42c8011be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b629708bdcb5c79 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f42c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c800fd30 con 0x559de06311e0 2024-07-24T22:03:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/2905952023 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555613e7a8b0 con 0x555613f37690 2024-07-24T22:03:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f9dcb7fe640 10 client.18903 handle_command_reply: tid=0 2024-07-24T22:03:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.547+0000 7f9df47ef740 1 client.18903 shutdown 2024-07-24T22:03:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559de0544ce0 con 0x559de06311e0 2024-07-24T22:03:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f42c8005020 con 0x559de06311e0 2024-07-24T22:03:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f42c8005220 con 0x559de06311e0 2024-07-24T22:03:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f42e2bd8740 10 client.28259 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559de064b700 con 0x559de06311e0 2024-07-24T22:03:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 msgr2=0x555613f071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.548+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006a90 tx=0x7f9ddc00cb70 comp rx=0 tx=0).stop 2024-07-24T22:03:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559de0544ce0 con 0x559de06311e0 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 shutdown_connections 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613f37690 0x555613f39af0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f0e0b0 0x555613f07c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f02120 0x555613f076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 --2- 172.21.15.60:0/2905952023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f01800 0x555613f071b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 >> 172.21.15.60:0/2905952023 conn(0x555613eff630 msgr2=0x555613f051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9dcb7fe640 0 client.18903 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 shutdown_connections 2024-07-24T22:03:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.549+0000 7f9df47ef740 1 -- 172.21.15.60:0/2905952023 wait complete. 2024-07-24T22:03:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f42c8018030 con 0x559de06311e0 2024-07-24T22:03:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 10 client.28259 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 10 client.28259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 20 client.28259 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de064cd00 0x559de064f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 4 client.28259 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559de05a81d0 con 0x559de064cd00 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.550+0000 7f42de423640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de064cd00 0x559de064f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.551+0000 7f42c17fa640 10 client.28259 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.551+0000 7f42de423640 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de064cd00 0x559de064f160 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca1f3faa6ab5d2c server_cookie=82debb6dce4d04b in_seq=0 out_seq=0 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.552+0000 7f42c17fa640 1 -- 172.21.15.60:0/3224358909 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559de05a81d0 con 0x559de064cd00 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.552+0000 7f42c17fa640 10 client.28259 handle_command_reply: tid=0 2024-07-24T22:03:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.552+0000 7f42e2bd8740 1 client.28259 shutdown 2024-07-24T22:03:24.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f9df47ef740 20 client.18903 trim_cache size 0 max 16384 2024-07-24T22:03:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 msgr2=0x559de0639d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f42c80058c0 tx=0x7f42c8011be0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 shutdown_connections 2024-07-24T22:03:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de064cd00 0x559de064f160 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de063a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0639d80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3224358909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de063c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 >> 172.21.15.60:0/3224358909 conn(0x559de062e6f0 msgr2=0x559de0634290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.556+0000 7f42c17fa640 0 client.28259 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.559+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 shutdown_connections 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.559+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3224358909 wait complete. 2024-07-24T22:03:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.559+0000 7f42e2bd8740 20 client.28259 trim_cache size 0 max 16384 2024-07-24T22:03:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.590+0000 7f9df47ef740 1 Processor -- start 2024-07-24T22:03:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.591+0000 7f9df47ef740 1 -- start start 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.591+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f3f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.591+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.591+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613f13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9debfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f3f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9debfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f3f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48162/0 (socket says 172.21.15.60:48162) 2024-07-24T22:03:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41968/0 (socket says 172.21.15.60:41968) 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9debfff640 1 -- 172.21.15.60:0/763232110 learned_addr learned my addr 172.21.15.60:0/763232110 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80113c0 con 0x555613f0e0b0 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd800ebd0 con 0x555613f02120 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.592+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd800ca40 con 0x555613f01800 2024-07-24T22:03:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9df09db640 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613f13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 -- 172.21.15.60:0/763232110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 msgr2=0x555613f3f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f3f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 -- 172.21.15.60:0/763232110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 msgr2=0x555613f13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613f13690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 -- 172.21.15.60:0/763232110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555613f00000 con 0x555613f02120 2024-07-24T22:03:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.593+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555613f3a330 tx=0x7f9de0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a53daea654c8d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.594+0000 7f42e2bd8740 1 Processor -- start 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.594+0000 7f42e2bd8740 1 -- start start 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 0x559de0654a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0654f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de0645160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc011110 con 0x559de06308c0 2024-07-24T22:03:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00e920 con 0x559de06311e0 2024-07-24T22:03:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00c820 con 0x559de063d170 2024-07-24T22:03:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.595+0000 7f42dd421640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0654f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42de423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de0645160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42de423640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 0x559de0654a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42de423640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 0x559de0654a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50438/0 (socket says 172.21.15.60:50438) 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42de423640 1 -- 172.21.15.60:0/2186425186 learned_addr learned my addr 172.21.15.60:0/2186425186 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42dd421640 1 -- 172.21.15.60:0/2186425186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 msgr2=0x559de0645160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42dd421640 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de0645160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42dd421640 1 -- 172.21.15.60:0/2186425186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 msgr2=0x559de0654a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42dd421640 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 0x559de0654a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.596+0000 7f42dd421640 1 -- 172.21.15.60:0/2186425186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de0631010 con 0x559de06311e0 2024-07-24T22:03:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.597+0000 7f42dd421640 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0654f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92734142cfb88822 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.599+0000 7f42c37fe640 1 -- 172.21.15.60:0/2186425186 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8019830 con 0x559de06311e0 2024-07-24T22:03:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.599+0000 7f42c37fe640 1 -- 172.21.15.60:0/2186425186 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c80053c0 con 0x559de06311e0 2024-07-24T22:03:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 -- 172.21.15.60:0/2186425186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 msgr2=0x559de0654f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0654f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c800f790 comp rx=0 tx=0).stop 2024-07-24T22:03:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.598+0000 7f9dd1ffb640 1 -- 172.21.15.60:0/763232110 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0016070 con 0x555613f02120 2024-07-24T22:03:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.598+0000 7f9dd1ffb640 1 -- 172.21.15.60:0/763232110 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de00047a0 con 0x555613f02120 2024-07-24T22:03:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.598+0000 7f9dd1ffb640 1 -- 172.21.15.60:0/763232110 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de001c7d0 con 0x555613f02120 2024-07-24T22:03:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.599+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 msgr2=0x555613f3f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.599+0000 7f9df47ef740 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x555613f3a330 tx=0x7f9de0012000 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.599+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 shutdown_connections 2024-07-24T22:03:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613f13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f3f900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 -- 172.21.15.60:0/2186425186 shutdown_connections 2024-07-24T22:03:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de063d170 0x559de0645160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de06311e0 0x559de0654f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/2186425186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de06308c0 0x559de0654a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f42e2bd8740 1 -- 172.21.15.60:0/2186425186 >> 172.21.15.60:0/2186425186 conn(0x559de062e6f0 msgr2=0x559de06385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 --2- 172.21.15.60:0/763232110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f3f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 >> 172.21.15.60:0/763232110 conn(0x555613eff630 msgr2=0x555613f062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 shutdown_connections 2024-07-24T22:03:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 -- 172.21.15.60:0/763232110 wait complete. 2024-07-24T22:03:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.600+0000 7f9df47ef740 1 -- start start 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.601+0000 7f9df47ef740 1 Processor -- start 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.601+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9debfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9debfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:48184/0 (socket says 172.21.15.60:48184) 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f1f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:41974/0 (socket says 172.21.15.60:41974) 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9df47ef740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613fa0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9debfff640 1 -- 172.21.15.60:0/1734459267 learned_addr learned my addr 172.21.15.60:0/1734459267 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd800d090 con 0x555613f0e0b0 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd800e500 con 0x555613f02120 2024-07-24T22:03:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.602+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd800d760 con 0x555613f01800 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f42e2bd8740 1 -- 172.21.15.60:0/2186425186 shutdown_connections 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f42e2bd8740 1 -- 172.21.15.60:0/2186425186 wait complete. 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f42e2bd8740 1 -- start start 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f42e2bd8740 1 Processor -- start 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de0645170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de06456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de0642140 0x559de06425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00cde0 con 0x559de063d170 2024-07-24T22:03:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00e250 con 0x559de0642140 2024-07-24T22:03:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc00d4b0 con 0x559de06308c0 2024-07-24T22:03:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.604+0000 7f42e2bd8740 10 client.0 _ll_register_callbacks cb 0x7f42c800c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9df47ef740 10 client.0 _ll_register_callbacks cb 0x7f9de000a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 -- 172.21.15.60:0/1734459267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 msgr2=0x555613f1f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f1f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 -- 172.21.15.60:0/1734459267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 msgr2=0x555613fa0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613fa0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555613fa0910 con 0x555613f01800 2024-07-24T22:03:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de00047a0 con 0x555613f01800 2024-07-24T22:03:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.603+0000 7f9debfff640 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc018560 tx=0x7f9ddc011530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d364c7cc2d628e4b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f9dcb7fe640 10 client.28153 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/1734459267 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc027050 con 0x555613f01800 2024-07-24T22:03:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f42dd421640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de06456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f42dd421640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de06456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50456/0 (socket says 172.21.15.60:50456) 2024-07-24T22:03:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f42dd421640 1 -- 172.21.15.60:0/3461002348 learned_addr learned my addr 172.21.15.60:0/3461002348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/1734459267 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f9ddc018b70 con 0x555613f01800 2024-07-24T22:03:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f42de423640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de0645170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.605+0000 7f42de423640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de0642140 0x559de06425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.606+0000 7f42de423640 1 -- 172.21.15.60:0/3461002348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de0642140 msgr2=0x559de06425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.606+0000 7f42de423640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de0642140 0x559de06425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.606+0000 7f42de423640 1 -- 172.21.15.60:0/3461002348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 msgr2=0x559de06456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.606+0000 7f42de423640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de06456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.606+0000 7f42de423640 1 -- 172.21.15.60:0/3461002348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de0642b00 con 0x559de06308c0 2024-07-24T22:03:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.610+0000 7f42de423640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de0645170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009530 tx=0x7f42d000dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd41859a7c88c0c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.611+0000 7f42c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:24.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.613+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/1734459267 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9ddc018d70 con 0x555613f01800 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.615+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0018040 con 0x559de06308c0 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.615+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559de05a3c70 con 0x559de06308c0 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.615+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f42d00240d0 con 0x559de06308c0 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.616+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f42d0013e20 con 0x559de06308c0 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.616+0000 7f42e2bd8740 10 client.28156 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.616+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559de06b6680 con 0x559de06308c0 2024-07-24T22:03:24.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.615+0000 7f9df47ef740 10 client.28153 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:24.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.615+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=46}) -- 0x555613fa1aa0 con 0x555613f01800 2024-07-24T22:03:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559de05a3c70 con 0x559de06308c0 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/1734459267 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f9ddc031020 con 0x555613f01800 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f9df47ef740 10 client.28153 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f9df47ef740 10 client.28153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f9df47ef740 20 client.28153 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.617+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613fa2150 0x555613fa4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f9df47ef740 4 client.28153 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:03:24.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555613e76350 con 0x555613fa2150 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f42d0029040 con 0x559de06308c0 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 10 client.28156 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 10 client.28156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 20 client.28156 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de06b6ee0 0x559de06b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 4 client.28156 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:03:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559de05a13b0 con 0x559de06b6ee0 2024-07-24T22:03:24.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613fa2150 0x555613fa4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f42dd421640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de06b6ee0 0x559de06b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:24.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f9deb7fe640 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613fa2150 0x555613fa4570 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fc778876292607 server_cookie=6f4aef97844582b2 in_seq=0 out_seq=0 2024-07-24T22:03:24.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.618+0000 7f9dcb7fe640 10 client.28153 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.619+0000 7f9dcb7fe640 1 -- 172.21.15.60:0/1734459267 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555613e76350 con 0x555613fa2150 2024-07-24T22:03:24.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.619+0000 7f9dcb7fe640 10 client.28153 handle_command_reply: tid=0 2024-07-24T22:03:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.619+0000 7f42dd421640 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de06b6ee0 0x559de06b9300 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d591612b20c8aa9 server_cookie=77d157b942690b45 in_seq=0 out_seq=0 2024-07-24T22:03:24.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.619+0000 7f9df47ef740 1 client.28153 shutdown 2024-07-24T22:03:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.619+0000 7f42c17fa640 10 client.28156 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.620+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 msgr2=0x555613f1f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.620+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc018560 tx=0x7f9ddc011530 comp rx=0 tx=0).stop 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 shutdown_connections 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x555613fa2150 0x555613fa4570 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555613f0e0b0 0x555613fa0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555613f02120 0x555613f1f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 --2- 172.21.15.60:0/1734459267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555613f01800 0x555613f1f3c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 >> 172.21.15.60:0/1734459267 conn(0x555613eff630 msgr2=0x555613f059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.621+0000 7f9dcb7fe640 0 client.28153 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.622+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 shutdown_connections 2024-07-24T22:03:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.622+0000 7f9df47ef740 1 -- 172.21.15.60:0/1734459267 wait complete. 2024-07-24T22:03:24.629 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:24.629 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:03:24.629 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:03:24.629 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:24.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.622+0000 7f9df47ef740 20 client.28153 trim_cache size 0 max 16384 2024-07-24T22:03:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 msgr2=0x7f9dd40b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 0x7f9dd40b8660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9dec105a40 tx=0x7f9de000a2d0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 msgr2=0x7f9dec19c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9de402aee0 tx=0x7f9de4031a20 comp rx=0 tx=0).stop 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 shutdown_connections 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9dd40b61a0 0x7f9dd40b8660 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dec107870 0x7f9dec19c120 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec106ef0 0x7f9dec196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 --2- 172.21.15.60:0/1815036366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec1014e0 0x7f9dec196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 >> 172.21.15.60:0/1815036366 conn(0x7f9dec0fcf90 msgr2=0x7f9dec0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 shutdown_connections 2024-07-24T22:03:24.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:24.624+0000 7f9df243e640 1 -- 172.21.15.60:0/1815036366 wait complete. 2024-07-24T22:03:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42c17fa640 1 -- 172.21.15.60:0/3461002348 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559de05a13b0 con 0x559de06b6ee0 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42c17fa640 10 client.28156 handle_command_reply: tid=0 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 client.28156 shutdown 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 msgr2=0x559de0645170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de0645170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009530 tx=0x7f42d000dfd0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 shutdown_connections 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x559de06b6ee0 0x559de06b9300 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559de0642140 0x559de06425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559de063d170 0x559de06456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 --2- 172.21.15.60:0/3461002348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559de06308c0 0x559de0645170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 >> 172.21.15.60:0/3461002348 conn(0x559de062e6f0 msgr2=0x559de063b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42c17fa640 0 client.28156 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 shutdown_connections 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 1 -- 172.21.15.60:0/3461002348 wait complete. 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.699+0000 7f42e2bd8740 20 client.28156 trim_cache size 0 max 16384 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:03:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.701+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 msgr2=0x7f42c40b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.701+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 0x7f42c40b62b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x559de0633a40 tx=0x7f42d403b470 comp rx=0 tx=0).stop 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.701+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 msgr2=0x7f42d811af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.701+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f42d401aa30 tx=0x7f42d4004a30 comp rx=0 tx=0).stop 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 shutdown_connections 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f42c40b3df0 0x7f42c40b62b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42d811ba90 0x7f42d8117440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42d8109f20 0x7f42d811b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 --2- 172.21.15.60:0/2373171844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8109b40 0x7f42d811af90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 >> 172.21.15.60:0/2373171844 conn(0x7f42d806d760 msgr2=0x7f42d80710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.702+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 shutdown_connections 2024-07-24T22:03:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:24.703+0000 7f42dfe86640 1 -- 172.21.15.60:0/2373171844 wait complete. 2024-07-24T22:03:25.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:03:25.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:03:25.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:25 smithi060 ceph-mon[33000]: pgmap v903: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:25 smithi123 ceph-mon[37143]: pgmap v903: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:25 smithi086 ceph-mon[37128]: pgmap v903: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:26.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:03:27.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:27 smithi060 ceph-mon[33000]: pgmap v904: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:27 smithi123 ceph-mon[37143]: pgmap v904: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:28.039 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:27 smithi086 ceph-mon[37128]: pgmap v904: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-07-24T22:03:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.342+0000 7ff2c6682640 1 -- 172.21.15.60:0/2958103774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c01134f0 msgr2=0x7ff2c01158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.342+0000 7ff2c6682640 1 --2- 172.21.15.60:0/2958103774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c01134f0 0x7ff2c01158e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009ca0 tx=0x7ff2b802a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.344+0000 7ff2c6682640 1 -- 172.21.15.60:0/2958103774 shutdown_connections 2024-07-24T22:03:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.344+0000 7ff2c6682640 1 --2- 172.21.15.60:0/2958103774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c01134f0 0x7ff2c01158e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.344+0000 7ff2c6682640 1 --2- 172.21.15.60:0/2958103774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c0075850 0x7ff2c0112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.344+0000 7ff2c6682640 1 --2- 172.21.15.60:0/2958103774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.344+0000 7ff2c6682640 1 -- 172.21.15.60:0/2958103774 >> 172.21.15.60:0/2958103774 conn(0x7ff2c00fe200 msgr2=0x7ff2c0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.346+0000 7ff2c6682640 1 -- 172.21.15.60:0/2958103774 shutdown_connections 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.346+0000 7ff2c6682640 1 -- 172.21.15.60:0/2958103774 wait complete. 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.347+0000 7ff2c6682640 1 Processor -- start 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.347+0000 7ff2c6682640 1 -- start start 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c01134f0 0x7ff2c010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51356/0 (socket says 172.21.15.60:51356) 2024-07-24T22:03:28.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53466/0 (socket says 172.21.15.60:53466) 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2bf7fe640 1 -- 172.21.15.60:0/1419785718 learned_addr learned my addr 172.21.15.60:0/1419785718 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0118dc0 con 0x7ff2c0075850 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0118c40 con 0x7ff2c01134f0 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0118f40 con 0x7ff2c0074f30 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.348+0000 7ff2c4c1f640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c01134f0 0x7ff2c010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.349+0000 7ff2bf7fe640 1 -- 172.21.15.60:0/1419785718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 msgr2=0x7ff2c0111540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.349+0000 7ff2bf7fe640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0111540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.349+0000 7ff2bf7fe640 1 -- 172.21.15.60:0/1419785718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c01134f0 msgr2=0x7ff2c010b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.349+0000 7ff2bf7fe640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c01134f0 0x7ff2c010b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.349+0000 7ff2bf7fe640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c010c040 con 0x7ff2c0075850 2024-07-24T22:03:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.350+0000 7ff2bf7fe640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e9f0 tx=0x7ff2b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.351+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400cd80 con 0x7ff2c0075850 2024-07-24T22:03:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.351+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b40108e0 con 0x7ff2c0075850 2024-07-24T22:03:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.351+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4018980 con 0x7ff2c0075850 2024-07-24T22:03:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.351+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c010c330 con 0x7ff2c0075850 2024-07-24T22:03:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.351+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c01124e0 con 0x7ff2c0075850 2024-07-24T22:03:28.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.355+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff2b4010430 con 0x7ff2c0075850 2024-07-24T22:03:28.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.355+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0076440 con 0x7ff2c0075850 2024-07-24T22:03:28.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.356+0000 7ff2bd7fa640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 0x7ff2a80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:28.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.356+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7ff2b40e4360 con 0x7ff2c0075850 2024-07-24T22:03:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.357+0000 7ff2bffff640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 0x7ff2a80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.358+0000 7ff2bffff640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 0x7ff2a80b6300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009bd0 tx=0x7ff2b0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:28.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.364+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff2b40aaf60 con 0x7ff2c0075850 2024-07-24T22:03:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.813+0000 7ff2c6682640 1 -- 172.21.15.60:0/1419785718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c0112980 con 0x7ff2c0075850 2024-07-24T22:03:28.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.818+0000 7ff2bd7fa640 1 -- 172.21.15.60:0/1419785718 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7ff2b40af4b0 con 0x7ff2c0075850 2024-07-24T22:03:28.821 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:03:28.821 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:03:28.821 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:03:28.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.825+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 msgr2=0x7ff2a80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:28.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.825+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 0x7ff2a80b6300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009bd0 tx=0x7ff2b0005f90 comp rx=0 tx=0).stop 2024-07-24T22:03:28.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.825+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 msgr2=0x7ff2c0111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:28.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.825+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e9f0 tx=0x7ff2b400eeb0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 shutdown_connections 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff2a80b3e40 0x7ff2a80b6300 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2c01134f0 0x7ff2c010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2c0075850 0x7ff2c0111a80 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 --2- 172.21.15.60:0/1419785718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0074f30 0x7ff2c0111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.829+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 >> 172.21.15.60:0/1419785718 conn(0x7ff2c00fe200 msgr2=0x7ff2c0115660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.830+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 shutdown_connections 2024-07-24T22:03:28.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:28.830+0000 7ff2a6ffd640 1 -- 172.21.15.60:0/1419785718 wait complete. 2024-07-24T22:03:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:29 smithi123 ceph-mon[37143]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1419785718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:29 smithi086 ceph-mon[37128]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:30.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1419785718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:30.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:29 smithi060 ceph-mon[33000]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:30.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1419785718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:30.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:03:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.760+0000 7f5dbac3f640 1 -- 172.21.15.60:0/2338686669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 msgr2=0x7f5db4115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.760+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/2338686669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 0x7f5db4115d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5da8007380 tx=0x7f5da802afd0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 -- 172.21.15.60:0/2338686669 shutdown_connections 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/2338686669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 0x7f5db4115d60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/2338686669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/2338686669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 -- 172.21.15.60:0/2338686669 >> 172.21.15.60:0/2338686669 conn(0x7f5db406dca0 msgr2=0x7f5db406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 -- 172.21.15.60:0/2338686669 shutdown_connections 2024-07-24T22:03:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.764+0000 7f5dbac3f640 1 -- 172.21.15.60:0/2338686669 wait complete. 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.765+0000 7f5dbac3f640 1 Processor -- start 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.765+0000 7f5dbac3f640 1 -- start start 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.765+0000 7f5dbac3f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db411ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5dbac3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5dbac3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 0x7f5db4118fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5dbac3f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4117a60 con 0x7f5db4109da0 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5dbac3f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41178e0 con 0x7f5db410ec70 2024-07-24T22:03:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5dbac3f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4117be0 con 0x7f5db410f610 2024-07-24T22:03:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48694/0 (socket says 172.21.15.60:48694) 2024-07-24T22:03:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 -- 172.21.15.60:0/3193138118 learned_addr learned my addr 172.21.15.60:0/3193138118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db89db640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db411ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:31.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 -- 172.21.15.60:0/3193138118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 msgr2=0x7f5db4118fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 0x7f5db4118fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 -- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 msgr2=0x7f5db411ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db411ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.766+0000 7f5db3fff640 1 -- 172.21.15.60:0/3193138118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4119890 con 0x7f5db410ec70 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.767+0000 7f5db89db640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db411ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.767+0000 7f5db3fff640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00efb0 tx=0x7f5dac00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.768+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac01b070 con 0x7f5db410ec70 2024-07-24T22:03:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.769+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db4119b80 con 0x7f5db410ec70 2024-07-24T22:03:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.769+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db407af70 con 0x7f5db410ec70 2024-07-24T22:03:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.769+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac00ba20 con 0x7f5db410ec70 2024-07-24T22:03:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.769+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac016b90 con 0x7f5db410ec70 2024-07-24T22:03:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.772+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db410a180 con 0x7f5db410ec70 2024-07-24T22:03:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.777+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5dac00bbd0 con 0x7f5db410ec70 2024-07-24T22:03:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.779+0000 7f5db1ffb640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 0x7f5d9c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.779+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f5dac0e4f70 con 0x7f5db410ec70 2024-07-24T22:03:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.780+0000 7f5db89db640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 0x7f5d9c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.781+0000 7f5db89db640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 0x7f5d9c0b63d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b50 tx=0x7f5da4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:31.783+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5dac025070 con 0x7f5db410ec70 2024-07-24T22:03:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:31 smithi060 ceph-mon[33000]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-07-24T22:03:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:31 smithi123 ceph-mon[37143]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-07-24T22:03:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:31 smithi086 ceph-mon[37128]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-07-24T22:03:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.242+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db411a250 con 0x7f5db410ec70 2024-07-24T22:03:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.244+0000 7f5db1ffb640 1 -- 172.21.15.60:0/3193138118 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f5dac0ac240 con 0x7f5db410ec70 2024-07-24T22:03:32.246 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:03:32.247 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:03:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:03:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 msgr2=0x7f5d9c0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 0x7f5d9c0b63d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b50 tx=0x7f5da4005f90 comp rx=0 tx=0).stop 2024-07-24T22:03:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 msgr2=0x7f5db411f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00efb0 tx=0x7f5dac00c800 comp rx=0 tx=0).stop 2024-07-24T22:03:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 shutdown_connections 2024-07-24T22:03:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d9c0b3f10 0x7f5d9c0b63d0 secure :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b50 tx=0x7f5da4005f90 comp rx=0 tx=0).stop 2024-07-24T22:03:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5db410f610 0x7f5db4118fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db410ec70 0x7f5db411f270 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 --2- 172.21.15.60:0/3193138118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5db4109da0 0x7f5db411ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.250+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 >> 172.21.15.60:0/3193138118 conn(0x7f5db406dca0 msgr2=0x7f5db4072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:32.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.253+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 shutdown_connections 2024-07-24T22:03:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:32.254+0000 7f5dbac3f640 1 -- 172.21.15.60:0/3193138118 wait complete. 2024-07-24T22:03:32.855 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:32 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3193138118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:32.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:32 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3193138118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:33.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:32 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3193138118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:33.302 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 1bb2ce4a --release --await' 2024-07-24T22:03:33.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 1bb2ce4a --release --await 2024-07-24T22:03:33.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:33 smithi060 ceph-mon[33000]: pgmap v907: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:33 smithi123 ceph-mon[37143]: pgmap v907: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:33 smithi086 ceph-mon[37128]: pgmap v907: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 1.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:34.047 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.854+0000 7f972f59e640 1 -- 172.21.15.60:0/918371398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728005790 msgr2=0x7f9728098ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.854+0000 7f972f59e640 1 --2- 172.21.15.60:0/918371398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728005790 0x7f9728098ff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f97300685c0 tx=0x7f97300978f0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.856+0000 7f972f59e640 1 -- 172.21.15.60:0/918371398 shutdown_connections 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.856+0000 7f972f59e640 1 --2- 172.21.15.60:0/918371398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728005790 0x7f9728098ff0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.856+0000 7f972f59e640 1 --2- 172.21.15.60:0/918371398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9728004cb0 0x7f97280050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.856+0000 7f972f59e640 1 --2- 172.21.15.60:0/918371398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972809a3f0 0x7f972809a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.856+0000 7f972f59e640 1 -- 172.21.15.60:0/918371398 >> 172.21.15.60:0/918371398 conn(0x7f972800a160 msgr2=0x7f972800c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.858+0000 7f972f59e640 1 -- 172.21.15.60:0/918371398 shutdown_connections 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.858+0000 7f972f59e640 1 -- 172.21.15.60:0/918371398 wait complete. 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.860+0000 7f972f59e640 1 Processor -- start 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.860+0000 7f972f59e640 1 -- start start 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.860+0000 7f972f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.861+0000 7f972f59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728005790 0x7f97281327e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.861+0000 7f972e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.861+0000 7f972e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51400/0 (socket says 172.21.15.60:51400) 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.861+0000 7f972e59c640 1 -- 172.21.15.60:0/2797025483 learned_addr learned my addr 172.21.15.60:0/2797025483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.861+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972809a3f0 0x7f9728137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.862+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f97280abd50 con 0x7f9728004cb0 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.862+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97280abbd0 con 0x7f972809a3f0 2024-07-24T22:03:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.862+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97280abed0 con 0x7f9728005790 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.864+0000 7f972e59c640 1 -- 172.21.15.60:0/2797025483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728005790 msgr2=0x7f97281327e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.864+0000 7f972e59c640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728005790 0x7f97281327e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.864+0000 7f972e59c640 1 -- 172.21.15.60:0/2797025483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972809a3f0 msgr2=0x7f9728137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.864+0000 7f972e59c640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972809a3f0 0x7f9728137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.864+0000 7f972e59c640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9728138350 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.865+0000 7f972e59c640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f972000d950 tx=0x7f972000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.866+0000 7f97277fe640 1 -- 172.21.15.60:0/2797025483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720017070 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.866+0000 7f97277fe640 1 -- 172.21.15.60:0/2797025483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9720009c30 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.866+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9728138640 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.866+0000 7f97277fe640 1 -- 172.21.15.60:0/2797025483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972000f620 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.866+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972813ed00 con 0x7f9728004cb0 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.870+0000 7f9736f32740 1 Processor -- start 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.871+0000 7f9736f32740 1 -- start start 2024-07-24T22:03:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.872+0000 7f9736f32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.872+0000 7f9736f32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f9736f32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac63100 0x55b39ac654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f9736f32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b39abcd9b0 con 0x55b39ac63100 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f9736f32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b39abbceb0 con 0x55b39ac56850 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f9736f32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b39abc0ef0 con 0x55b39ac57170 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:48730/0 (socket says 172.21.15.60:48730) 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f972ed9d640 1 -- 172.21.15.60:0/3557086550 learned_addr learned my addr 172.21.15.60:0/3557086550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.873+0000 7f97277fe640 1 -- 172.21.15.60:0/2797025483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f972000f7c0 con 0x7f9728004cb0 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.874+0000 7f972ed9d640 1 -- 172.21.15.60:0/3557086550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 msgr2=0x55b39ac62bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.874+0000 7f972e59c640 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.875+0000 7f972ed9d640 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac62bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.875+0000 7f972ed9d640 1 -- 172.21.15.60:0/3557086550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac63100 msgr2=0x55b39ac654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.875+0000 7f972ed9d640 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac63100 0x55b39ac654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.875+0000 7f972ed9d640 1 -- 172.21.15.60:0/3557086550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39ab46320 con 0x55b39ac56850 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.875+0000 7f972e59c640 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.876+0000 7f972ed9d640 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9730051e20 tx=0x7f973009e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=badf71d65bedb5 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.876+0000 7f97277fe640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 0x7f97140b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.876+0000 7f9724ff9640 1 -- 172.21.15.60:0/3557086550 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97300a3040 con 0x55b39ac56850 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.876+0000 7f9724ff9640 1 -- 172.21.15.60:0/3557086550 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973009ea50 con 0x55b39ac56850 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.879+0000 7f97277fe640 1 -- 172.21.15.60:0/2797025483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f97200e3ce0 con 0x7f9728004cb0 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.879+0000 7f9724ff9640 1 -- 172.21.15.60:0/3557086550 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973006b910 con 0x55b39ac56850 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 -- 172.21.15.60:0/3557086550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 msgr2=0x55b39ac56c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9730051e20 tx=0x7f973009e4b0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 -- 172.21.15.60:0/3557086550 shutdown_connections 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac63100 0x55b39ac654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f9736f32740 1 --2- 172.21.15.60:0/3557086550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac56850 0x55b39ac56c30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.880+0000 7f972e59c640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 0x7f97140b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.881+0000 7f9736f32740 1 -- 172.21.15.60:0/3557086550 >> 172.21.15.60:0/3557086550 conn(0x55b39ac54680 msgr2=0x55b39ac55820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.881+0000 7f972e59c640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 0x7f97140b63d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b39ac55560 tx=0x7f972000dfd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.882+0000 7f9736f32740 1 -- 172.21.15.60:0/3557086550 shutdown_connections 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.883+0000 7f9736f32740 1 -- 172.21.15.60:0/3557086550 wait complete. 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.883+0000 7f9736f32740 1 -- start start 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 Processor -- start 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55b39aa20700 con 0x55b39ac56850 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.884+0000 7f9736f32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b39ab5f5f0 con 0x55b39ac60c40 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.885+0000 7f9736f32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b39ac76d80 con 0x55b39ac57170 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.885+0000 7f9736f32740 10 client.0 _ll_register_callbacks cb 0x55b39ab7f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.885+0000 7f972dd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.885+0000 7f972dd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53530/0 (socket says 172.21.15.60:53530) 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.885+0000 7f972dd9b640 1 -- 172.21.15.60:0/1381962838 learned_addr learned my addr 172.21.15.60:0/1381962838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972dd9b640 1 -- 172.21.15.60:0/1381962838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 msgr2=0x55b39ac80160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac80160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972dd9b640 1 -- 172.21.15.60:0/1381962838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 msgr2=0x55b39ac601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.886+0000 7f972dd9b640 1 -- 172.21.15.60:0/1381962838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39ab52680 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.887+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.887+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f971c00baf0 tx=0x7f971c00d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf180b81cb0c0ec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.888+0000 7f9711ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.888+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c016070 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.888+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f971c010460 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.888+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973009ea50 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.888+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f971c022050 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.890+0000 7f9736f32740 10 client.28171 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.890+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b39a91d650 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.891+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973009ea50 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f971c00f710 con 0x55b39ac57170 2024-07-24T22:03:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 10 client.28171 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 10 client.28171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 20 client.28171 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39ac84520 0x55b39ac86980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 4 client.28171 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.892+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39abcf900 con 0x55b39ac84520 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.896+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39ac84520 0x55b39ac86980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.898+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39ac84520 0x55b39ac86980 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=291a8ff17956ddc5 server_cookie=2cc9e3b741e32923 in_seq=0 out_seq=0 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.898+0000 7f9711ffb640 10 client.28171 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.901+0000 7f9711ffb640 1 -- 172.21.15.60:0/1381962838 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b39abcf900 con 0x55b39ac84520 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.901+0000 7f9711ffb640 10 client.28171 handle_command_reply: tid=0 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.901+0000 7f9736f32740 1 client.28171 shutdown 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 msgr2=0x55b39ac60700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f971c00baf0 tx=0x7f971c00d2e0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 shutdown_connections 2024-07-24T22:03:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39ac84520 0x55b39ac86980 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac80160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac60700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 --2- 172.21.15.60:0/1381962838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 >> 172.21.15.60:0/1381962838 conn(0x55b39ac54680 msgr2=0x55b39ac5a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9711ffb640 0 client.28171 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.903+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 shutdown_connections 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.904+0000 7f9736f32740 1 -- 172.21.15.60:0/1381962838 wait complete. 2024-07-24T22:03:34.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.904+0000 7f9736f32740 20 client.28171 trim_cache size 0 max 16384 2024-07-24T22:03:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.932+0000 7f9736f32740 1 Processor -- start 2024-07-24T22:03:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.932+0000 7f9736f32740 1 -- start start 2024-07-24T22:03:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.932+0000 7f9736f32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f9736f32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 0x55b39ac8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f9736f32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac60c40 0x55b39ac8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51414/0 (socket says 172.21.15.60:51414) 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f972ed9d640 1 -- 172.21.15.60:0/452985334 learned_addr learned my addr 172.21.15.60:0/452985334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f972dd9b640 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 0x55b39ac8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9728141940 con 0x55b39ac56850 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f972813f150 con 0x55b39ac57170 2024-07-24T22:03:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.933+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97281333d0 con 0x55b39ac60c40 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 -- 172.21.15.60:0/452985334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac60c40 msgr2=0x55b39ac8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac60c40 0x55b39ac8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 -- 172.21.15.60:0/452985334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 msgr2=0x55b39ac7ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 -- 172.21.15.60:0/452985334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39ac544b0 con 0x55b39ac57170 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.934+0000 7f972dd9b640 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 0x55b39ac8c290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f971c00bfd0 tx=0x7f971c00b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73300767dfae1a76 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.935+0000 7f9713fff640 1 -- 172.21.15.60:0/452985334 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c02b070 con 0x55b39ac57170 2024-07-24T22:03:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.935+0000 7f9713fff640 1 -- 172.21.15.60:0/452985334 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971c01f070 con 0x55b39ac57170 2024-07-24T22:03:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.935+0000 7f9713fff640 1 -- 172.21.15.60:0/452985334 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c016070 con 0x55b39ac57170 2024-07-24T22:03:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.936+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 msgr2=0x55b39ac8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.936+0000 7f9736f32740 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 0x55b39ac8c290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f971c00bfd0 tx=0x7f971c00b5c0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.936+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 shutdown_connections 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.936+0000 7f9736f32740 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac60c40 0x55b39ac8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.936+0000 7f9736f32740 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac57170 0x55b39ac8c290 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.937+0000 7f9736f32740 1 --2- 172.21.15.60:0/452985334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.937+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 >> 172.21.15.60:0/452985334 conn(0x55b39ac54680 msgr2=0x55b39ac557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.937+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 shutdown_connections 2024-07-24T22:03:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.937+0000 7f9736f32740 1 -- 172.21.15.60:0/452985334 wait complete. 2024-07-24T22:03:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.938+0000 7f9736f32740 1 -- start start 2024-07-24T22:03:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.938+0000 7f9736f32740 1 Processor -- start 2024-07-24T22:03:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f9736f32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f9736f32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f9736f32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f972dd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f972dd9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:53536/0 (socket says 172.21.15.60:53536) 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f972dd9b640 1 -- 172.21.15.60:0/1434992028 learned_addr learned my addr 172.21.15.60:0/1434992028 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.939+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.940+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f97280a2fd0 con 0x55b39ac56850 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.940+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97280a43c0 con 0x55b39ac60c40 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.940+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97280a3620 con 0x55b39ac57170 2024-07-24T22:03:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.940+0000 7f9736f32740 10 client.0 _ll_register_callbacks cb 0x7f971c005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 -- 172.21.15.60:0/1434992028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 msgr2=0x55b39ac780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 -- 172.21.15.60:0/1434992028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 msgr2=0x55b39ac7c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39ac78860 con 0x55b39ac57170 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972dd9b640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f971c00b810 tx=0x7f971c00ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f42553e9ae21f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f9711ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.941+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c02b070 con 0x55b39ac57170 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.942+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971c01f070 con 0x55b39ac57170 2024-07-24T22:03:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.942+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f971c00f7d0 con 0x55b39ac57170 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.942+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f971c016070 con 0x55b39ac57170 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.942+0000 7f9736f32740 10 client.28174 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.942+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b39acef210 con 0x55b39ac57170 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971c01f070 con 0x55b39ac57170 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f971c003a10 con 0x55b39ac57170 2024-07-24T22:03:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 10 client.28174 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 10 client.28174 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 20 client.28174 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39acef930 0x55b39acf1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 4 client.28174 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.943+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39abcb3a0 con 0x55b39acef930 2024-07-24T22:03:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.944+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39acef930 0x55b39acf1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.945+0000 7f972ed9d640 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39acef930 0x55b39acf1d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce3e3e8678004a6c server_cookie=68156e090d846b8 in_seq=0 out_seq=0 2024-07-24T22:03:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.945+0000 7f9711ffb640 10 client.28174 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:03:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.945+0000 7f9711ffb640 1 -- 172.21.15.60:0/1434992028 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39abcb3a0 con 0x55b39acef930 2024-07-24T22:03:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.945+0000 7f9711ffb640 10 client.28174 handle_command_reply: tid=0 2024-07-24T22:03:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.945+0000 7f9736f32740 1 client.28174 shutdown 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 msgr2=0x55b39ac77b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f971c00b810 tx=0x7f971c00ff30 comp rx=0 tx=0).stop 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 shutdown_connections 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55b39acef930 0x55b39acf1d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b39ac60c40 0x55b39ac780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b39ac57170 0x55b39ac77b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 --2- 172.21.15.60:0/1434992028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b39ac56850 0x55b39ac7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 >> 172.21.15.60:0/1434992028 conn(0x55b39ac54680 msgr2=0x55b39ac617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.951+0000 7f9711ffb640 0 client.28174 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:03:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.955+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 shutdown_connections 2024-07-24T22:03:34.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.956+0000 7f9736f32740 1 -- 172.21.15.60:0/1434992028 wait complete. 2024-07-24T22:03:34.959 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:03:34.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.956+0000 7f9736f32740 20 client.28174 trim_cache size 0 max 16384 2024-07-24T22:03:34.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.957+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 msgr2=0x7f97140b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.957+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 0x7f97140b63d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b39ac55560 tx=0x7f972000dfd0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.958+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 msgr2=0x7f97281322a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:34.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.958+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f972000d950 tx=0x7f972000de10 comp rx=0 tx=0).stop 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 shutdown_connections 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f97140b3f10 0x7f97140b63d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972809a3f0 0x7f9728137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728005790 0x7f97281327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 --2- 172.21.15.60:0/2797025483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9728004cb0 0x7f97281322a0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.960+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 >> 172.21.15.60:0/2797025483 conn(0x7f972800a160 msgr2=0x7f9728096fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:34.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.961+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 shutdown_connections 2024-07-24T22:03:34.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:34.963+0000 7f972f59e640 1 -- 172.21.15.60:0/2797025483 wait complete. 2024-07-24T22:03:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:35 smithi123 ceph-mon[37143]: pgmap v908: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:35 smithi086 ceph-mon[37128]: pgmap v908: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:36.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:35 smithi060 ceph-mon[33000]: pgmap v908: 97 pgs: 38 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 40 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:36.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:03:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.320+0000 7f91cd05f640 1 -- 172.21.15.60:0/727814243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0031dd0 con 0x7f91c811ba90 2024-07-24T22:03:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 -- 172.21.15.60:0/727814243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c811ba90 msgr2=0x7f91c811de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 --2- 172.21.15.60:0/727814243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c811ba90 0x7f91c811de80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009ca0 tx=0x7f91c002a8f0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 -- 172.21.15.60:0/727814243 shutdown_connections 2024-07-24T22:03:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 --2- 172.21.15.60:0/727814243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c811ba90 0x7f91c811de80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 --2- 172.21.15.60:0/727814243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c8109f20 0x7f91c810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 --2- 172.21.15.60:0/727814243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c8110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.321+0000 7f91cf063640 1 -- 172.21.15.60:0/727814243 >> 172.21.15.60:0/727814243 conn(0x7f91c806c4f0 msgr2=0x7f91c806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.328+0000 7f91cf063640 1 -- 172.21.15.60:0/727814243 shutdown_connections 2024-07-24T22:03:37.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.329+0000 7f91cf063640 1 -- 172.21.15.60:0/727814243 wait complete. 2024-07-24T22:03:37.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 Processor -- start 2024-07-24T22:03:37.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 -- start start 2024-07-24T22:03:37.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c81a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c811ba90 0x7f91c81ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91c8121440 con 0x7f91c8109f20 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81212c0 con 0x7f91c8109b40 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.330+0000 7f91cf063640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81215c0 con 0x7f91c811ba90 2024-07-24T22:03:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55596/0 (socket says 172.21.15.60:55596) 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 -- 172.21.15.60:0/849836163 learned_addr learned my addr 172.21.15.60:0/849836163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91ce862640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c811ba90 0x7f91c81ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91ce061640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c81a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 -- 172.21.15.60:0/849836163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c811ba90 msgr2=0x7f91c81ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c811ba90 0x7f91c81ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 -- 172.21.15.60:0/849836163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 msgr2=0x7f91c81a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c81a78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91cd860640 1 -- 172.21.15.60:0/849836163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c81ad970 con 0x7f91c8109f20 2024-07-24T22:03:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.335+0000 7f91ce061640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c81a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.336+0000 7f91cd860640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f91b800d750 tx=0x7f91b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.337+0000 7f91bf7fe640 1 -- 172.21.15.60:0/849836163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8017070 con 0x7f91c8109f20 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.337+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c81adc60 con 0x7f91c8109f20 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.337+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c8111270 con 0x7f91c8109f20 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.337+0000 7f91bf7fe640 1 -- 172.21.15.60:0/849836163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8004020 con 0x7f91c8109f20 2024-07-24T22:03:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.337+0000 7f91bf7fe640 1 -- 172.21.15.60:0/849836163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8003c00 con 0x7f91c8109f20 2024-07-24T22:03:37.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.341+0000 7f91d2676740 1 Processor -- start 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.342+0000 7f91bf7fe640 1 -- 172.21.15.60:0/849836163 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f91b80041d0 con 0x7f91c8109f20 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 -- start start 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c624b0 0x557d60c648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d60bccdd0 con 0x557d60c55c00 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d60bbc2d0 con 0x557d60c624b0 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.344+0000 7f91d2676740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d60bc0310 con 0x557d60c56520 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55608/0 (socket says 172.21.15.60:55608) 2024-07-24T22:03:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 -- 172.21.15.60:0/2662950818 learned_addr learned my addr 172.21.15.60:0/2662950818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91cd860640 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c624b0 0x557d60c648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce061640 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 -- 172.21.15.60:0/2662950818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 msgr2=0x557d60c61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c61f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 -- 172.21.15.60:0/2662950818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c624b0 msgr2=0x557d60c648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c624b0 0x557d60c648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.345+0000 7f91ce862640 1 -- 172.21.15.60:0/2662950818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6091cb00 con 0x557d60c55c00 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.346+0000 7f91bf7fe640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 0x7f91b40b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.346+0000 7f91bf7fe640 1 -- 172.21.15.60:0/849836163 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f91b80e36e0 con 0x7f91c8109f20 2024-07-24T22:03:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.348+0000 7f91ce061640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 0x7f91b40b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.348+0000 7f91ce862640 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f91c002ae60 tx=0x7f91c0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ced22649527baa6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.349+0000 7f91ce061640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 0x7f91b40b64d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91c40058c0 tx=0x7f91c4005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.351+0000 7f91bcde4640 1 -- 172.21.15.60:0/2662950818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0030670 con 0x557d60c55c00 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.351+0000 7f91bcde4640 1 -- 172.21.15.60:0/2662950818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0036040 con 0x557d60c55c00 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.352+0000 7f91d2676740 1 -- 172.21.15.60:0/2662950818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 msgr2=0x557d60c55fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.352+0000 7f91d2676740 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f91c002ae60 tx=0x7f91c0005ef0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.354+0000 7f91d2676740 1 -- 172.21.15.60:0/2662950818 shutdown_connections 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.354+0000 7f91d2676740 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c624b0 0x557d60c648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.354+0000 7f91d2676740 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.354+0000 7f91d2676740 1 --2- 172.21.15.60:0/2662950818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c55c00 0x557d60c55fe0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.354+0000 7f91d2676740 1 -- 172.21.15.60:0/2662950818 >> 172.21.15.60:0/2662950818 conn(0x557d60c53a30 msgr2=0x557d60c54bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.360+0000 7f91d2676740 1 -- 172.21.15.60:0/2662950818 shutdown_connections 2024-07-24T22:03:37.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.364+0000 7f91d2676740 1 -- 172.21.15.60:0/2662950818 wait complete. 2024-07-24T22:03:37.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 -- start start 2024-07-24T22:03:37.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 Processor -- start 2024-07-24T22:03:37.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557d60a1fbb0 con 0x557d60c624b0 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d60b5e900 con 0x557d60c55c00 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d60c650c0 con 0x557d60c56520 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.365+0000 7f91d2676740 10 client.0 _ll_register_callbacks cb 0x557d60b68c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.366+0000 7f91ce061640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.366+0000 7f91ce061640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49664/0 (socket says 172.21.15.60:49664) 2024-07-24T22:03:37.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.366+0000 7f91ce061640 1 -- 172.21.15.60:0/2771940447 learned_addr learned my addr 172.21.15.60:0/2771940447 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.366+0000 7f91ce862640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91cd860640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 -- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 msgr2=0x557d60c5f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 -- 172.21.15.60:0/2771940447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 msgr2=0x557d60c5fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 -- 172.21.15.60:0/2771940447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d60993960 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.367+0000 7f91ce061640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b880 tx=0x7f91c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8f8c2e8c3c0784 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4023070 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c0036040 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f91c40118b0 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f91c402f050 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91d2676740 10 client.28307 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.368+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557d60c70cc0 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.369+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0036040 con 0x557d60c55c00 2024-07-24T22:03:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f91c400f040 con 0x557d60c55c00 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 10 client.28307 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 10 client.28307 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 20 client.28307 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60c72170 0x557d60c745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 4 client.28307 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.370+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d60bced20 con 0x557d60c72170 2024-07-24T22:03:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.371+0000 7f91cd860640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60c72170 0x557d60c745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.374+0000 7f91cd860640 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60c72170 0x557d60c745d0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9301db5c722a80 server_cookie=1519cbbb4cb62dff in_seq=0 out_seq=0 2024-07-24T22:03:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.374+0000 7f91b1ffb640 10 client.28307 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.375+0000 7f91b1ffb640 1 -- 172.21.15.60:0/2771940447 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d60bced20 con 0x557d60c72170 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.375+0000 7f91b1ffb640 10 client.28307 handle_command_reply: tid=0 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.375+0000 7f91d2676740 1 client.28307 shutdown 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.376+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 msgr2=0x557d60c61f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.376+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b880 tx=0x7f91c4004730 comp rx=0 tx=0).stop 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 shutdown_connections 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60c72170 0x557d60c745d0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 --2- 172.21.15.60:0/2771940447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c61f20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 >> 172.21.15.60:0/2771940447 conn(0x557d60c53a30 msgr2=0x557d60c595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91b1ffb640 0 client.28307 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 shutdown_connections 2024-07-24T22:03:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.380+0000 7f91d2676740 1 -- 172.21.15.60:0/2771940447 wait complete. 2024-07-24T22:03:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.381+0000 7f91d2676740 20 client.28307 trim_cache size 0 max 16384 2024-07-24T22:03:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.419+0000 7f91d2676740 1 Processor -- start 2024-07-24T22:03:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 -- start start 2024-07-24T22:03:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 0x557d60c59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c56520 0x557d60c79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac0113e0 con 0x557d60c624b0 2024-07-24T22:03:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00ebf0 con 0x557d60c56520 2024-07-24T22:03:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.422+0000 7f91d2676740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00ca60 con 0x557d60c55c00 2024-07-24T22:03:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.423+0000 7f91cd860640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c56520 0x557d60c79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.423+0000 7f91ce862640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.423+0000 7f91ce862640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c7a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55638/0 (socket says 172.21.15.60:55638) 2024-07-24T22:03:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.423+0000 7f91ce862640 1 -- 172.21.15.60:0/58513556 learned_addr learned my addr 172.21.15.60:0/58513556 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.424+0000 7f91ce061640 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 0x557d60c59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.424+0000 7f91ce061640 1 -- 172.21.15.60:0/58513556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c56520 msgr2=0x557d60c79ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.425+0000 7f91ce061640 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c56520 0x557d60c79ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.425+0000 7f91ce061640 1 -- 172.21.15.60:0/58513556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 msgr2=0x557d60c7a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.425+0000 7f91ce061640 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c7a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.425+0000 7f91ce061640 1 -- 172.21.15.60:0/58513556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d60c5fe70 con 0x557d60c55c00 2024-07-24T22:03:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.429+0000 7f91ce061640 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 0x557d60c59450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b830 tx=0x7f91c401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423a698388e2150b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.434+0000 7f91b3fff640 1 -- 172.21.15.60:0/58513556 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4023040 con 0x557d60c55c00 2024-07-24T22:03:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.434+0000 7f91b3fff640 1 -- 172.21.15.60:0/58513556 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c4037030 con 0x557d60c55c00 2024-07-24T22:03:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.434+0000 7f91b3fff640 1 -- 172.21.15.60:0/58513556 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c401f6b0 con 0x557d60c55c00 2024-07-24T22:03:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 -- 172.21.15.60:0/58513556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 msgr2=0x557d60c59450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 0x557d60c59450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b830 tx=0x7f91c401f000 comp rx=0 tx=0).stop 2024-07-24T22:03:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 -- 172.21.15.60:0/58513556 shutdown_connections 2024-07-24T22:03:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c7a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c56520 0x557d60c79ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 --2- 172.21.15.60:0/58513556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c55c00 0x557d60c59450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.435+0000 7f91d2676740 1 -- 172.21.15.60:0/58513556 >> 172.21.15.60:0/58513556 conn(0x557d60c53a30 msgr2=0x557d60c58d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.440+0000 7f91d2676740 1 -- 172.21.15.60:0/58513556 shutdown_connections 2024-07-24T22:03:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.443+0000 7f91d2676740 1 -- 172.21.15.60:0/58513556 wait complete. 2024-07-24T22:03:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.443+0000 7f91d2676740 1 -- start start 2024-07-24T22:03:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.443+0000 7f91d2676740 1 Processor -- start 2024-07-24T22:03:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00d0b0 con 0x557d60c624b0 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00e520 con 0x557d60c55c00 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00d780 con 0x557d60c56520 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91d2676740 10 client.0 _ll_register_callbacks cb 0x7f91c400bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91cd860640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91cd860640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37878/0 (socket says 172.21.15.60:37878) 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91cd860640 1 -- 172.21.15.60:0/3245466125 learned_addr learned my addr 172.21.15.60:0/3245466125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 -- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 msgr2=0x557d60c5c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 -- 172.21.15.60:0/3245466125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 msgr2=0x557d60c5cb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5cb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.444+0000 7f91ce061640 1 -- 172.21.15.60:0/3245466125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d60cdbeb0 con 0x557d60c55c00 2024-07-24T22:03:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.445+0000 7f91ce061640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c6ad20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f91c4011fa0 tx=0x7f91c400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1f724a82969c57 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4040020 con 0x557d60c55c00 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f91c4004220 con 0x557d60c55c00 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c4037030 con 0x557d60c55c00 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f91c4023050 con 0x557d60c55c00 2024-07-24T22:03:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91d2676740 10 client.28313 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.448+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557d60cdd1a0 con 0x557d60c55c00 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c4037030 con 0x557d60c55c00 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f91c401f690 con 0x557d60c55c00 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 10 client.28313 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 10 client.28313 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 20 client.28313 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60cdd810 0x557d60cdfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 4 client.28313 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1bb2ce4a", "release": true, "await": true}] 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.449+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1bb2ce4a", "release": true, "await": true}) -- 0x557d60c4f150 con 0x557d60cdd810 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.450+0000 7f91cd860640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60cdd810 0x557d60cdfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.450+0000 7f91b1ffb640 10 client.28313 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.451+0000 7f91cd860640 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60cdd810 0x557d60cdfc30 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f41d980fb57ad11 server_cookie=2b15a287866d0129 in_seq=0 out_seq=0 2024-07-24T22:03:37.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:37 smithi060 ceph-mon[33000]: pgmap v909: 97 pgs: 36 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:03:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbee3fff640 1 -- 172.21.15.60:0/2235425867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80318a0 con 0x7fbeec075850 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 -- 172.21.15.60:0/2235425867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 msgr2=0x7fbeec112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 --2- 172.21.15.60:0/2235425867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec112ed0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009900 tx=0x7fbee802a400 comp rx=0 tx=0).stop 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 -- 172.21.15.60:0/2235425867 shutdown_connections 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 --2- 172.21.15.60:0/2235425867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec113410 0x7fbeec115800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 --2- 172.21.15.60:0/2235425867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec112ed0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 --2- 172.21.15.60:0/2235425867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec074f30 0x7fbeec075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.864+0000 7fbef32be640 1 -- 172.21.15.60:0/2235425867 >> 172.21.15.60:0/2235425867 conn(0x7fbeec0fe100 msgr2=0x7fbeec100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.865+0000 7fbef32be640 1 -- 172.21.15.60:0/2235425867 shutdown_connections 2024-07-24T22:03:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.865+0000 7fbef32be640 1 -- 172.21.15.60:0/2235425867 wait complete. 2024-07-24T22:03:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.866+0000 7fbef32be640 1 Processor -- start 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 -- start start 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 0x7fbeec0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec118d10 con 0x7fbeec075850 2024-07-24T22:03:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec118b90 con 0x7fbeec074f30 2024-07-24T22:03:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.868+0000 7fbef32be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec118e90 con 0x7fbeec113410 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef185b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 0x7fbeec0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49708/0 (socket says 172.21.15.60:49708) 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef185b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 0x7fbeec0720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37882/0 (socket says 172.21.15.60:37882) 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 -- 172.21.15.60:0/2075405196 learned_addr learned my addr 172.21.15.60:0/2075405196 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef0859640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 -- 172.21.15.60:0/2075405196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 msgr2=0x7fbeec0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 0x7fbeec0720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 -- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 msgr2=0x7fbeec071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.869+0000 7fbef105a640 1 -- 172.21.15.60:0/2075405196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec10f700 con 0x7fbeec074f30 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbef105a640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d980 tx=0x7fbedc00de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbee27fc640 1 -- 172.21.15.60:0/2075405196 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbeec074f30 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec10f9f0 con 0x7fbeec074f30 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec10ff10 con 0x7fbeec074f30 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbee27fc640 1 -- 172.21.15.60:0/2075405196 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbedc009c30 con 0x7fbeec074f30 2024-07-24T22:03:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.870+0000 7fbee27fc640 1 -- 172.21.15.60:0/2075405196 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00f620 con 0x7fbeec074f30 2024-07-24T22:03:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.874+0000 7fbef6015740 1 Processor -- start 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 -- start start 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d71e0 0x55bb972e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972e3170 0x55bb972e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bb9724c280 con 0x55bb972e3170 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb9723b780 con 0x55bb972d68c0 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbef6015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb9723f7c0 con 0x55bb972d71e0 2024-07-24T22:03:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.875+0000 7fbee27fc640 1 -- 172.21.15.60:0/2075405196 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fbedc010460 con 0x7fbeec074f30 2024-07-24T22:03:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.876+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.876+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49716/0 (socket says 172.21.15.60:49716) 2024-07-24T22:03:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.876+0000 7fbef185b640 1 -- 172.21.15.60:0/3674696024 learned_addr learned my addr 172.21.15.60:0/3674696024 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.876+0000 7fbef105a640 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d71e0 0x55bb972e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef185b640 1 -- 172.21.15.60:0/3674696024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d71e0 msgr2=0x55bb972e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef185b640 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d71e0 0x55bb972e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef185b640 1 -- 172.21.15.60:0/3674696024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972e3170 msgr2=0x55bb972e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef185b640 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972e3170 0x55bb972e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef185b640 1 -- 172.21.15.60:0/3674696024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb96f9f790 con 0x55bb972d68c0 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbee27fc640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 0x7fbed80b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.877+0000 7fbef105a640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 0x7fbed80b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.879+0000 7fbee27fc640 1 -- 172.21.15.60:0/2075405196 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fbedc0e3b10 con 0x7fbeec074f30 2024-07-24T22:03:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.879+0000 7fbef105a640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 0x7fbed80b6230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0e12b0 tx=0x7fbedc0b3040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.879+0000 7fbef185b640 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009fc0 tx=0x7fbee4010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a563134d82076e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.880+0000 7fbed77fe640 1 -- 172.21.15.60:0/3674696024 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4016070 con 0x55bb972d68c0 2024-07-24T22:03:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.880+0000 7fbed77fe640 1 -- 172.21.15.60:0/3674696024 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee4004d20 con 0x55bb972d68c0 2024-07-24T22:03:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.880+0000 7fbed77fe640 1 -- 172.21.15.60:0/3674696024 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4005370 con 0x55bb972d68c0 2024-07-24T22:03:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.882+0000 7fbef6015740 1 -- 172.21.15.60:0/3674696024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 msgr2=0x55bb972d6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.882+0000 7fbef6015740 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009fc0 tx=0x7fbee4010800 comp rx=0 tx=0).stop 2024-07-24T22:03:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.886+0000 7fbef6015740 1 -- 172.21.15.60:0/3674696024 shutdown_connections 2024-07-24T22:03:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.886+0000 7fbef6015740 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972e3170 0x55bb972e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.886+0000 7fbef6015740 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d71e0 0x55bb972e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.886+0000 7fbef6015740 1 --2- 172.21.15.60:0/3674696024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972d6ca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.886+0000 7fbef6015740 1 -- 172.21.15.60:0/3674696024 >> 172.21.15.60:0/3674696024 conn(0x55bb972d46f0 msgr2=0x55bb972d5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.888+0000 7fbef6015740 1 -- 172.21.15.60:0/3674696024 shutdown_connections 2024-07-24T22:03:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.890+0000 7fbef6015740 1 -- 172.21.15.60:0/3674696024 wait complete. 2024-07-24T22:03:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.890+0000 7fbef6015740 1 -- start start 2024-07-24T22:03:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.890+0000 7fbef6015740 1 Processor -- start 2024-07-24T22:03:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.891+0000 7fbef6015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.891+0000 7fbef6015740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.891+0000 7fbef6015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef6015740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bb97175ab0 con 0x55bb972d71e0 2024-07-24T22:03:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef6015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb971c39e0 con 0x55bb972d68c0 2024-07-24T22:03:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef6015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb972e5e50 con 0x55bb972e3170 2024-07-24T22:03:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49732/0 (socket says 172.21.15.60:49732) 2024-07-24T22:03:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef185b640 1 -- 172.21.15.60:0/3971229249 learned_addr learned my addr 172.21.15.60:0/3971229249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef185b640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.892+0000 7fbef6015740 10 client.0 _ll_register_callbacks cb 0x55bb96f4ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.893+0000 7fbef0859640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.895+0000 7fbef185b640 1 -- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 msgr2=0x55bb972e0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.895+0000 7fbef185b640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972e0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.895+0000 7fbef185b640 1 -- 172.21.15.60:0/3971229249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 msgr2=0x55bb972dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.895+0000 7fbef185b640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.895+0000 7fbef185b640 1 -- 172.21.15.60:0/3971229249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb97018620 con 0x55bb972d68c0 2024-07-24T22:03:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.897+0000 7fbef185b640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbee4012db0 tx=0x7fbee4012e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b90d64875876529 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.897+0000 7fbed4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.897+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4022040 con 0x55bb972d68c0 2024-07-24T22:03:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.897+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fbee4004c40 con 0x55bb972d68c0 2024-07-24T22:03:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.898+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb971eace0 con 0x55bb972d68c0 2024-07-24T22:03:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.901+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbee402f050 con 0x55bb972d68c0 2024-07-24T22:03:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.903+0000 7fbef6015740 10 client.28325 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.903+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bb972f1640 con 0x55bb972d68c0 2024-07-24T22:03:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.905+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee4004e40 con 0x55bb972d68c0 2024-07-24T22:03:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.905+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fbee4028070 con 0x55bb972d68c0 2024-07-24T22:03:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.905+0000 7fbef6015740 10 client.28325 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.905+0000 7fbef6015740 10 client.28325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.906+0000 7fbef6015740 20 client.28325 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.906+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb972f29f0 0x55bb972f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.906+0000 7fbef6015740 4 client.28325 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.906+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb971eace0 con 0x55bb972f29f0 2024-07-24T22:03:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.907+0000 7fbef0859640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb972f29f0 0x55bb972f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.915+0000 7fbef0859640 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb972f29f0 0x55bb972f4de0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ded2dc4e528ff5 server_cookie=89529f3cf9609709 in_seq=0 out_seq=0 2024-07-24T22:03:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.915+0000 7fbed4ff9640 10 client.28325 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.916+0000 7fbed4ff9640 1 -- 172.21.15.60:0/3971229249 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb971eace0 con 0x55bb972f29f0 2024-07-24T22:03:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.916+0000 7fbed4ff9640 10 client.28325 handle_command_reply: tid=0 2024-07-24T22:03:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.917+0000 7fbef6015740 1 client.28325 shutdown 2024-07-24T22:03:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.923+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 msgr2=0x55bb972e28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.923+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbee4012db0 tx=0x7fbee4012e10 comp rx=0 tx=0).stop 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 shutdown_connections 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb972f29f0 0x55bb972f4de0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 --2- 172.21.15.60:0/3971229249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e28e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 >> 172.21.15.60:0/3971229249 conn(0x55bb972d46f0 msgr2=0x55bb972da290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.925+0000 7fbed4ff9640 0 client.28325 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.931+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 shutdown_connections 2024-07-24T22:03:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.933+0000 7fbef6015740 1 -- 172.21.15.60:0/3971229249 wait complete. 2024-07-24T22:03:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.933+0000 7fbef6015740 20 client.28325 trim_cache size 0 max 16384 2024-07-24T22:03:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.966+0000 7fbef6015740 1 Processor -- start 2024-07-24T22:03:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.966+0000 7fbef6015740 1 -- start start 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972fa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbed00112d0 con 0x55bb972d71e0 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000eae0 con 0x55bb972d68c0 2024-07-24T22:03:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef6015740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000c950 con 0x55bb972e3170 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef185b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:49744/0 (socket says 172.21.15.60:49744) 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.967+0000 7fbef185b640 1 -- 172.21.15.60:0/2908576698 learned_addr learned my addr 172.21.15.60:0/2908576698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef0859640 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 -- 172.21.15.60:0/2908576698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 msgr2=0x55bb972fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 -- 172.21.15.60:0/2908576698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 msgr2=0x55bb972fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972fa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.968+0000 7fbef185b640 1 -- 172.21.15.60:0/2908576698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb972cffa0 con 0x55bb972d68c0 2024-07-24T22:03:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.969+0000 7fbef185b640 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbee4012620 tx=0x7fbee4012590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fb4b90de5274d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.969+0000 7fbed6ffd640 1 -- 172.21.15.60:0/2908576698 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee403a020 con 0x55bb972d68c0 2024-07-24T22:03:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.970+0000 7fbed6ffd640 1 -- 172.21.15.60:0/2908576698 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee4011cd0 con 0x55bb972d68c0 2024-07-24T22:03:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.970+0000 7fbed6ffd640 1 -- 172.21.15.60:0/2908576698 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4016040 con 0x55bb972d68c0 2024-07-24T22:03:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 -- 172.21.15.60:0/2908576698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 msgr2=0x55bb972e02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbee4012620 tx=0x7fbee4012590 comp rx=0 tx=0).stop 2024-07-24T22:03:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 -- 172.21.15.60:0/2908576698 shutdown_connections 2024-07-24T22:03:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972e3170 0x55bb972fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 --2- 172.21.15.60:0/2908576698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972d68c0 0x55bb972e02a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.973+0000 7fbef6015740 1 -- 172.21.15.60:0/2908576698 >> 172.21.15.60:0/2908576698 conn(0x55bb972d46f0 msgr2=0x55bb972e1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.975+0000 7fbef6015740 1 -- 172.21.15.60:0/2908576698 shutdown_connections 2024-07-24T22:03:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.975+0000 7fbef6015740 1 -- 172.21.15.60:0/2908576698 wait complete. 2024-07-24T22:03:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.975+0000 7fbef6015740 1 -- start start 2024-07-24T22:03:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.975+0000 7fbef6015740 1 Processor -- start 2024-07-24T22:03:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37922/0 (socket says 172.21.15.60:37922) 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 -- 172.21.15.60:0/2096831563 learned_addr learned my addr 172.21.15.60:0/2096831563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972e3170 0x55bb972e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000cfa0 con 0x55bb972d71e0 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000e410 con 0x55bb972e3170 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000d670 con 0x55bb972d68c0 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef6015740 10 client.0 _ll_register_callbacks cb 0x7fbee4012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972e3170 0x55bb972e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 -- 172.21.15.60:0/2096831563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972e3170 msgr2=0x55bb972e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972e3170 0x55bb972e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 -- 172.21.15.60:0/2096831563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 msgr2=0x55bb972e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.977+0000 7fbef185b640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.978+0000 7fbef185b640 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb972e6e40 con 0x55bb972d68c0 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.978+0000 7fbef185b640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005d70 tx=0x7fbee4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369f1bd2353b7dad server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.978+0000 7fbed4ff9640 10 client.28204 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.978+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee4011cd0 con 0x55bb972d68c0 2024-07-24T22:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.978+0000 7fbed4ff9640 1 -- 172.21.15.60:0/2096831563 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee403a070 con 0x55bb972d68c0 2024-07-24T22:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.979+0000 7fbed4ff9640 1 -- 172.21.15.60:0/2096831563 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fbee40042c0 con 0x55bb972d68c0 2024-07-24T22:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.979+0000 7fbed4ff9640 1 -- 172.21.15.60:0/2096831563 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbee4016040 con 0x55bb972d68c0 2024-07-24T22:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.979+0000 7fbef6015740 10 client.28204 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.979+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55bb9735c3c0 con 0x55bb972d68c0 2024-07-24T22:03:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.980+0000 7fbed4ff9640 1 -- 172.21.15.60:0/2096831563 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fbee4028040 con 0x55bb972d68c0 2024-07-24T22:03:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 10 client.28204 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 10 client.28204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 20 client.28204 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb9735ca10 0x55bb9735ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 4 client.28204 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:03:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.981+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbee4011cd0 con 0x55bb9735ca10 2024-07-24T22:03:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.982+0000 7f91b1ffb640 1 -- 172.21.15.60:0/3245466125 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557d60c4f150 con 0x557d60cdd810 2024-07-24T22:03:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.982+0000 7f91b1ffb640 10 client.28313 handle_command_reply: tid=0 2024-07-24T22:03:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.982+0000 7f91d2676740 1 client.28313 shutdown 2024-07-24T22:03:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 msgr2=0x557d60c6ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c6ad20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f91c4011fa0 tx=0x7f91c400be90 comp rx=0 tx=0).stop 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 shutdown_connections 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557d60cdd810 0x557d60cdfc30 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557d60c624b0 0x557d60c5cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d60c56520 0x557d60c5c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 --2- 172.21.15.60:0/3245466125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d60c55c00 0x557d60c6ad20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 >> 172.21.15.60:0/3245466125 conn(0x557d60c53a30 msgr2=0x557d60c611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91b1ffb640 0 client.28313 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 shutdown_connections 2024-07-24T22:03:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.983+0000 7f91d2676740 1 -- 172.21.15.60:0/3245466125 wait complete. 2024-07-24T22:03:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.984+0000 7fbef0859640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb9735ca10 0x55bb9735ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 46, 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 52, 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "1bb2ce4a": { 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 52, 2024-07-24T22:03:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.3, 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.984+0000 7f91d2676740 20 client.28313 trim_cache size 0 max 16384 2024-07-24T22:03:37.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 msgr2=0x7f91b40b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 0x7f91b40b64d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91c40058c0 tx=0x7f91c4005850 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 msgr2=0x7f91c81a7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f91b800d750 tx=0x7f91b800dc10 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 shutdown_connections 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f91b40b4010 0x7f91b40b64d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91c811ba90 0x7f91c81ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91c8109f20 0x7f91c81a7e00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 --2- 172.21.15.60:0/849836163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91c8109b40 0x7f91c81a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 >> 172.21.15.60:0/849836163 conn(0x7f91c806c4f0 msgr2=0x7f91c811bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 shutdown_connections 2024-07-24T22:03:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:37.985+0000 7f91cf063640 1 -- 172.21.15.60:0/849836163 wait complete. 2024-07-24T22:03:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.988+0000 7fbef0859640 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb9735ca10 0x55bb9735ee50 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb5bf3dd769d84d server_cookie=b24db556ce44b43c in_seq=0 out_seq=0 2024-07-24T22:03:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:37.988+0000 7fbed4ff9640 10 client.28204 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:37 smithi123 ceph-mon[37143]: pgmap v909: 97 pgs: 36 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:03:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.025+0000 7fbed4ff9640 1 -- 172.21.15.60:0/2096831563 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbee4011cd0 con 0x55bb9735ca10 2024-07-24T22:03:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.025+0000 7fbed4ff9640 10 client.28204 handle_command_reply: tid=0 2024-07-24T22:03:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.025+0000 7fbef6015740 1 client.28204 shutdown 2024-07-24T22:03:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.025+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 msgr2=0x55bb972eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.025+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005d70 tx=0x7fbee4000f40 comp rx=0 tx=0).stop 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 shutdown_connections 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bb9735ca10 0x55bb9735ee50 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbed4ff9640 0 client.28204 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb972e3170 0x55bb972e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb972d71e0 0x55bb972e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 --2- 172.21.15.60:0/2096831563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb972d68c0 0x55bb972eb7d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 >> 172.21.15.60:0/2096831563 conn(0x55bb972d46f0 msgr2=0x55bb972d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 shutdown_connections 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 1 -- 172.21.15.60:0/2096831563 wait complete. 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.026+0000 7fbef6015740 20 client.28204 trim_cache size 0 max 16384 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:03:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:03:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 msgr2=0x7fbed80b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 0x7fbed80b6230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0e12b0 tx=0x7fbedc0b3040 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 msgr2=0x7fbeec074750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d980 tx=0x7fbedc00de40 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 shutdown_connections 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fbed80b3d70 0x7fbed80b6230 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec113410 0x7fbeec0720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbeec075850 0x7fbeec071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 --2- 172.21.15.60:0/2075405196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeec074f30 0x7fbeec074750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 >> 172.21.15.60:0/2075405196 conn(0x7fbeec0fe100 msgr2=0x7fbeec0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 shutdown_connections 2024-07-24T22:03:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:38.027+0000 7fbef32be640 1 -- 172.21.15.60:0/2075405196 wait complete. 2024-07-24T22:03:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:37 smithi086 ceph-mon[37128]: pgmap v909: 97 pgs: 36 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-24T22:03:39.937 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:39 smithi060 ceph-mon[33000]: pgmap v910: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:39.987 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1bb2ce4a', total seconds elapsed: 40.4 2024-07-24T22:03:39.988 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.3 2024-07-24T22:03:39.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:03:39.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:39 smithi123 ceph-mon[37143]: pgmap v910: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:39 smithi086 ceph-mon[37128]: pgmap v910: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-07-24T22:03:40.694 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7957fff640 1 -- 172.21.15.60:0/1054949508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c0035f0 con 0x7f7960109b40 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 -- 172.21.15.60:0/1054949508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7960109b40 msgr2=0x7f7960110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 --2- 172.21.15.60:0/1054949508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7960109b40 0x7f7960110ba0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f795c009900 tx=0x7f795c02a4d0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 -- 172.21.15.60:0/1054949508 shutdown_connections 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 --2- 172.21.15.60:0/1054949508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796011ba90 0x7f796011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 --2- 172.21.15.60:0/1054949508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f796010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 --2- 172.21.15.60:0/1054949508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7960109b40 0x7f7960110ba0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.473+0000 7f7967a51640 1 -- 172.21.15.60:0/1054949508 >> 172.21.15.60:0/1054949508 conn(0x7f796006d980 msgr2=0x7f796006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.474+0000 7f7967a51640 1 -- 172.21.15.60:0/1054949508 shutdown_connections 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.474+0000 7f7967a51640 1 -- 172.21.15.60:0/1054949508 wait complete. 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 Processor -- start 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 -- start start 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7960109b40 0x7f79601a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f79601a8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 0x7f79601ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7960121550 con 0x7f796011ba90 2024-07-24T22:03:41.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79601213d0 con 0x7f7960109f20 2024-07-24T22:03:41.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.475+0000 7f7967a51640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79601216d0 con 0x7f7960109b40 2024-07-24T22:03:41.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.478+0000 7f7965fee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 0x7f79601ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.478+0000 7f7965fee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 0x7f79601ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55714/0 (socket says 172.21.15.60:55714) 2024-07-24T22:03:41.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.478+0000 7f7965fee640 1 -- 172.21.15.60:0/2820413268 learned_addr learned my addr 172.21.15.60:0/2820413268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f79601a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 -- 172.21.15.60:0/2820413268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7960109b40 msgr2=0x7f79601a7b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7960109b40 0x7f79601a7b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 -- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 msgr2=0x7f79601ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 0x7f79601ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.479+0000 7f7964fec640 1 -- 172.21.15.60:0/2820413268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601adbb0 con 0x7f7960109f20 2024-07-24T22:03:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.480+0000 7f7964fec640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f79601a8040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f795000d6d0 tx=0x7f795000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2820413268 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950017070 con 0x7f7960109f20 2024-07-24T22:03:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2820413268 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7950009820 con 0x7f7960109f20 2024-07-24T22:03:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2820413268 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950004b90 con 0x7f7960109f20 2024-07-24T22:03:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.481+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601adea0 con 0x7f7960109f20 2024-07-24T22:03:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.481+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796007ada0 con 0x7f7960109f20 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 Processor -- start 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 -- start start 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bf7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bf7f90 0x557675bfa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557675b519b0 con 0x557675bdaff0 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557675b40eb0 con 0x557675bf7f90 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.485+0000 7f7969e02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557675b44ef0 con 0x557675bda6d0 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.487+0000 7f7965fee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.487+0000 7f7965fee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37948/0 (socket says 172.21.15.60:37948) 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.487+0000 7f7965fee640 1 -- 172.21.15.60:0/1516305266 learned_addr learned my addr 172.21.15.60:0/1516305266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 -- 172.21.15.60:0/1516305266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bf7f90 msgr2=0x557675bfa380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bf7f90 0x557675bfa380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 -- 172.21.15.60:0/1516305266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 msgr2=0x557675bf7a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bf7a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 -- 172.21.15.60:0/1516305266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557675aca320 con 0x557675bda6d0 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f7965fee640 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f79580333f0 tx=0x7f79580338b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92bc1adaac5758d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f794b7fe640 1 -- 172.21.15.60:0/1516305266 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958040070 con 0x557675bda6d0 2024-07-24T22:03:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.488+0000 7f794b7fe640 1 -- 172.21.15.60:0/1516305266 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7958031a00 con 0x557675bda6d0 2024-07-24T22:03:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.489+0000 7f7969e02740 1 -- 172.21.15.60:0/1516305266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 msgr2=0x557675bdaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.489+0000 7f7969e02740 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f79580333f0 tx=0x7f79580338b0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.491+0000 7f79567fc640 1 -- 172.21.15.60:0/2820413268 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7950004d30 con 0x7f7960109f20 2024-07-24T22:03:41.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.492+0000 7f79567fc640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 0x7f794c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f79567fc640 1 -- 172.21.15.60:0/2820413268 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f79500e3ce0 con 0x7f7960109f20 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7965fee640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 0x7f794c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7969e02740 1 -- 172.21.15.60:0/1516305266 shutdown_connections 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7969e02740 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bf7f90 0x557675bfa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7969e02740 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bf7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7969e02740 1 --2- 172.21.15.60:0/1516305266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bda6d0 0x557675bdaab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.493+0000 7f7969e02740 1 -- 172.21.15.60:0/1516305266 >> 172.21.15.60:0/1516305266 conn(0x557675bd8500 msgr2=0x557675bd96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.494+0000 7f7965fee640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 0x7f794c0b6410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7958009fd0 tx=0x7f7958009b70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.496+0000 7f7969e02740 1 -- 172.21.15.60:0/1516305266 shutdown_connections 2024-07-24T22:03:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.496+0000 7f7969e02740 1 -- 172.21.15.60:0/1516305266 wait complete. 2024-07-24T22:03:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.497+0000 7f7969e02740 1 -- start start 2024-07-24T22:03:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.497+0000 7f7969e02740 1 Processor -- start 2024-07-24T22:03:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675be4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5576759a4700 con 0x557675bda6d0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557675ae35f0 con 0x557675bdaff0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557675be0ce0 con 0x557675bf7f90 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7969e02740 10 client.0 _ll_register_callbacks cb 0x557675b03960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7965fee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7965fee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37954/0 (socket says 172.21.15.60:37954) 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7965fee640 1 -- 172.21.15.60:0/1707549287 learned_addr learned my addr 172.21.15.60:0/1707549287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f79657ed640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.498+0000 7f7964fec640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675be4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.500+0000 7f7965fee640 1 -- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 msgr2=0x557675be4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:41.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.500+0000 7f7965fee640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675be4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.500+0000 7f7965fee640 1 -- 172.21.15.60:0/1707549287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 msgr2=0x557675be4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:41.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.500+0000 7f7965fee640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.500+0000 7f7965fee640 1 -- 172.21.15.60:0/1707549287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557675ad6680 con 0x557675bf7f90 2024-07-24T22:03:41.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.501+0000 7f7965fee640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f79580333f0 tx=0x7f795803b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bae0f570a2a2928c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.501+0000 7f7948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.502+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958056060 con 0x557675bf7f90 2024-07-24T22:03:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.502+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f79580449e0 con 0x557675bf7f90 2024-07-24T22:03:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.506+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7958031a00 con 0x557675bf7f90 2024-07-24T22:03:41.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.507+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7958040040 con 0x557675bf7f90 2024-07-24T22:03:41.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.507+0000 7f7969e02740 10 client.28213 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:41.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.507+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5576758a1650 con 0x557675bf7f90 2024-07-24T22:03:41.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.509+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7958031a00 con 0x557675bf7f90 2024-07-24T22:03:41.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.512+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f795804e070 con 0x557675bf7f90 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 10 client.28213 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 10 client.28213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 20 client.28213 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675bf4520 0x557675bf6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 4 client.28213 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557675b53900 con 0x557675bf4520 2024-07-24T22:03:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.513+0000 7f79657ed640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675bf4520 0x557675bf6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.514+0000 7f79657ed640 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675bf4520 0x557675bf6940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6561110bf9c9c1b4 server_cookie=618f9122aac647ca in_seq=0 out_seq=0 2024-07-24T22:03:41.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.514+0000 7f7948ff9640 10 client.28213 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:03:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.517+0000 7f7948ff9640 1 -- 172.21.15.60:0/1707549287 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557675b53900 con 0x557675bf4520 2024-07-24T22:03:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.517+0000 7f7948ff9640 10 client.28213 handle_command_reply: tid=0 2024-07-24T22:03:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.517+0000 7f7969e02740 1 client.28213 shutdown 2024-07-24T22:03:41.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.517+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 msgr2=0x557675be9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.517+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f79580333f0 tx=0x7f795803b620 comp rx=0 tx=0).stop 2024-07-24T22:03:41.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 shutdown_connections 2024-07-24T22:03:41.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675bf4520 0x557675bf6940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675be9dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675be4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 --2- 172.21.15.60:0/1707549287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 >> 172.21.15.60:0/1707549287 conn(0x557675bd8500 msgr2=0x557675bde030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.520+0000 7f7948ff9640 0 client.28213 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:03:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.522+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 shutdown_connections 2024-07-24T22:03:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.522+0000 7f7969e02740 1 -- 172.21.15.60:0/1707549287 wait complete. 2024-07-24T22:03:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.522+0000 7f7969e02740 20 client.28213 trim_cache size 0 max 16384 2024-07-24T22:03:41.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.550+0000 7f7969e02740 1 Processor -- start 2024-07-24T22:03:41.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.550+0000 7f7969e02740 1 -- start start 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.550+0000 7f7969e02740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f7969e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675bf84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f7969e02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bf9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f7969e02740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f796007dc30 con 0x557675bda6d0 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f7969e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f796007b440 con 0x557675bdaff0 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f7969e02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7960117cc0 con 0x557675bf7f90 2024-07-24T22:03:41.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f79657ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f79657ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55726/0 (socket says 172.21.15.60:55726) 2024-07-24T22:03:41.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.551+0000 7f79657ed640 1 -- 172.21.15.60:0/112607394 learned_addr learned my addr 172.21.15.60:0/112607394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:41.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 -- 172.21.15.60:0/112607394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 msgr2=0x557675bf9030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bf9030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 -- 172.21.15.60:0/112607394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 msgr2=0x557675bf84b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675bf84b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 -- 172.21.15.60:0/112607394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557675be2a80 con 0x557675bda6d0 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.552+0000 7f79657ed640 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x557675bd93e0 tx=0x7f795c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee555dc693e6e6c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.553+0000 7f794affd640 1 -- 172.21.15.60:0/112607394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c004040 con 0x557675bda6d0 2024-07-24T22:03:41.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.553+0000 7f794affd640 1 -- 172.21.15.60:0/112607394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795c04a070 con 0x557675bda6d0 2024-07-24T22:03:41.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.554+0000 7f7969e02740 1 -- 172.21.15.60:0/112607394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 msgr2=0x557675be2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.554+0000 7f7969e02740 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x557675bd93e0 tx=0x7f795c0049e0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.555+0000 7f7969e02740 1 -- 172.21.15.60:0/112607394 shutdown_connections 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.555+0000 7f7969e02740 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bf9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.555+0000 7f7969e02740 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bdaff0 0x557675bf84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.555+0000 7f7969e02740 1 --2- 172.21.15.60:0/112607394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bda6d0 0x557675be2000 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.555+0000 7f7969e02740 1 -- 172.21.15.60:0/112607394 >> 172.21.15.60:0/112607394 conn(0x557675bd8500 msgr2=0x557675bf9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.556+0000 7f7969e02740 1 -- 172.21.15.60:0/112607394 shutdown_connections 2024-07-24T22:03:41.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.556+0000 7f7969e02740 1 -- 172.21.15.60:0/112607394 wait complete. 2024-07-24T22:03:41.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.557+0000 7f7969e02740 1 -- start start 2024-07-24T22:03:41.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.557+0000 7f7969e02740 1 Processor -- start 2024-07-24T22:03:41.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bda6d0 0x557675bff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bfc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7960118310 con 0x557675bdaff0 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7960119780 con 0x557675bda6d0 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79601189e0 con 0x557675bf7f90 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7969e02740 10 client.0 _ll_register_callbacks cb 0x7f795803e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7965fee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bfc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7964fec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7964fec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55740/0 (socket says 172.21.15.60:55740) 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.558+0000 7f7964fec640 1 -- 172.21.15.60:0/1007830679 learned_addr learned my addr 172.21.15.60:0/1007830679 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.559+0000 7f7964fec640 1 -- 172.21.15.60:0/1007830679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 msgr2=0x557675bfc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.559+0000 7f7964fec640 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bfc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.559+0000 7f7964fec640 1 -- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bda6d0 msgr2=0x557675bff9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.559+0000 7f7964fec640 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bda6d0 0x557675bff9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.559+0000 7f7964fec640 1 -- 172.21.15.60:0/1007830679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557675bfcef0 con 0x557675bdaff0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.560+0000 7f7964fec640 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7950009610 tx=0x7f79500aa140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da61a9d31a25c222 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.560+0000 7f7948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.560+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500aa8b0 con 0x557675bdaff0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.560+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795c04a070 con 0x557675bdaff0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.561+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f79500e15f0 con 0x557675bdaff0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.561+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f79500e17f0 con 0x557675bdaff0 2024-07-24T22:03:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.562+0000 7f7969e02740 10 client.18978 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.562+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557675c61200 con 0x557675bdaff0 2024-07-24T22:03:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.562+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795c04a070 con 0x557675bdaff0 2024-07-24T22:03:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f79500a9b10 con 0x557675bdaff0 2024-07-24T22:03:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 10 client.18978 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 10 client.18978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:03:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 20 client.18978 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675c61a10 0x557675c63e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 4 client.18978 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:03:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557675b4f3a0 con 0x557675c61a10 2024-07-24T22:03:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.563+0000 7f79657ed640 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675c61a10 0x557675c63e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.565+0000 7f79657ed640 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675c61a10 0x557675c63e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af2a73715b5b342d server_cookie=536190368700ef61 in_seq=0 out_seq=0 2024-07-24T22:03:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.565+0000 7f7948ff9640 10 client.18978 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:03:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7948ff9640 1 -- 172.21.15.60:0/1007830679 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557675b4f3a0 con 0x557675c61a10 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7948ff9640 10 client.18978 handle_command_reply: tid=0 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 client.18978 shutdown 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 msgr2=0x557675bfff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7950009610 tx=0x7f79500aa140 comp rx=0 tx=0).stop 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 shutdown_connections 2024-07-24T22:03:41.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x557675c61a10 0x557675c63e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557675bf7f90 0x557675bfc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557675bdaff0 0x557675bfff20 secure :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7950009610 tx=0x7f79500aa140 comp rx=0 tx=0).stop 2024-07-24T22:03:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 --2- 172.21.15.60:0/1007830679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557675bda6d0 0x557675bff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.566+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 >> 172.21.15.60:0/1007830679 conn(0x557675bd8500 msgr2=0x557675bf9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.567+0000 7f7948ff9640 0 client.18978 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:03:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.568+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 shutdown_connections 2024-07-24T22:03:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.568+0000 7f7969e02740 1 -- 172.21.15.60:0/1007830679 wait complete. 2024-07-24T22:03:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.569+0000 7f7969e02740 20 client.18978 trim_cache size 0 max 16384 2024-07-24T22:03:41.574 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:03:41.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.574+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 msgr2=0x7f794c0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.574+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 0x7f794c0b6410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7958009fd0 tx=0x7f7958009b70 comp rx=0 tx=0).stop 2024-07-24T22:03:41.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.574+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 msgr2=0x7f79601a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:41.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.574+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f79601a8040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f795000d6d0 tx=0x7f795000db90 comp rx=0 tx=0).stop 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 shutdown_connections 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f794c0b3f50 0x7f794c0b6410 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796011ba90 0x7f79601ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960109f20 0x7f79601a8040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 --2- 172.21.15.60:0/2820413268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7960109b40 0x7f79601a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.577+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 >> 172.21.15.60:0/2820413268 conn(0x7f796006d980 msgr2=0x7f796011c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.578+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 shutdown_connections 2024-07-24T22:03:41.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:41.578+0000 7f7967a51640 1 -- 172.21.15.60:0/2820413268 wait complete. 2024-07-24T22:03:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:41 smithi060 ceph-mon[33000]: pgmap v911: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-07-24T22:03:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:41 smithi123 ceph-mon[37143]: pgmap v911: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-07-24T22:03:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:41 smithi086 ceph-mon[37128]: pgmap v911: 97 pgs: 34 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-07-24T22:03:43.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:03:43.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:43 smithi060 ceph-mon[33000]: pgmap v912: 97 pgs: 34 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-07-24T22:03:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:43 smithi123 ceph-mon[37143]: pgmap v912: 97 pgs: 34 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-07-24T22:03:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:43 smithi086 ceph-mon[37128]: pgmap v912: 97 pgs: 34 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-07-24T22:03:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 -- 172.21.15.60:0/3515622877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810f580 msgr2=0x7f0ec8115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/3515622877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810f580 0x7f0ec8115d10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0005fd0 tx=0x7f0ec002a2e0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 -- 172.21.15.60:0/3515622877 shutdown_connections 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/3515622877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810f580 0x7f0ec8115d10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/3515622877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec810ec00 0x7f0ec810f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/3515622877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec8109da0 0x7f0ec810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 -- 172.21.15.60:0/3515622877 >> 172.21.15.60:0/3515622877 conn(0x7f0ec806d980 msgr2=0x7f0ec806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 -- 172.21.15.60:0/3515622877 shutdown_connections 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.696+0000 7f0ecd35a640 1 -- 172.21.15.60:0/3515622877 wait complete. 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.698+0000 7f0ecd35a640 1 Processor -- start 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.698+0000 7f0ecd35a640 1 -- start start 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.698+0000 7f0ecd35a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.698+0000 7f0ecd35a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810ec00 0x7f0ec819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ecd35a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec810f580 0x7f0ec81a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ec6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ec6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:37992/0 (socket says 172.21.15.60:37992) 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ec6ffd640 1 -- 172.21.15.60:0/816379075 learned_addr learned my addr 172.21.15.60:0/816379075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec8118d20 con 0x7f0ec810ec00 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec8118ba0 con 0x7f0ec810f580 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec8118ea0 con 0x7f0ec8109da0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.699+0000 7f0ec67fc640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810ec00 0x7f0ec819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 -- 172.21.15.60:0/816379075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec810f580 msgr2=0x7f0ec81a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec810f580 0x7f0ec81a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 -- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810ec00 msgr2=0x7f0ec819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810ec00 0x7f0ec819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec81a5080 con 0x7f0ec8109da0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.700+0000 7f0ec6ffd640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009f20 tx=0x7f0eb800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.701+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800cd80 con 0x7f0ec8109da0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.701+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb8013c80 con 0x7f0ec8109da0 2024-07-24T22:03:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.701+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb801bcc0 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.703+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec81a5370 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.703+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec81aba30 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.706+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0eb80137d0 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.708+0000 7f0eaffff640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 0x7f0eb00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.707+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec8074660 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.708+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f0eb80e8450 con 0x7f0ec8109da0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.709+0000 7f0ec67fc640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 0x7f0eb00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.710+0000 7f0ec67fc640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 0x7f0eb00b6390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806dfc0 tx=0x7f0ebc00b500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:43.714+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f0eb80af7b0 con 0x7f0ec8109da0 2024-07-24T22:03:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.147+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ec81abd10 con 0x7f0ec8109da0 2024-07-24T22:03:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.148+0000 7f0eaffff640 1 -- 172.21.15.60:0/816379075 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f0eb80af0b0 con 0x7f0ec8109da0 2024-07-24T22:03:44.150 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:03:44.151 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:03:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:03:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 msgr2=0x7f0eb00b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 0x7f0eb00b6390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806dfc0 tx=0x7f0ebc00b500 comp rx=0 tx=0).stop 2024-07-24T22:03:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 msgr2=0x7f0ec819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009f20 tx=0x7f0eb800ee80 comp rx=0 tx=0).stop 2024-07-24T22:03:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 shutdown_connections 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0eb00b3ed0 0x7f0eb00b6390 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec810f580 0x7f0ec81a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec810ec00 0x7f0ec819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 --2- 172.21.15.60:0/816379075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8109da0 0x7f0ec819efd0 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009f20 tx=0x7f0eb800ee80 comp rx=0 tx=0).stop 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 >> 172.21.15.60:0/816379075 conn(0x7f0ec806d980 msgr2=0x7f0ec80729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 shutdown_connections 2024-07-24T22:03:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:44.154+0000 7f0ecd35a640 1 -- 172.21.15.60:0/816379075 wait complete. 2024-07-24T22:03:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:44 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/816379075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:44.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0' 2024-07-24T22:03:44.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0 2024-07-24T22:03:44.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:44 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/816379075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:45.039 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:45.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:44 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/816379075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:03:45.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:45 smithi060 ceph-mon[33000]: pgmap v913: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 375 KiB/s wr, 42 op/s 2024-07-24T22:03:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:45 smithi123 ceph-mon[37143]: pgmap v913: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 375 KiB/s wr, 42 op/s 2024-07-24T22:03:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:45 smithi086 ceph-mon[37128]: pgmap v913: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 375 KiB/s wr, 42 op/s 2024-07-24T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.415+0000 7f688e04e640 1 -- 172.21.15.60:0/1868073217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6888101440 msgr2=0x7f6888101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.415+0000 7f688e04e640 1 --2- 172.21.15.60:0/1868073217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6888101440 0x7f6888101820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f687801a910 tx=0x7f687803b470 comp rx=0 tx=0).stop 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 -- 172.21.15.60:0/1868073217 shutdown_connections 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 --2- 172.21.15.60:0/1868073217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 --2- 172.21.15.60:0/1868073217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 --2- 172.21.15.60:0/1868073217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6888101440 0x7f6888101820 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f687801a910 tx=0x7f687803b470 comp rx=0 tx=0).stop 2024-07-24T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 -- 172.21.15.60:0/1868073217 >> 172.21.15.60:0/1868073217 conn(0x7f68880fd0b0 msgr2=0x7f68880ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 -- 172.21.15.60:0/1868073217 shutdown_connections 2024-07-24T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.416+0000 7f688e04e640 1 -- 172.21.15.60:0/1868073217 wait complete. 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 Processor -- start 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 -- start start 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888073820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f688810a150 con 0x7f6888106880 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6888109fd0 con 0x7f6888101d60 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.417+0000 7f688e04e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f688810a2d0 con 0x7f688810cab0 2024-07-24T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6887fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6887fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51660/0 (socket says 172.21.15.60:51660) 2024-07-24T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6887fff640 1 -- 172.21.15.60:0/1558987172 learned_addr learned my addr 172.21.15.60:0/1558987172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f68877fe640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6887fff640 1 -- 172.21.15.60:0/1558987172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 msgr2=0x7f6888073820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.418+0000 7f6887fff640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888073820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f6887fff640 1 -- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 msgr2=0x7f6888073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f6887fff640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f6887fff640 1 -- 172.21.15.60:0/1558987172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688810d4a0 con 0x7f688810cab0 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f68877fe640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.419+0000 7f6887fff640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f688000b970 tx=0x7f688000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.420+0000 7f6884ff9640 1 -- 172.21.15.60:0/1558987172 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880019070 con 0x7f688810cab0 2024-07-24T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.420+0000 7f6884ff9640 1 -- 172.21.15.60:0/1558987172 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6880014440 con 0x7f688810cab0 2024-07-24T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.420+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688810d790 con 0x7f688810cab0 2024-07-24T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.420+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68881ac3f0 con 0x7f688810cab0 2024-07-24T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.421+0000 7f6884ff9640 1 -- 172.21.15.60:0/1558987172 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800075d0 con 0x7f688810cab0 2024-07-24T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.424+0000 7f68903ff740 1 Processor -- start 2024-07-24T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.424+0000 7f6884ff9640 1 -- 172.21.15.60:0/1558987172 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f688001c070 con 0x7f688810cab0 2024-07-24T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.426+0000 7f6884ff9640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 0x7f68700b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.426+0000 7f6884ff9640 1 -- 172.21.15.60:0/1558987172 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f68800a65d0 con 0x7f688810cab0 2024-07-24T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.426+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 0x7f68700b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.426+0000 7f68903ff740 1 -- start start 2024-07-24T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 0x7f68700b6380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f688810fd30 tx=0x7f687c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557c7cb389b0 con 0x557c7cbcdf80 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c7cb27eb0 con 0x557c7cbc16d0 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f68903ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c7cb2bef0 con 0x557c7cbc1ff0 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f6887fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51666/0 (socket says 172.21.15.60:51666) 2024-07-24T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.427+0000 7f6886ffd640 1 -- 172.21.15.60:0/1021717 learned_addr learned my addr 172.21.15.60:0/1021717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f68877fe640 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f6886ffd640 1 -- 172.21.15.60:0/1021717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 msgr2=0x557c7cbc1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f6886ffd640 1 -- 172.21.15.60:0/1021717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 msgr2=0x557c7cbd0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbd0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.428+0000 7f6886ffd640 1 -- 172.21.15.60:0/1021717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7cab1320 con 0x557c7cbc1ff0 2024-07-24T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.429+0000 7f6887fff640 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.431+0000 7f6886ffd640 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f687c00ea80 tx=0x7f687c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f19e47dbd9b2b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.432+0000 7f686dffb640 1 -- 172.21.15.60:0/1021717 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c025070 con 0x557c7cbc1ff0 2024-07-24T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.432+0000 7f686dffb640 1 -- 172.21.15.60:0/1021717 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c015c30 con 0x557c7cbc1ff0 2024-07-24T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.432+0000 7f686dffb640 1 -- 172.21.15.60:0/1021717 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00f040 con 0x557c7cbc1ff0 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 -- 172.21.15.60:0/1021717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 msgr2=0x557c7cbcda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f687c00ea80 tx=0x7f687c011000 comp rx=0 tx=0).stop 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 -- 172.21.15.60:0/1021717 shutdown_connections 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcda40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 --2- 172.21.15.60:0/1021717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 -- 172.21.15.60:0/1021717 >> 172.21.15.60:0/1021717 conn(0x557c7cbbf500 msgr2=0x557c7cbc06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.433+0000 7f68903ff740 1 -- 172.21.15.60:0/1021717 shutdown_connections 2024-07-24T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.436+0000 7f68903ff740 1 -- 172.21.15.60:0/1021717 wait complete. 2024-07-24T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.437+0000 7f68903ff740 1 -- start start 2024-07-24T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.437+0000 7f68903ff740 1 Processor -- start 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.438+0000 7f68903ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.438+0000 7f68903ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.438+0000 7f68903ff740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbcb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.438+0000 7f68877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68903ff740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557c7c98b700 con 0x557c7cbcdf80 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35050/0 (socket says 172.21.15.60:35050) 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68877fe640 1 -- 172.21.15.60:0/2629599384 learned_addr learned my addr 172.21.15.60:0/2629599384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c7caca5f0 con 0x557c7cbc16d0 2024-07-24T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c7cbd0c00 con 0x557c7cbc1ff0 2024-07-24T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f6886ffd640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f68903ff740 10 client.0 _ll_register_callbacks cb 0x557c7caea960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.439+0000 7f6887fff640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbcb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 -- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 msgr2=0x557c7cbcb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 -- 172.21.15.60:0/2629599384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 msgr2=0x557c7cbcaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7cabd680 con 0x557c7cbcdf80 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f68877fe640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.440+0000 7f6887fff640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbcb990 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f68800e0a40 tx=0x7f68800b2ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc9e008e0022c71 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800a67d0 con 0x557c7cbcdf80 2024-07-24T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f688000cb40 con 0x557c7cbcdf80 2024-07-24T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c015c30 con 0x557c7cbcdf80 2024-07-24T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f68800a6970 con 0x557c7cbcdf80 2024-07-24T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f68903ff740 10 client.18993 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.442+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557c7c888650 con 0x557c7cbcdf80 2024-07-24T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68800a9fe0 con 0x557c7cbcdf80 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f68800a9750 con 0x557c7cbcdf80 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f68903ff740 10 client.18993 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f68903ff740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f68903ff740 20 client.18993 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cbdda10 0x557c7cbdfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.443+0000 7f68903ff740 4 client.18993 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.444+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f687c015c30 con 0x557c7cbdda10 2024-07-24T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.444+0000 7f68877fe640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cbdda10 0x557c7cbdfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.444+0000 7f68877fe640 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cbdda10 0x557c7cbdfe00 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3b2b4f6e47fb02 server_cookie=8eac08b2c6307aa2 in_seq=0 out_seq=0 2024-07-24T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.444+0000 7f6866ffd640 10 client.18993 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.445+0000 7f6866ffd640 1 -- 172.21.15.60:0/2629599384 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f687c015c30 con 0x557c7cbdda10 2024-07-24T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.445+0000 7f6866ffd640 10 client.18993 handle_command_reply: tid=0 2024-07-24T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.445+0000 7f68903ff740 1 client.18993 shutdown 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 msgr2=0x557c7cbcb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbcb990 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f68800e0a40 tx=0x7f68800b2ae0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 shutdown_connections 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cbdda10 0x557c7cbdfe00 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbcb990 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.446+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f68903ff740 1 --2- 172.21.15.60:0/2629599384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbcaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 >> 172.21.15.60:0/2629599384 conn(0x557c7cbbf500 msgr2=0x557c7cbc50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f6866ffd640 0 client.18993 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 shutdown_connections 2024-07-24T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f68903ff740 1 -- 172.21.15.60:0/2629599384 wait complete. 2024-07-24T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.447+0000 7f68903ff740 20 client.18993 trim_cache size 0 max 16384 2024-07-24T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.475+0000 7f68903ff740 1 Processor -- start 2024-07-24T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.475+0000 7f68903ff740 1 -- start start 2024-07-24T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f68903ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f68903ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f68903ff740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f6887fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f6887fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41708/0 (socket says 172.21.15.60:41708) 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f6887fff640 1 -- 172.21.15.60:0/2700565848 learned_addr learned my addr 172.21.15.60:0/2700565848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.476+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68740112b0 con 0x557c7cbcdf80 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f687400eac0 con 0x557c7cbc16d0 2024-07-24T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687400c8f0 con 0x557c7cbc1ff0 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f6887fff640 1 -- 172.21.15.60:0/2700565848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 msgr2=0x557c7cbcbcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f6887fff640 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcbcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f6887fff640 1 -- 172.21.15.60:0/2700565848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 msgr2=0x557c7cbc5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f6887fff640 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.477+0000 7f6887fff640 1 -- 172.21.15.60:0/2700565848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7cbc9410 con 0x557c7cbcdf80 2024-07-24T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.478+0000 7f6887fff640 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f68800a6e80 tx=0x7f6880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc26614906e44ee1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.479+0000 7f686d7fa640 1 -- 172.21.15.60:0/2700565848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800b7070 con 0x557c7cbcdf80 2024-07-24T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.479+0000 7f686d7fa640 1 -- 172.21.15.60:0/2700565848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6880007d80 con 0x557c7cbcdf80 2024-07-24T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.479+0000 7f686d7fa640 1 -- 172.21.15.60:0/2700565848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800a6970 con 0x557c7cbcdf80 2024-07-24T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 msgr2=0x557c7cbe5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f68800a6e80 tx=0x7f6880004a40 comp rx=0 tx=0).stop 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 shutdown_connections 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbcdf80 0x557c7cbe5740 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbcbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 --2- 172.21.15.60:0/2700565848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbc16d0 0x557c7cbc5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 >> 172.21.15.60:0/2700565848 conn(0x557c7cbbf500 msgr2=0x557c7cbccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.480+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 shutdown_connections 2024-07-24T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.481+0000 7f68903ff740 1 -- 172.21.15.60:0/2700565848 wait complete. 2024-07-24T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.481+0000 7f68903ff740 1 -- start start 2024-07-24T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.481+0000 7f68903ff740 1 Processor -- start 2024-07-24T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f68903ff740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbc16d0 0x557c7cbd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f68903ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbd67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f68903ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f68903ff740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f687400cf80 con 0x557c7cbc16d0 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f68903ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f687400e3f0 con 0x557c7cbcdf80 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f6887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35076/0 (socket says 172.21.15.60:35076) 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 -- 172.21.15.60:0/377382642 learned_addr learned my addr 172.21.15.60:0/377382642 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.482+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbd67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687400d650 con 0x557c7cbc1ff0 2024-07-24T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f68877fe640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbc16d0 0x557c7cbd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f68903ff740 10 client.0 _ll_register_callbacks cb 0x7f6880017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 -- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 msgr2=0x557c7cbd67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbd67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 -- 172.21.15.60:0/377382642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbc16d0 msgr2=0x557c7cbd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbc16d0 0x557c7cbd6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 -- 172.21.15.60:0/377382642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7cbd21a0 con 0x557c7cbcdf80 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6886ffd640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbd67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6887fff640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68800a6150 tx=0x7f688000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce5c81b6fd35056 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.483+0000 7f6866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68800b7030 con 0x557c7cbcdf80 2024-07-24T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f68800c4070 con 0x557c7cbcdf80 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880007d80 con 0x557c7cbcdf80 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f68800a6b40 con 0x557c7cbcdf80 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f68903ff740 10 client.28355 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.486+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557c7cbd21a0 con 0x557c7cbcdf80 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68800a6970 con 0x557c7cbcdf80 2024-07-24T22:03:47.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f68800a9750 con 0x557c7cbcdf80 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 10 client.28355 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 10 client.28355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 20 client.28355 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cc486e0 0x557c7cc4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 4 client.28355 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.487+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6880007d80 con 0x557c7cc486e0 2024-07-24T22:03:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f68877fe640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cc486e0 0x557c7cc4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:47.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f6866ffd640 10 client.28355 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:47.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f68877fe640 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cc486e0 0x557c7cc4ab00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7ffd88e5bfc7e2 server_cookie=5bf141d7086f720e in_seq=0 out_seq=0 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f6866ffd640 1 -- 172.21.15.60:0/377382642 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6880007d80 con 0x557c7cc486e0 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f6866ffd640 10 client.28355 handle_command_reply: tid=0 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.488+0000 7f68903ff740 1 client.28355 shutdown 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 msgr2=0x557c7cbd1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f68800a6150 tx=0x7f688000cd70 comp rx=0 tx=0).stop 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 shutdown_connections 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557c7cc486e0 0x557c7cc4ab00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f6866ffd640 0 client.28355 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c7cbcdf80 0x557c7cbd1970 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c7cbc1ff0 0x557c7cbd67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 --2- 172.21.15.60:0/377382642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557c7cbc16d0 0x557c7cbd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 >> 172.21.15.60:0/377382642 conn(0x557c7cbbf500 msgr2=0x557c7cbc9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 shutdown_connections 2024-07-24T22:03:47.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 1 -- 172.21.15.60:0/377382642 wait complete. 2024-07-24T22:03:47.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.489+0000 7f68903ff740 20 client.28355 trim_cache size 0 max 16384 2024-07-24T22:03:47.493 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 msgr2=0x7f68700b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 0x7f68700b6380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f688810fd30 tx=0x7f687c008040 comp rx=0 tx=0).stop 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 msgr2=0x7f688810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f688000b970 tx=0x7f688000be30 comp rx=0 tx=0).stop 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 shutdown_connections 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f68700b3ec0 0x7f68700b6380 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f688810cab0 0x7f688810cf60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6888106880 0x7f6888073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.490+0000 7f688e04e640 1 --2- 172.21.15.60:0/1558987172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6888101d60 0x7f6888073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:47.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.491+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 >> 172.21.15.60:0/1558987172 conn(0x7f68880fd0b0 msgr2=0x7f6888104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:47.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.491+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 shutdown_connections 2024-07-24T22:03:47.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:47.491+0000 7f688e04e640 1 -- 172.21.15.60:0/1558987172 wait complete. 2024-07-24T22:03:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:47 smithi060 ceph-mon[33000]: pgmap v914: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 784 KiB/s wr, 72 op/s 2024-07-24T22:03:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:47 smithi123 ceph-mon[37143]: pgmap v914: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 784 KiB/s wr, 72 op/s 2024-07-24T22:03:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:47 smithi086 ceph-mon[37128]: pgmap v914: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 784 KiB/s wr, 72 op/s 2024-07-24T22:03:49.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:49 smithi060 ceph-mon[33000]: pgmap v915: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-07-24T22:03:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:49 smithi123 ceph-mon[37143]: pgmap v915: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-07-24T22:03:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:49 smithi086 ceph-mon[37128]: pgmap v915: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-07-24T22:03:50.053 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T22:03:50.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T22:03:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.623+0000 7fae7c466640 1 -- 172.21.15.60:0/627372449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 msgr2=0x7fae7410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.623+0000 7fae7c466640 1 --2- 172.21.15.60:0/627372449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410f300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009ca0 tx=0x7fae6c02a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.625+0000 7fae7c466640 1 -- 172.21.15.60:0/627372449 shutdown_connections 2024-07-24T22:03:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 --2- 172.21.15.60:0/627372449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410f300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 --2- 172.21.15.60:0/627372449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 0x7fae741085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 --2- 172.21.15.60:0/627372449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae74102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 -- 172.21.15.60:0/627372449 >> 172.21.15.60:0/627372449 conn(0x7fae740fe1a0 msgr2=0x7fae741005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 -- 172.21.15.60:0/627372449 shutdown_connections 2024-07-24T22:03:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.626+0000 7fae7c466640 1 -- 172.21.15.60:0/627372449 wait complete. 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 Processor -- start 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 -- start start 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 0x7fae7410c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae74076780 con 0x7fae74108180 2024-07-24T22:03:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.627+0000 7fae7c466640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae74076600 con 0x7fae74102770 2024-07-24T22:03:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7c466640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae74076900 con 0x7fae74108b00 2024-07-24T22:03:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35086/0 (socket says 172.21.15.60:35086) 2024-07-24T22:03:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 -- 172.21.15.60:0/4143140458 learned_addr learned my addr 172.21.15.60:0/4143140458 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae79a01640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 0x7fae7410c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7aa03640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 -- 172.21.15.60:0/4143140458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 msgr2=0x7fae7410c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 -- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 msgr2=0x7fae7410c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 0x7fae7410c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.628+0000 7fae7a202640 1 -- 172.21.15.60:0/4143140458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7410cc30 con 0x7fae74102770 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.629+0000 7fae7a202640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fae7000e9f0 tx=0x7fae7000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.629+0000 7fae6b7fe640 1 -- 172.21.15.60:0/4143140458 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7000ccc0 con 0x7fae74102770 2024-07-24T22:03:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.629+0000 7fae6b7fe640 1 -- 172.21.15.60:0/4143140458 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae700108e0 con 0x7fae74102770 2024-07-24T22:03:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.630+0000 7fae6b7fe640 1 -- 172.21.15.60:0/4143140458 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70018880 con 0x7fae74102770 2024-07-24T22:03:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.630+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae741ab700 con 0x7fae74102770 2024-07-24T22:03:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.630+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741abaa0 con 0x7fae74102770 2024-07-24T22:03:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.635+0000 7fae6b7fe640 1 -- 172.21.15.60:0/4143140458 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fae70010430 con 0x7fae74102770 2024-07-24T22:03:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.636+0000 7fae7e817740 1 Processor -- start 2024-07-24T22:03:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.636+0000 7fae7e817740 1 -- start start 2024-07-24T22:03:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.637+0000 7fae6b7fe640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 0x7fae600b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.637+0000 7fae6b7fe640 1 -- 172.21.15.60:0/4143140458 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fae700e43f0 con 0x7fae74102770 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c345a0 0x564f84c36990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f84b9e670 con 0x564f84c345a0 2024-07-24T22:03:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f84b8db70 con 0x564f84c28770 2024-07-24T22:03:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7e817740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f84b91bb0 con 0x564f84c29cd0 2024-07-24T22:03:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51724/0 (socket says 172.21.15.60:51724) 2024-07-24T22:03:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 -- 172.21.15.60:0/1596198719 learned_addr learned my addr 172.21.15.60:0/1596198719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae79a01640 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 -- 172.21.15.60:0/1596198719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 msgr2=0x564f84c28b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c28b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 -- 172.21.15.60:0/1596198719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c345a0 msgr2=0x564f84c36990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c345a0 0x564f84c36990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 -- 172.21.15.60:0/1596198719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f84b25d30 con 0x564f84c29cd0 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.638+0000 7fae7a202640 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae700a6fc0 tx=0x7fae700ab780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be858f956b17d255 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.639+0000 7fae68de4640 1 -- 172.21.15.60:0/1596198719 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700b3ff0 con 0x564f84c29cd0 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.639+0000 7fae68de4640 1 -- 172.21.15.60:0/1596198719 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae700a7530 con 0x564f84c29cd0 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.639+0000 7fae68de4640 1 -- 172.21.15.60:0/1596198719 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700a68b0 con 0x564f84c29cd0 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.640+0000 7fae7e817740 1 -- 172.21.15.60:0/1596198719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 msgr2=0x564f84c28230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.640+0000 7fae7e817740 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae700a6fc0 tx=0x7fae700ab780 comp rx=0 tx=0).stop 2024-07-24T22:03:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7aa03640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 0x7fae600b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7e817740 1 -- 172.21.15.60:0/1596198719 shutdown_connections 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7e817740 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c345a0 0x564f84c36990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7e817740 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7e817740 1 --2- 172.21.15.60:0/1596198719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c29cd0 0x564f84c28230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.641+0000 7fae7e817740 1 -- 172.21.15.60:0/1596198719 >> 172.21.15.60:0/1596198719 conn(0x564f84bf2720 msgr2=0x564f84c26060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.642+0000 7fae7e817740 1 -- 172.21.15.60:0/1596198719 shutdown_connections 2024-07-24T22:03:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.642+0000 7fae7e817740 1 -- 172.21.15.60:0/1596198719 wait complete. 2024-07-24T22:03:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.642+0000 7fae7aa03640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 0x7fae600b6110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fae6c006010 tx=0x7fae6c0093f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.643+0000 7fae7e817740 1 -- start start 2024-07-24T22:03:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.643+0000 7fae7e817740 1 Processor -- start 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c28770 0x564f84c31520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c345a0 0x564f84c31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564f849f17a0 con 0x564f84c29cd0 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f84b2fff0 con 0x564f84c345a0 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f84c37400 con 0x564f84c28770 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae7e817740 10 client.0 _ll_register_callbacks cb 0x564f8489b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae79a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae79a01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41758/0 (socket says 172.21.15.60:41758) 2024-07-24T22:03:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.645+0000 7fae79a01640 1 -- 172.21.15.60:0/2197336358 learned_addr learned my addr 172.21.15.60:0/2197336358 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae7a202640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c28770 0x564f84c31520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 -- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c28770 msgr2=0x564f84c31520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c28770 0x564f84c31520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 -- 172.21.15.60:0/2197336358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c345a0 msgr2=0x564f84c31fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c345a0 0x564f84c31fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 -- 172.21.15.60:0/2197336358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f84b340a0 con 0x564f84c29cd0 2024-07-24T22:03:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.646+0000 7fae79a01640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fae64010a50 tx=0x7fae64010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eff9efc247e0dc server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.647+0000 7fae5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.647+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400fbd0 con 0x564f84c29cd0 2024-07-24T22:03:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.647+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fae6400fd70 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.647+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae700a7530 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.647+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae64005020 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.648+0000 7fae7e817740 10 client.19005 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.648+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564f848fdb90 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.648+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae700a7530 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fae64019410 con 0x564f84c29cd0 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 10 client.19005 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 10 client.19005 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 20 client.19005 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84c44740 0x564f84c46b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 4 client.19005 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f84ba05c0 con 0x564f84c44740 2024-07-24T22:03:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.649+0000 7fae7a202640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84c44740 0x564f84c46b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.650+0000 7fae7a202640 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84c44740 0x564f84c46b70 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8261eb074b94d64e server_cookie=bf567d62912cb913 in_seq=0 out_seq=0 2024-07-24T22:03:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.650+0000 7fae5dffb640 10 client.19005 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.651+0000 7fae5dffb640 1 -- 172.21.15.60:0/2197336358 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fae7008be80 con 0x564f84c44740 2024-07-24T22:03:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.651+0000 7fae5dffb640 10 client.19005 handle_command_reply: tid=0 2024-07-24T22:03:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.651+0000 7fae7e817740 1 client.19005 shutdown 2024-07-24T22:03:50.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 msgr2=0x564f84c31a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fae64010a50 tx=0x7fae64010f10 comp rx=0 tx=0).stop 2024-07-24T22:03:50.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 shutdown_connections 2024-07-24T22:03:50.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84c44740 0x564f84c46b70 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c345a0 0x564f84c31fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c31a60 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 --2- 172.21.15.60:0/2197336358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c28770 0x564f84c31520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 >> 172.21.15.60:0/2197336358 conn(0x564f84bf2720 msgr2=0x564f84c2bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.652+0000 7fae5dffb640 0 client.19005 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:50.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.653+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 shutdown_connections 2024-07-24T22:03:50.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.653+0000 7fae7e817740 1 -- 172.21.15.60:0/2197336358 wait complete. 2024-07-24T22:03:50.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.654+0000 7fae7e817740 20 client.19005 trim_cache size 0 max 16384 2024-07-24T22:03:50.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.686+0000 7fae7e817740 1 Processor -- start 2024-07-24T22:03:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.686+0000 7fae7e817740 1 -- start start 2024-07-24T22:03:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.687+0000 7fae7e817740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c25f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.687+0000 7fae7e817740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7e817740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c25f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c25f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35100/0 (socket says 172.21.15.60:35100) 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7a202640 1 -- 172.21.15.60:0/2698394055 learned_addr learned my addr 172.21.15.60:0/2698394055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7e817740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae741bf660 con 0x564f84c29cd0 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae741bce90 con 0x564f84c28770 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae7410d380 con 0x564f84c345a0 2024-07-24T22:03:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.688+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 -- 172.21.15.60:0/2698394055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 msgr2=0x564f84c25f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c25f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 -- 172.21.15.60:0/2698394055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 msgr2=0x564f84c3fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 -- 172.21.15.60:0/2698394055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f84c21990 con 0x564f84c345a0 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.689+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c40190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009ca0 tx=0x7fae6c03e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc703a19e66c73a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.690+0000 7fae5ffff640 1 -- 172.21.15.60:0/2698394055 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c031920 con 0x564f84c345a0 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.690+0000 7fae5ffff640 1 -- 172.21.15.60:0/2698394055 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae6c036040 con 0x564f84c345a0 2024-07-24T22:03:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.690+0000 7fae5ffff640 1 -- 172.21.15.60:0/2698394055 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c02f610 con 0x564f84c345a0 2024-07-24T22:03:50.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.691+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 msgr2=0x564f84c40190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:50.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.691+0000 7fae7e817740 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c40190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009ca0 tx=0x7fae6c03e000 comp rx=0 tx=0).stop 2024-07-24T22:03:50.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.691+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 shutdown_connections 2024-07-24T22:03:50.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.691+0000 7fae7e817740 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c40190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.691+0000 7fae7e817740 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.692+0000 7fae7e817740 1 --2- 172.21.15.60:0/2698394055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c25f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.692+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 >> 172.21.15.60:0/2698394055 conn(0x564f84bf2720 msgr2=0x564f84c3f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:50.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.692+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 shutdown_connections 2024-07-24T22:03:50.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.692+0000 7fae7e817740 1 -- 172.21.15.60:0/2698394055 wait complete. 2024-07-24T22:03:50.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.693+0000 7fae7e817740 1 -- start start 2024-07-24T22:03:50.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.693+0000 7fae7e817740 1 Processor -- start 2024-07-24T22:03:50.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7a202640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35106/0 (socket says 172.21.15.60:35106) 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae74073330 con 0x564f84c29cd0 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae740747a0 con 0x564f84c28770 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae74073a00 con 0x564f84c345a0 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7e817740 10 client.0 _ll_register_callbacks cb 0x7fae6400a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7aa03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7aa03640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51756/0 (socket says 172.21.15.60:51756) 2024-07-24T22:03:50.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.694+0000 7fae7aa03640 1 -- 172.21.15.60:0/2299141621 learned_addr learned my addr 172.21.15.60:0/2299141621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:50.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae79a01640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 -- 172.21.15.60:0/2299141621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 msgr2=0x564f84c3c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 -- 172.21.15.60:0/2299141621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 msgr2=0x564f84c3cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 -- 172.21.15.60:0/2299141621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f84c39bd0 con 0x564f84c345a0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7a202640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae7aa03640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004760 tx=0x7fae6c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba582530f89792f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.695+0000 7fae5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c060060 con 0x564f84c345a0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fae6c003b80 con 0x564f84c345a0 2024-07-24T22:03:50.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae6c036040 con 0x564f84c345a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae6c031920 con 0x564f84c345a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 10 client.28246 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564f84cafbc0 con 0x564f84c345a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae6c036040 con 0x564f84c345a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fae6c0305c0 con 0x564f84c345a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 10 client.28246 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 10 client.28246 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 20 client.28246 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.696+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84cb02a0 0x564f84cb26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.697+0000 7fae7e817740 4 client.28246 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}] 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.697+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}) -- 0x564f84b9c060 con 0x564f84cb02a0 2024-07-24T22:03:50.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.697+0000 7fae7a202640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84cb02a0 0x564f84cb26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.697+0000 7fae5dffb640 10 client.28246 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:50.697+0000 7fae7a202640 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84cb02a0 0x564f84cb26c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371f80fd0e1e82a server_cookie=61c5ee620c34badd in_seq=0 out_seq=0 2024-07-24T22:03:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.180+0000 7fafd12e9640 1 -- 172.21.15.60:0/309771362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc101590 msgr2=0x7fafcc101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.180+0000 7fafd12e9640 1 --2- 172.21.15.60:0/309771362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc101590 0x7fafcc101970 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fafb400d910 tx=0x7fafb402e5c0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 -- 172.21.15.60:0/309771362 shutdown_connections 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 --2- 172.21.15.60:0/309771362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc107910 0x7fafcc10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 --2- 172.21.15.60:0/309771362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc106f90 0x7fafcc1073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 --2- 172.21.15.60:0/309771362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc101590 0x7fafcc101970 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 -- 172.21.15.60:0/309771362 >> 172.21.15.60:0/309771362 conn(0x7fafcc0fd010 msgr2=0x7fafcc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 -- 172.21.15.60:0/309771362 shutdown_connections 2024-07-24T22:03:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.181+0000 7fafd12e9640 1 -- 172.21.15.60:0/309771362 wait complete. 2024-07-24T22:03:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.183+0000 7fafd12e9640 1 Processor -- start 2024-07-24T22:03:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.183+0000 7fafd12e9640 1 -- start start 2024-07-24T22:03:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.183+0000 7fafd12e9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafd12e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 0x7fafcc1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafcaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafcaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51778/0 (socket says 172.21.15.60:51778) 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafcaffd640 1 -- 172.21.15.60:0/75589175 learned_addr learned my addr 172.21.15.60:0/75589175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 0x7fafcc19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1105b0 con 0x7fafcc106f90 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc110430 con 0x7fafcc107910 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafc27fc640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 0x7fafcc1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.184+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 0x7fafcc19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.185+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc110730 con 0x7fafcc101590 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcaffd640 1 -- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 msgr2=0x7fafcc19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcaffd640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 0x7fafcc19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcaffd640 1 -- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 msgr2=0x7fafcc1970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcaffd640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 0x7fafcc1970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcaffd640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc19cc30 con 0x7fafcc101590 2024-07-24T22:03:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafc27fc640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 0x7fafcc1970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.186+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 0x7fafcc19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.187+0000 7fafcaffd640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009230 tx=0x7fafb4005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.187+0000 7fafc8ff9640 1 -- 172.21.15.60:0/75589175 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4044070 con 0x7fafcc101590 2024-07-24T22:03:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.188+0000 7fafc8ff9640 1 -- 172.21.15.60:0/75589175 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb4004ad0 con 0x7fafcc101590 2024-07-24T22:03:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.188+0000 7fafc8ff9640 1 -- 172.21.15.60:0/75589175 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003cb0 con 0x7fafcc101590 2024-07-24T22:03:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.188+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc19cec0 con 0x7fafcc101590 2024-07-24T22:03:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.189+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a35e0 con 0x7fafcc101590 2024-07-24T22:03:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.192+0000 7fafd4040740 1 Processor -- start 2024-07-24T22:03:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.192+0000 7fafd4040740 1 -- start start 2024-07-24T22:03:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fdd81e0 0x55688fde3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde4170 0x55688fde6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55688fd4d280 con 0x55688fdd81e0 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55688fd3c780 con 0x55688fde4170 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafd4040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55688fd407c0 con 0x55688fdd78c0 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51784/0 (socket says 172.21.15.60:51784) 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 -- 172.21.15.60:0/1351361149 learned_addr learned my addr 172.21.15.60:0/1351361149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafc27fc640 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde4170 0x55688fde6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 -- 172.21.15.60:0/1351361149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde4170 msgr2=0x55688fde6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde4170 0x55688fde6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 -- 172.21.15.60:0/1351361149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fdd81e0 msgr2=0x55688fde3c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fdd81e0 0x55688fde3c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.193+0000 7fafcb7fe640 1 -- 172.21.15.60:0/1351361149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688faa0790 con 0x55688fdd78c0 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.194+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafc400a940 tx=0x7fafc400ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a892b339ceb9e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.194+0000 7fafc17fa640 1 -- 172.21.15.60:0/1351361149 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4015070 con 0x55688fdd78c0 2024-07-24T22:03:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.194+0000 7fafc17fa640 1 -- 172.21.15.60:0/1351361149 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc400cc00 con 0x55688fdd78c0 2024-07-24T22:03:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.194+0000 7fafc17fa640 1 -- 172.21.15.60:0/1351361149 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40105b0 con 0x55688fdd78c0 2024-07-24T22:03:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 -- 172.21.15.60:0/1351361149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 msgr2=0x55688fdd7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafc400a940 tx=0x7fafc400ae00 comp rx=0 tx=0).stop 2024-07-24T22:03:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 -- 172.21.15.60:0/1351361149 shutdown_connections 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde4170 0x55688fde6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fdd81e0 0x55688fde3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 --2- 172.21.15.60:0/1351361149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fdd7ca0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 -- 172.21.15.60:0/1351361149 >> 172.21.15.60:0/1351361149 conn(0x55688fdd56f0 msgr2=0x55688fdd6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafd4040740 1 -- 172.21.15.60:0/1351361149 shutdown_connections 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.195+0000 7fafc8ff9640 1 -- 172.21.15.60:0/75589175 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fafb4047070 con 0x7fafcc101590 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.196+0000 7fafd4040740 1 -- 172.21.15.60:0/1351361149 wait complete. 2024-07-24T22:03:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.196+0000 7fafd4040740 1 -- start start 2024-07-24T22:03:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.196+0000 7fafd4040740 1 Processor -- start 2024-07-24T22:03:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fde0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde12a0 0x55688fde1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55688fc76ab0 con 0x55688fde4170 2024-07-24T22:03:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55688fcc49e0 con 0x55688fde12a0 2024-07-24T22:03:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55688fde6ea0 con 0x55688fdd78c0 2024-07-24T22:03:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafd4040740 10 client.0 _ll_register_callbacks cb 0x55688fa4ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51794/0 (socket says 172.21.15.60:51794) 2024-07-24T22:03:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 -- 172.21.15.60:0/172521892 learned_addr learned my addr 172.21.15.60:0/172521892 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafc27fc640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde12a0 0x55688fde1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 -- 172.21.15.60:0/172521892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde12a0 msgr2=0x55688fde1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde12a0 0x55688fde1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 -- 172.21.15.60:0/172521892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 msgr2=0x55688fde0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fde0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.197+0000 7fafcb7fe640 1 -- 172.21.15.60:0/172521892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688fb19620 con 0x55688fdd78c0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafc400a120 tx=0x7fafc40116a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a24f81479db7de server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4029060 con 0x55688fdd78c0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fafc401a650 con 0x55688fdd78c0 2024-07-24T22:03:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55688fcebce0 con 0x55688fdd78c0 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fafc4015040 con 0x55688fdd78c0 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafd4040740 10 client.28255 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55688fe0c290 con 0x55688fdd78c0 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.198+0000 7fafc8ff9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 0x7fafb80b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55688fcebce0 con 0x55688fdd78c0 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fafc40194d0 con 0x55688fdd78c0 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafc8ff9640 1 -- 172.21.15.60:0/75589175 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fafb4108980 con 0x7fafcc101590 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 10 client.28255 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 10 client.28255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 20 client.28255 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe0d5e0 0x55688fe0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafcaffd640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 0x7fafb80b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 4 client.28255 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55688fd4f1d0 con 0x55688fe0d5e0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.199+0000 7fafc27fc640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe0d5e0 0x55688fe0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.200+0000 7fafcaffd640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 0x7fafb80b6320 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55688fddaa40 tx=0x7fafb40053b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.200+0000 7fafb27fc640 10 client.28255 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.200+0000 7fafc27fc640 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe0d5e0 0x55688fe0f9d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0468075883a2ca server_cookie=647f02050ac0d3c6 in_seq=0 out_seq=0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafb27fc640 1 -- 172.21.15.60:0/172521892 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55688fd4f1d0 con 0x55688fe0d5e0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafb27fc640 10 client.28255 handle_command_reply: tid=0 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 client.28255 shutdown 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 msgr2=0x55688fde3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafc400a120 tx=0x7fafc40116a0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 shutdown_connections 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe0d5e0 0x55688fe0f9d0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fde12a0 0x55688fde1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fde0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 --2- 172.21.15.60:0/172521892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fdd78c0 0x55688fde3920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 >> 172.21.15.60:0/172521892 conn(0x55688fdd56f0 msgr2=0x55688fddb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafb27fc640 0 client.28255 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.202+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 shutdown_connections 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.203+0000 7fafd4040740 1 -- 172.21.15.60:0/172521892 wait complete. 2024-07-24T22:03:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.203+0000 7fafd4040740 20 client.28255 trim_cache size 0 max 16384 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.228+0000 7fafd4040740 1 Processor -- start 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.228+0000 7fafd4040740 1 -- start start 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fde35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fe15030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1a62e0 con 0x55688fde4170 2024-07-24T22:03:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35140/0 (socket says 172.21.15.60:35140) 2024-07-24T22:03:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafcb7fe640 1 -- 172.21.15.60:0/2602497253 learned_addr learned my addr 172.21.15.60:0/2602497253 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1a3af0 con 0x55688fdd78c0 2024-07-24T22:03:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc072e50 con 0x55688fde12a0 2024-07-24T22:03:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.229+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fe15030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 -- 172.21.15.60:0/2602497253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 msgr2=0x55688fde35d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fde35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 -- 172.21.15.60:0/2602497253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 msgr2=0x55688fe15030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fe15030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 -- 172.21.15.60:0/2602497253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688fddfca0 con 0x55688fdd78c0 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.230+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006e50 tx=0x7fafc400afb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22e0b2a31090fcf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.231+0000 7fafc0ff9640 1 -- 172.21.15.60:0/2602497253 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4029050 con 0x55688fdd78c0 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.231+0000 7fafc0ff9640 1 -- 172.21.15.60:0/2602497253 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc4011bc0 con 0x55688fdd78c0 2024-07-24T22:03:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.231+0000 7fafc0ff9640 1 -- 172.21.15.60:0/2602497253 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40196b0 con 0x55688fdd78c0 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 msgr2=0x55688fdd6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006e50 tx=0x7fafc400afb0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 shutdown_connections 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fe15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fde35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 --2- 172.21.15.60:0/2602497253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdd6f10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.232+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 >> 172.21.15.60:0/2602497253 conn(0x55688fdd56f0 msgr2=0x55688fdd6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.233+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 shutdown_connections 2024-07-24T22:03:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.233+0000 7fafd4040740 1 -- 172.21.15.60:0/2602497253 wait complete. 2024-07-24T22:03:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.233+0000 7fafd4040740 1 -- start start 2024-07-24T22:03:51.236 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.233+0000 7fafd4040740 1 Processor -- start 2024-07-24T22:03:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fdf0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc0734a0 con 0x55688fde4170 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafc27fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafc27fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51812/0 (socket says 172.21.15.60:51812) 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafc27fc640 1 -- 172.21.15.60:0/2213355728 learned_addr learned my addr 172.21.15.60:0/2213355728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc074910 con 0x55688fdd78c0 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc073b70 con 0x55688fde12a0 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.234+0000 7fafd4040740 10 client.0 _ll_register_callbacks cb 0x7fafc400a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 -- 172.21.15.60:0/2213355728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 msgr2=0x55688fdef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 -- 172.21.15.60:0/2213355728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 msgr2=0x55688fdf0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fdf0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688fde1a90 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafc27fc640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fafbc007bd0 tx=0x7fafbc0129f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c94b4855b5de4a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0108d0 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.235+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fafbc010a70 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc4011bc0 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fafbc010c70 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 10 client.28258 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55688fe772a0 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc4011bc0 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fafbc02c4d0 con 0x55688fde12a0 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 10 client.28258 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 10 client.28258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 20 client.28258 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe77ae0 0x55688fe79f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 4 client.28258 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.236+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55688fde3cb0 con 0x55688fe77ae0 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.237+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe77ae0 0x55688fe79f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.237+0000 7fafcb7fe640 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe77ae0 0x55688fe79f00 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1d3b358bc1447e server_cookie=5451b928b160ddf1 in_seq=0 out_seq=0 2024-07-24T22:03:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.237+0000 7fafb27fc640 10 client.28258 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.248+0000 7fafb27fc640 1 -- 172.21.15.60:0/2213355728 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55688fde3cb0 con 0x55688fe77ae0 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.248+0000 7fafb27fc640 10 client.28258 handle_command_reply: tid=0 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.248+0000 7fafd4040740 1 client.28258 shutdown 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 msgr2=0x55688fdefec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fafbc007bd0 tx=0x7fafbc0129f0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 shutdown_connections 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55688fe77ae0 0x55688fe79f00 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55688fde4170 0x55688fdf0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafb27fc640 0 client.28258 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688fde12a0 0x55688fdefec0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 --2- 172.21.15.60:0/2213355728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55688fdd78c0 0x55688fdef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 >> 172.21.15.60:0/2213355728 conn(0x55688fdd56f0 msgr2=0x55688fdd6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 shutdown_connections 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 1 -- 172.21.15.60:0/2213355728 wait complete. 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.249+0000 7fafd4040740 20 client.28258 trim_cache size 0 max 16384 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:03:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:03:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:03:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:03:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 msgr2=0x7fafb80b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 0x7fafb80b6320 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55688fddaa40 tx=0x7fafb40053b0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 msgr2=0x7fafcc196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009230 tx=0x7fafb4005200 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 shutdown_connections 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fafb80b3e60 0x7fafb80b6320 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafcc107910 0x7fafcc19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafcc106f90 0x7fafcc1970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 --2- 172.21.15.60:0/75589175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafcc101590 0x7fafcc196b90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 >> 172.21.15.60:0/75589175 conn(0x7fafcc0fd010 msgr2=0x7fafcc0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 shutdown_connections 2024-07-24T22:03:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:03:51.250+0000 7fafd12e9640 1 -- 172.21.15.60:0/75589175 wait complete. 2024-07-24T22:03:51.722 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:51 smithi060 ceph-mon[33000]: pgmap v916: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-07-24T22:03:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:51 smithi123 ceph-mon[37143]: pgmap v916: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-07-24T22:03:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:51 smithi086 ceph-mon[37128]: pgmap v916: 97 pgs: 27 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 60 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-07-24T22:03:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae5dffb640 1 -- 172.21.15.60:0/2299141621 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564f84b9c060 con 0x564f84cb02a0 2024-07-24T22:03:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae5dffb640 10 client.28246 handle_command_reply: tid=0 2024-07-24T22:03:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 client.28246 shutdown 2024-07-24T22:03:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 msgr2=0x564f84c39690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004760 tx=0x7fae6c004790 comp rx=0 tx=0).stop 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 shutdown_connections 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x564f84cb02a0 0x564f84cb26c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f84c345a0 0x564f84c39690 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f84c29cd0 0x564f84c3cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae5dffb640 0 client.28246 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 --2- 172.21.15.60:0/2299141621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f84c28770 0x564f84c3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 >> 172.21.15.60:0/2299141621 conn(0x564f84bf2720 msgr2=0x564f84c3fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 shutdown_connections 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 1 -- 172.21.15.60:0/2299141621 wait complete. 2024-07-24T22:03:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.090+0000 7fae7e817740 20 client.28246 trim_cache size 0 max 16384 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 46, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 54, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "874f7a0d": { 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 54, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:03:53.095 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.3, 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "QUIESCED", 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 msgr2=0x7fae600b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 0x7fae600b6110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fae6c006010 tx=0x7fae6c0093f0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 msgr2=0x7fae7410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fae7000e9f0 tx=0x7fae7000edf0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 shutdown_connections 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae600b3c50 0x7fae600b6110 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae74108b00 0x7fae7410c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae74108180 0x7fae7410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 --2- 172.21.15.60:0/4143140458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae74102770 0x7fae7410ee30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 >> 172.21.15.60:0/4143140458 conn(0x7fae740fe1a0 msgr2=0x7fae74101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 shutdown_connections 2024-07-24T22:03:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.091+0000 7fae7c466640 1 -- 172.21.15.60:0/4143140458 wait complete. 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 -- 172.21.15.60:0/953116109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac1081c0 msgr2=0x7fcbac108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 --2- 172.21.15.60:0/953116109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac1081c0 0x7fcbac108600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ca0 tx=0x7fcba002a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 -- 172.21.15.60:0/953116109 shutdown_connections 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 --2- 172.21.15.60:0/953116109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac108b40 0x7fcbac10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 --2- 172.21.15.60:0/953116109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac1081c0 0x7fcbac108600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 --2- 172.21.15.60:0/953116109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1027b0 0x7fcbac102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 -- 172.21.15.60:0/953116109 >> 172.21.15.60:0/953116109 conn(0x7fcbac0fe200 msgr2=0x7fcbac100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 -- 172.21.15.60:0/953116109 shutdown_connections 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.245+0000 7fcbb438a640 1 -- 172.21.15.60:0/953116109 wait complete. 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 Processor -- start 2024-07-24T22:03:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 -- start start 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1081c0 0x7fcbac0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac108b40 0x7fcbac075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac112750 con 0x7fcbac1027b0 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1125d0 con 0x7fcbac108b40 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb438a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1128d0 con 0x7fcbac1081c0 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb2126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb2126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41840/0 (socket says 172.21.15.60:41840) 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.246+0000 7fcbb2126640 1 -- 172.21.15.60:0/4025707401 learned_addr learned my addr 172.21.15.60:0/4025707401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb1925640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1081c0 0x7fcbac0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2927640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac108b40 0x7fcbac075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 -- 172.21.15.60:0/4025707401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1081c0 msgr2=0x7fcbac0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1081c0 0x7fcbac0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 -- 172.21.15.60:0/4025707401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac108b40 msgr2=0x7fcbac075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac108b40 0x7fcbac075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 -- 172.21.15.60:0/4025707401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac1a7060 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb2126640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d950 tx=0x7fcb9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbab7fe640 1 -- 172.21.15.60:0/4025707401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c017070 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac1a7350 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbab7fe640 1 -- 172.21.15.60:0/4025707401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb9c009c30 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbab7fe640 1 -- 172.21.15.60:0/4025707401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00f620 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.247+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1a78e0 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.250+0000 7fcbab7fe640 1 -- 172.21.15.60:0/4025707401 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fcb9c010460 con 0x7fcbac1027b0 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.251+0000 7fcbb673b740 1 Processor -- start 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.251+0000 7fcbb673b740 1 -- start start 2024-07-24T22:03:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6ce1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6ce3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbab7fe640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 0x7fcb940b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb673b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634a6c4c9f0 con 0x5634a6ce17d0 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb2927640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb673b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634a6c3bef0 con 0x5634a6cd5840 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbab7fe640 1 -- 172.21.15.60:0/4025707401 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fcb9c0e3bf0 con 0x7fcbac1027b0 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb2927640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:51834/0 (socket says 172.21.15.60:51834) 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.252+0000 7fcbb2927640 1 -- 172.21.15.60:0/1892910579 learned_addr learned my addr 172.21.15.60:0/1892910579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.253+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634a6c3ff30 con 0x5634a6cd4f20 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.253+0000 7fcbb2927640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 0x7fcb940b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.253+0000 7fcbb1925640 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6ce3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 -- 172.21.15.60:0/1892910579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 msgr2=0x5634a6ce1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6ce1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 -- 172.21.15.60:0/1892910579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 msgr2=0x5634a6ce3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6ce3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 -- 172.21.15.60:0/1892910579 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a699c930 con 0x5634a6cd4f20 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcba40117b0 tx=0x7fcba401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f983f13ea49c96e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.254+0000 7fcbb2927640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 0x7fcb940b6200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcba40139b0 tx=0x7fcba40098b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.255+0000 7fcba8de4640 1 -- 172.21.15.60:0/1892910579 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba402c070 con 0x5634a6cd4f20 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.255+0000 7fcba8de4640 1 -- 172.21.15.60:0/1892910579 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba400ec60 con 0x5634a6cd4f20 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.255+0000 7fcba8de4640 1 -- 172.21.15.60:0/1892910579 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4027a20 con 0x5634a6cd4f20 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 msgr2=0x5634a6cd5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcba40117b0 tx=0x7fcba401d000 comp rx=0 tx=0).stop 2024-07-24T22:03:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 shutdown_connections 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6ce3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6ce1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1892910579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cd5300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 >> 172.21.15.60:0/1892910579 conn(0x5634a6cd2d50 msgr2=0x5634a6cd3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 shutdown_connections 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.256+0000 7fcbb673b740 1 -- 172.21.15.60:0/1892910579 wait complete. 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.257+0000 7fcbb673b740 1 -- start start 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.257+0000 7fcbb673b740 1 Processor -- start 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.257+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.257+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.257+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6ce17d0 0x5634a6cdf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41860/0 (socket says 172.21.15.60:41860) 2024-07-24T22:03:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 -- 172.21.15.60:0/242083778 learned_addr learned my addr 172.21.15.60:0/242083778 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5634a6a9f890 con 0x5634a6cd5840 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634a6bde390 con 0x5634a6ce17d0 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634a6ce43f0 con 0x5634a6cd4f20 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb673b740 10 client.0 _ll_register_callbacks cb 0x5634a6be8500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb2126640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 -- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 msgr2=0x5634a6cde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cde800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 -- 172.21.15.60:0/242083778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6ce17d0 msgr2=0x5634a6cdf280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6ce17d0 0x5634a6cdf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a6bd8d60 con 0x5634a6cd5840 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.258+0000 7fcbb1925640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ca0 tx=0x7fcba0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adcd595e5233538 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.259+0000 7fcb91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.263+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0031720 con 0x5634a6cd5840 2024-07-24T22:03:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.263+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fcba00318c0 con 0x5634a6cd5840 2024-07-24T22:03:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.263+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba400ec60 con 0x5634a6cd5840 2024-07-24T22:03:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.264+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fcba0004b40 con 0x5634a6cd5840 2024-07-24T22:03:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.265+0000 7fcbb673b740 10 client.19026 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.265+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5634a6d09ba0 con 0x5634a6cd5840 2024-07-24T22:03:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba400ec60 con 0x5634a6cd5840 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fcba0004460 con 0x5634a6cd5840 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 10 client.19026 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 10 client.19026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 20 client.19026 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d0b100 0x5634a6d0d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 4 client.19026 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.267+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a6c4e940 con 0x5634a6d0b100 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.268+0000 7fcbb2126640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d0b100 0x5634a6d0d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.268+0000 7fcbb2126640 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d0b100 0x5634a6d0d520 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68994b28adf0b1d server_cookie=aad8fe9e6d3767 in_seq=0 out_seq=0 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.268+0000 7fcb91ffb640 10 client.19026 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcb91ffb640 1 -- 172.21.15.60:0/242083778 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634a6c4e940 con 0x5634a6d0b100 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcb91ffb640 10 client.19026 handle_command_reply: tid=0 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 client.19026 shutdown 2024-07-24T22:03:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 msgr2=0x5634a6cded40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ca0 tx=0x7fcba0009540 comp rx=0 tx=0).stop 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 shutdown_connections 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d0b100 0x5634a6d0d520 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6ce17d0 0x5634a6cdf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd5840 0x5634a6cded40 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 --2- 172.21.15.60:0/242083778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 >> 172.21.15.60:0/242083778 conn(0x5634a6cd2d50 msgr2=0x5634a6cd88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.270+0000 7fcb91ffb640 0 client.19026 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.272+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 shutdown_connections 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.273+0000 7fcbb673b740 1 -- 172.21.15.60:0/242083778 wait complete. 2024-07-24T22:03:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.273+0000 7fcbb673b740 20 client.19026 trim_cache size 0 max 16384 2024-07-24T22:03:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 Processor -- start 2024-07-24T22:03:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 -- start start 2024-07-24T22:03:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cdeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6d12df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac073520 con 0x5634a6ce17d0 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac070d30 con 0x5634a6cd5840 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb673b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac06eba0 con 0x5634a6cd4f20 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb2927640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb2927640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41876/0 (socket says 172.21.15.60:41876) 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.293+0000 7fcbb2927640 1 -- 172.21.15.60:0/3259510113 learned_addr learned my addr 172.21.15.60:0/3259510113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb1925640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6d12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2126640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cdeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 -- 172.21.15.60:0/3259510113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 msgr2=0x5634a6cdeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cdeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 -- 172.21.15.60:0/3259510113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 msgr2=0x5634a6d12df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6d12df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 -- 172.21.15.60:0/3259510113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a6cd3720 con 0x5634a6ce17d0 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb1925640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6d12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcbb2927640 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005940 tx=0x7fcba4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508de09862f24ecf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcb93fff640 1 -- 172.21.15.60:0/3259510113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba402c070 con 0x5634a6ce17d0 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcb93fff640 1 -- 172.21.15.60:0/3259510113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba4009b80 con 0x5634a6ce17d0 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.294+0000 7fcb93fff640 1 -- 172.21.15.60:0/3259510113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4004680 con 0x5634a6ce17d0 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 -- 172.21.15.60:0/3259510113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 msgr2=0x5634a6d13330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005940 tx=0x7fcba4011ef0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 -- 172.21.15.60:0/3259510113 shutdown_connections 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6ce17d0 0x5634a6d13330 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6d12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 --2- 172.21.15.60:0/3259510113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6cd4f20 0x5634a6cdeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 -- 172.21.15.60:0/3259510113 >> 172.21.15.60:0/3259510113 conn(0x5634a6cd2d50 msgr2=0x5634a6cde7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 -- 172.21.15.60:0/3259510113 shutdown_connections 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.295+0000 7fcbb673b740 1 -- 172.21.15.60:0/3259510113 wait complete. 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 -- start start 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 Processor -- start 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6cf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6ce17d0 0x5634a6ceeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac06f1f0 con 0x5634a6cd4f20 2024-07-24T22:03:53.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac070660 con 0x5634a6cd5840 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac06f8c0 con 0x5634a6ce17d0 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb2126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb673b740 10 client.0 _ll_register_callbacks cb 0x7fcba002f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb2126640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41888/0 (socket says 172.21.15.60:41888) 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb2126640 1 -- 172.21.15.60:0/1856538690 learned_addr learned my addr 172.21.15.60:0/1856538690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb1925640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6cf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.296+0000 7fcbb2927640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6ce17d0 0x5634a6ceeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 -- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6ce17d0 msgr2=0x5634a6ceeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6ce17d0 0x5634a6ceeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 -- 172.21.15.60:0/1856538690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 msgr2=0x5634a6cf2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6cf2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 -- 172.21.15.60:0/1856538690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a6cf2b90 con 0x5634a6cd4f20 2024-07-24T22:03:53.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb2126640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0e1390 tx=0x7fcb9c0920b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0ae6a19b2bed32 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:53.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcb91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:53.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0c3020 con 0x5634a6cd4f20 2024-07-24T22:03:53.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7fcb9c0cd0b0 con 0x5634a6cd4f20 2024-07-24T22:03:53.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba4009b80 con 0x5634a6cd4f20 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c020040 con 0x5634a6cd4f20 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb673b740 10 client.19032 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5634a6d76450 con 0x5634a6cd4f20 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.297+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba4009b80 con 0x5634a6cd4f20 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fcb9c0a61c0 con 0x5634a6cd4f20 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 10 client.19032 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 10 client.19032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 20 client.19032 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d76c00 0x5634a6d79020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 4 client.19032 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5634a6c4a3e0 con 0x5634a6d76c00 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.298+0000 7fcbb1925640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d76c00 0x5634a6d79020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb1925640 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d76c00 0x5634a6d79020 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1515ae25b0622044 server_cookie=d9675c083bfc6373 in_seq=0 out_seq=0 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcb91ffb640 10 client.19032 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcb91ffb640 1 -- 172.21.15.60:0/1856538690 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5634a6c4a3e0 con 0x5634a6d76c00 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcb91ffb640 10 client.19032 handle_command_reply: tid=0 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 client.19032 shutdown 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 msgr2=0x5634a6cf1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0e1390 tx=0x7fcb9c0920b0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 shutdown_connections 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5634a6d76c00 0x5634a6d79020 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634a6ce17d0 0x5634a6ceeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a6cd5840 0x5634a6cf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 --2- 172.21.15.60:0/1856538690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634a6cd4f20 0x5634a6cf1f10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 >> 172.21.15.60:0/1856538690 conn(0x5634a6cd2d50 msgr2=0x5634a6cf55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcb91ffb640 0 client.19032 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 shutdown_connections 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.299+0000 7fcbb673b740 1 -- 172.21.15.60:0/1856538690 wait complete. 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb673b740 20 client.19032 trim_cache size 0 max 16384 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T22:03:53.307 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "c5f27398-b632-47d0-b441-da220c04ff39", 2024-07-24T22:03:53.308 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T22:03:53.308 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 msgr2=0x7fcb940b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 0x7fcb940b6200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcba40139b0 tx=0x7fcba40098b0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 msgr2=0x7fcbac074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d950 tx=0x7fcb9c00de10 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 shutdown_connections 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fcb940b3d40 0x7fcb940b6200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac108b40 0x7fcbac075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbac1081c0 0x7fcbac0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 --2- 172.21.15.60:0/4025707401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbac1027b0 0x7fcbac074e90 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 >> 172.21.15.60:0/4025707401 conn(0x7fcbac0fe200 msgr2=0x7fcbac1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.300+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 shutdown_connections 2024-07-24T22:03:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:53.301+0000 7fcbb438a640 1 -- 172.21.15.60:0/4025707401 wait complete. 2024-07-24T22:03:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:53 smithi123 ceph-mon[37143]: pgmap v917: 97 pgs: 17 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 73 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-07-24T22:03:54.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:53 smithi086 ceph-mon[37128]: pgmap v917: 97 pgs: 17 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 73 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-07-24T22:03:54.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:53 smithi060 ceph-mon[33000]: pgmap v917: 97 pgs: 17 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 73 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-07-24T22:03:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:54 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T22:03:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:03:54.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:54 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:03:54.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:54 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:54 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T22:03:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:03:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:54 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:03:54.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:54 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:55.035 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '874f7a0d', quiesce took 15.0 seconds. Will release after: 12.3 2024-07-24T22:03:55.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:54 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T22:03:55.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:55.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:03:55.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:54 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:03:55.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:54 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:55.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:03:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:55 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:55.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:55 smithi060 ceph-mon[33000]: pgmap v918: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-07-24T22:03:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:03:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:55 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:55 smithi123 ceph-mon[37143]: pgmap v918: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-07-24T22:03:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:55 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:03:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:55 smithi086 ceph-mon[37128]: pgmap v918: 97 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 88 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-07-24T22:03:57.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 -- 172.21.15.60:0/647625646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 msgr2=0x7f50e4111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 --2- 172.21.15.60:0/647625646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e4111170 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009ca0 tx=0x7f50dc02a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 -- 172.21.15.60:0/647625646 shutdown_connections 2024-07-24T22:03:57.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 --2- 172.21.15.60:0/647625646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e4111170 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 --2- 172.21.15.60:0/647625646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e40757f0 0x7f50e410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 --2- 172.21.15.60:0/647625646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e4074ed0 0x7f50e40752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 -- 172.21.15.60:0/647625646 >> 172.21.15.60:0/647625646 conn(0x7f50e40fe200 msgr2=0x7f50e4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 -- 172.21.15.60:0/647625646 shutdown_connections 2024-07-24T22:03:57.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.489+0000 7f50e91ea640 1 -- 172.21.15.60:0/647625646 wait complete. 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 Processor -- start 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 -- start start 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e4074ed0 0x7f50e419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e40757f0 0x7f50e419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41149c0 con 0x7f50e410ed80 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50e4114840 con 0x7f50e40757f0 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e91ea640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50e4114b40 con 0x7f50e4074ed0 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54088/0 (socket says 172.21.15.60:54088) 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.490+0000 7f50e359e640 1 -- 172.21.15.60:0/2902417158 learned_addr learned my addr 172.21.15.60:0/2902417158 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e259c640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e40757f0 0x7f50e419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e4074ed0 0x7f50e419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 -- 172.21.15.60:0/2902417158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e4074ed0 msgr2=0x7f50e419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e4074ed0 0x7f50e419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 -- 172.21.15.60:0/2902417158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e40757f0 msgr2=0x7f50e419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e40757f0 0x7f50e419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 -- 172.21.15.60:0/2902417158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e41a0fa0 con 0x7f50e410ed80 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e359e640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0319f0 tx=0x7f50dc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50cbfff640 1 -- 172.21.15.60:0/2902417158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003bf0 con 0x7f50e410ed80 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50cbfff640 1 -- 172.21.15.60:0/2902417158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50dc003d90 con 0x7f50e410ed80 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e41a1230 con 0x7f50e410ed80 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50cbfff640 1 -- 172.21.15.60:0/2902417158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc02fc00 con 0x7f50e410ed80 2024-07-24T22:03:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.491+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e41a79d0 con 0x7f50e410ed80 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50cbfff640 1 -- 172.21.15.60:0/2902417158 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f50dc0305c0 con 0x7f50e410ed80 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 Processor -- start 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 -- start start 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd4f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5635bd45e960 con 0x5635bd4e7680 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635bd44de60 con 0x5635bd4e7fa0 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50eb59b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5635bd451ea0 con 0x5635bd4f3f30 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54100/0 (socket says 172.21.15.60:54100) 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50e259c640 1 -- 172.21.15.60:0/899099452 learned_addr learned my addr 172.21.15.60:0/899099452 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.494+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd4f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 -- 172.21.15.60:0/899099452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 msgr2=0x5635bd4f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd4f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 -- 172.21.15.60:0/899099452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 msgr2=0x5635bd4f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 -- 172.21.15.60:0/899099452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635bd3d72d0 con 0x5635bd4e7680 2024-07-24T22:03:57.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e259c640 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f50d800b8f0 tx=0x7f50d8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1242ea39856ded6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50cbfff640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 0x7f50cc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50c97fa640 1 -- 172.21.15.60:0/899099452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8003d70 con 0x5635bd4e7680 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 0x7f50cc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50cbfff640 1 -- 172.21.15.60:0/2902417158 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f50dc105630 con 0x7f50e410ed80 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50c97fa640 1 -- 172.21.15.60:0/899099452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d80054d0 con 0x5635bd4e7680 2024-07-24T22:03:57.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.495+0000 7f50c97fa640 1 -- 172.21.15.60:0/899099452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8010460 con 0x5635bd4e7680 2024-07-24T22:03:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- 172.21.15.60:0/899099452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 msgr2=0x5635bd4e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 0x7f50cc0b6200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5635bd4e6390 tx=0x7f50d4008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f50d800b8f0 tx=0x7f50d8004250 comp rx=0 tx=0).stop 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- 172.21.15.60:0/899099452 shutdown_connections 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd4f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 --2- 172.21.15.60:0/899099452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4e7a60 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- 172.21.15.60:0/899099452 >> 172.21.15.60:0/899099452 conn(0x5635bd4e54b0 msgr2=0x5635bd4e6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- 172.21.15.60:0/899099452 shutdown_connections 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- 172.21.15.60:0/899099452 wait complete. 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.496+0000 7f50eb59b740 1 -- start start 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 Processor -- start 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 0x5635bd4f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5635bd2b1700 con 0x5635bd4f3f30 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635bd3f05a0 con 0x5635bd4e7680 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5635bd4f6bb0 con 0x5635bd4e7fa0 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50eb59b740 10 client.0 _ll_register_callbacks cb 0x5635bd410910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54102/0 (socket says 172.21.15.60:54102) 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 -- 172.21.15.60:0/1770816220 learned_addr learned my addr 172.21.15.60:0/1770816220 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 0x5635bd4f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 -- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7fa0 msgr2=0x5635bd4f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 -- 172.21.15.60:0/1770816220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 msgr2=0x5635bd4f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 0x5635bd4f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e359e640 1 -- 172.21.15.60:0/1770816220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635bd3e3630 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.497+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 0x5635bd4f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50e359e640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0c8370 tx=0x7f50dc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f381d2c9d5fb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0cc2f0 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f50dc0c8580 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d80054d0 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f50dc0d08f0 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 10 client.19044 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5635bd1ae650 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d80054d0 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f50dc0d4fa0 con 0x5635bd4f3f30 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 10 client.19044 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:57.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 10 client.19044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 20 client.19044 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd503a50 0x5635bd505e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 4 client.19044 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.498+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635bd4608b0 con 0x5635bd503a50 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.499+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd503a50 0x5635bd505e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.499+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd503a50 0x5635bd505e60 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56aac803f1b93ca7 server_cookie=31b33f8197505636 in_seq=0 out_seq=0 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.499+0000 7f50c6ff5640 10 client.19044 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50c6ff5640 1 -- 172.21.15.60:0/1770816220 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635bd4608b0 con 0x5635bd503a50 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50c6ff5640 10 client.19044 handle_command_reply: tid=0 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 client.19044 shutdown 2024-07-24T22:03:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 msgr2=0x5635bd4f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0c8370 tx=0x7f50dc009680 comp rx=0 tx=0).stop 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 shutdown_connections 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd503a50 0x5635bd505e60 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4f1910 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 --2- 172.21.15.60:0/1770816220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7680 0x5635bd4f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 >> 172.21.15.60:0/1770816220 conn(0x5635bd4e54b0 msgr2=0x5635bd4eb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50c6ff5640 0 client.19044 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 shutdown_connections 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 1 -- 172.21.15.60:0/1770816220 wait complete. 2024-07-24T22:03:57.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.500+0000 7f50eb59b740 20 client.19044 trim_cache size 0 max 16384 2024-07-24T22:03:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.518+0000 7f50eb59b740 1 Processor -- start 2024-07-24T22:03:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.518+0000 7f50eb59b740 1 -- start start 2024-07-24T22:03:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7680 0x5635bd50b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd50bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41b2dd0 con 0x5635bd4f3f30 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41b05e0 con 0x5635bd4e7fa0 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50eb59b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41ae450 con 0x5635bd4e7680 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54104/0 (socket says 172.21.15.60:54104) 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 -- 172.21.15.60:0/357314895 learned_addr learned my addr 172.21.15.60:0/357314895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7680 0x5635bd50b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e259c640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd50bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 -- 172.21.15.60:0/357314895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7680 msgr2=0x5635bd50b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7680 0x5635bd50b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 -- 172.21.15.60:0/357314895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 msgr2=0x5635bd50bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd50bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 -- 172.21.15.60:0/357314895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635bd4dfa00 con 0x5635bd4f3f30 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e259c640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd50bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.519+0000 7f50e359e640 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f50dc003fd0 tx=0x7f50dc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31800e7cd66edd35 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50c8ff9640 1 -- 172.21.15.60:0/357314895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0cc2f0 con 0x5635bd4f3f30 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50c8ff9640 1 -- 172.21.15.60:0/357314895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50dc036020 con 0x5635bd4f3f30 2024-07-24T22:03:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50c8ff9640 1 -- 172.21.15.60:0/357314895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc004610 con 0x5635bd4f3f30 2024-07-24T22:03:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 -- 172.21.15.60:0/357314895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 msgr2=0x5635bd4ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f50dc003fd0 tx=0x7f50dc02a8c0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 -- 172.21.15.60:0/357314895 shutdown_connections 2024-07-24T22:03:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4f3f30 0x5635bd4ee0a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd50bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 --2- 172.21.15.60:0/357314895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4e7680 0x5635bd50b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 -- 172.21.15.60:0/357314895 >> 172.21.15.60:0/357314895 conn(0x5635bd4e54b0 msgr2=0x5635bd4f2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 -- 172.21.15.60:0/357314895 shutdown_connections 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.520+0000 7f50eb59b740 1 -- 172.21.15.60:0/357314895 wait complete. 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 -- start start 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 Processor -- start 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd56d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41aeaa0 con 0x5635bd4e7680 2024-07-24T22:03:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41aff10 con 0x5635bd4e7fa0 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50e41af170 con 0x5635bd4f3f30 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50eb59b740 10 client.0 _ll_register_callbacks cb 0x7f50dc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54112/0 (socket says 172.21.15.60:54112) 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 -- 172.21.15.60:0/2206495723 learned_addr learned my addr 172.21.15.60:0/2206495723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e259c640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e359e640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd56d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 -- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 msgr2=0x5635bd56d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd56d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 -- 172.21.15.60:0/2206495723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 msgr2=0x5635bd4f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e2d9d640 1 -- 172.21.15.60:0/2206495723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635bd56dd00 con 0x5635bd4e7680 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.521+0000 7f50e359e640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd56d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:03:57.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50e2d9d640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f50d4016730 tx=0x7f50d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fcda275ebcc251 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d401c040 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8609+0+0 (secure 0 0 0) 0x7f50d40237f0 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50dc036020 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f50d403a050 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 10 client.19050 fetch_fsmap learned FSMap version 46 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5635bd56efe0 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50dc036020 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f50d4014b60 con 0x5635bd4e7680 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 10 client.19050 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 10 client.19050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 20 client.19050 populate_metadata read hostname 'smithi060' 2024-07-24T22:03:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd56f7f0 0x5635bd571c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 4 client.19050 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.522+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635bd45c350 con 0x5635bd56f7f0 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50e259c640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd56f7f0 0x5635bd571c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50e259c640 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd56f7f0 0x5635bd571c10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38e6851668ef37e server_cookie=cb73c49c0d160a46 in_seq=0 out_seq=0 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50c6ff5640 10 client.19050 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50c6ff5640 1 -- 172.21.15.60:0/2206495723 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5635bd45c350 con 0x5635bd56f7f0 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50c6ff5640 10 client.19050 handle_command_reply: tid=0 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50eb59b740 1 client.19050 shutdown 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 msgr2=0x5635bd4f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.523+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f50d4016730 tx=0x7f50d40046c0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 shutdown_connections 2024-07-24T22:03:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5635bd56f7f0 0x5635bd571c10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5635bd4f3f30 0x5635bd56d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635bd4e7fa0 0x5635bd4f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 --2- 172.21.15.60:0/2206495723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5635bd4e7680 0x5635bd4f8450 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 >> 172.21.15.60:0/2206495723 conn(0x5635bd4e54b0 msgr2=0x5635bd4ea7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50c6ff5640 0 client.19050 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 shutdown_connections 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 1 -- 172.21.15.60:0/2206495723 wait complete. 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50eb59b740 20 client.19050 trim_cache size 0 max 16384 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:03:57.532 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: "c5f27398-b632-47d0-b441-da220c04ff39": { 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "c5f27398-b632-47d0-b441-da220c04ff39", 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:03:57.533 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 msgr2=0x7f50cc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 0x7f50cc0b6200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5635bd4e6390 tx=0x7f50d4008040 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 msgr2=0x7f50e41a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0319f0 tx=0x7f50dc031a20 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 shutdown_connections 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f50cc0b3d40 0x7f50cc0b6200 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e410ed80 0x7f50e41a07d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e40757f0 0x7f50e419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 --2- 172.21.15.60:0/2902417158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50e4074ed0 0x7f50e419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 >> 172.21.15.60:0/2902417158 conn(0x7f50e40fe200 msgr2=0x7f50e4110f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 shutdown_connections 2024-07-24T22:03:57.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:03:57.524+0000 7f50e91ea640 1 -- 172.21.15.60:0/2902417158 wait complete. 2024-07-24T22:03:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:57 smithi060 ceph-mon[33000]: pgmap v919: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 394 op/s 2024-07-24T22:03:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:57 smithi123 ceph-mon[37143]: pgmap v919: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 394 op/s 2024-07-24T22:03:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:57 smithi086 ceph-mon[37128]: pgmap v919: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 394 op/s 2024-07-24T22:03:58.441 INFO:tasks.cephfs.filesystem:scrub status for tag:c5f27398-b632-47d0-b441-da220c04ff39 - {'path': '/', 'tag': 'c5f27398-b632-47d0-b441-da220c04ff39', 'options': 'recursive,force'} 2024-07-24T22:03:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:03:59 smithi123 ceph-mon[37143]: pgmap v920: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 649 op/s 2024-07-24T22:04:00.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:03:59 smithi086 ceph-mon[37128]: pgmap v920: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 649 op/s 2024-07-24T22:04:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:03:59 smithi060 ceph-mon[33000]: pgmap v920: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 649 op/s 2024-07-24T22:04:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:01 smithi123 ceph-mon[37143]: pgmap v921: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 675 op/s 2024-07-24T22:04:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:01 smithi086 ceph-mon[37128]: pgmap v921: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 675 op/s 2024-07-24T22:04:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:01 smithi060 ceph-mon[33000]: pgmap v921: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 675 op/s 2024-07-24T22:04:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:04:02.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:02 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:04:03.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:04:03.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:02 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:04:03.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:04:03.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:02 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:04:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:03 smithi123 ceph-mon[37143]: pgmap v922: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 575 KiB/s wr, 661 op/s 2024-07-24T22:04:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:03 smithi123 ceph-mon[37143]: osdmap e112: 12 total, 12 up, 12 in 2024-07-24T22:04:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:03 smithi086 ceph-mon[37128]: pgmap v922: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 575 KiB/s wr, 661 op/s 2024-07-24T22:04:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:03 smithi086 ceph-mon[37128]: osdmap e112: 12 total, 12 up, 12 in 2024-07-24T22:04:04.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:03 smithi060 ceph-mon[33000]: pgmap v922: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 575 KiB/s wr, 661 op/s 2024-07-24T22:04:04.075 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:03 smithi060 ceph-mon[33000]: osdmap e112: 12 total, 12 up, 12 in 2024-07-24T22:04:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.406+0000 7ff07c6a9640 1 -- 172.21.15.60:0/1842819296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074106f90 msgr2=0x7ff0741073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.406+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/1842819296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074106f90 0x7ff0741073d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff064009860 tx=0x7ff06402a360 comp rx=0 tx=0).stop 2024-07-24T22:04:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.411+0000 7ff07c6a9640 1 -- 172.21.15.60:0/1842819296 shutdown_connections 2024-07-24T22:04:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.411+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/1842819296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074107910 0x7ff07406aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.411+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/1842819296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074106f90 0x7ff0741073d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.411+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/1842819296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff074101590 0x7ff074101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.411+0000 7ff07c6a9640 1 -- 172.21.15.60:0/1842819296 >> 172.21.15.60:0/1842819296 conn(0x7ff0740fd010 msgr2=0x7ff0740ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.412+0000 7ff07c6a9640 1 -- 172.21.15.60:0/1842819296 shutdown_connections 2024-07-24T22:04:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.416+0000 7ff07c6a9640 1 -- 172.21.15.60:0/1842819296 wait complete. 2024-07-24T22:04:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.416+0000 7ff07c6a9640 1 Processor -- start 2024-07-24T22:04:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff07c6a9640 1 -- start start 2024-07-24T22:04:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff07c6a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074101590 0x7ff074110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff07c6a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff07c6a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff074107910 0x7ff074110ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff079c44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff079c44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54124/0 (socket says 172.21.15.60:54124) 2024-07-24T22:04:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.419+0000 7ff079c44640 1 -- 172.21.15.60:0/3510036985 learned_addr learned my addr 172.21.15.60:0/3510036985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.421+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff074114950 con 0x7ff074106f90 2024-07-24T22:04:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.421+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741147d0 con 0x7ff074101590 2024-07-24T22:04:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.421+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff074114ad0 con 0x7ff074107910 2024-07-24T22:04:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.422+0000 7ff079c44640 1 -- 172.21.15.60:0/3510036985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff074107910 msgr2=0x7ff074110ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:04:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.422+0000 7ff079c44640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff074107910 0x7ff074110ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.422+0000 7ff079c44640 1 -- 172.21.15.60:0/3510036985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074101590 msgr2=0x7ff074110440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:04:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.422+0000 7ff079c44640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074101590 0x7ff074110440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.422+0000 7ff079c44640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff074073210 con 0x7ff074106f90 2024-07-24T22:04:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff079c44640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff064006010 tx=0x7ff064004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff06b7fe640 1 -- 172.21.15.60:0/3510036985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064037050 con 0x7ff074106f90 2024-07-24T22:04:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff06b7fe640 1 -- 172.21.15.60:0/3510036985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff064038040 con 0x7ff074106f90 2024-07-24T22:04:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff06b7fe640 1 -- 172.21.15.60:0/3510036985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064003c60 con 0x7ff074106f90 2024-07-24T22:04:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0740734a0 con 0x7ff074106f90 2024-07-24T22:04:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.424+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff074073940 con 0x7ff074106f90 2024-07-24T22:04:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.429+0000 7ff07f403740 1 Processor -- start 2024-07-24T22:04:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.429+0000 7ff07f403740 1 -- start start 2024-07-24T22:04:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.429+0000 7ff07f403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aae8c0 0x55bba1aaeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.429+0000 7ff07f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07f403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1abb170 0x55bba1abd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bba1a24280 con 0x55bba1aaf1e0 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07f403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bba1a13780 con 0x55bba1abb170 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07f403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bba1a177c0 con 0x55bba1aae8c0 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07a445640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07a445640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54140/0 (socket says 172.21.15.60:54140) 2024-07-24T22:04:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.430+0000 7ff07a445640 1 -- 172.21.15.60:0/1908375963 learned_addr learned my addr 172.21.15.60:0/1908375963 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07ac46640 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aae8c0 0x55bba1aaeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 -- 172.21.15.60:0/1908375963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aae8c0 msgr2=0x55bba1aaeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aae8c0 0x55bba1aaeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 -- 172.21.15.60:0/1908375963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1abb170 msgr2=0x55bba1abd560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1abb170 0x55bba1abd560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 -- 172.21.15.60:0/1908375963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba1777790 con 0x55bba1aaf1e0 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff07a445640 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff07000ba80 tx=0x7ff07000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eeab2cf6962f4b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.431+0000 7ff06b7fe640 1 -- 172.21.15.60:0/3510036985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff064045030 con 0x7ff074106f90 2024-07-24T22:04:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.432+0000 7ff068ff9640 1 -- 172.21.15.60:0/1908375963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070016070 con 0x55bba1aaf1e0 2024-07-24T22:04:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.432+0000 7ff068ff9640 1 -- 172.21.15.60:0/1908375963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0700054d0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.432+0000 7ff068ff9640 1 -- 172.21.15.60:0/1908375963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070010460 con 0x55bba1aaf1e0 2024-07-24T22:04:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.433+0000 7ff06b7fe640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 0x7ff0600b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.433+0000 7ff06b7fe640 1 -- 172.21.15.60:0/3510036985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7ff064103bf0 con 0x7ff074106f90 2024-07-24T22:04:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.433+0000 7ff07f403740 1 -- 172.21.15.60:0/1908375963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 msgr2=0x55bba1abac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.433+0000 7ff07f403740 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff07000ba80 tx=0x7ff07000bf40 comp rx=0 tx=0).stop 2024-07-24T22:04:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.435+0000 7ff07ac46640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 0x7ff0600b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07ac46640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 0x7ff0600b6410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff06c005e30 tx=0x7ff06c005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07f403740 1 -- 172.21.15.60:0/1908375963 shutdown_connections 2024-07-24T22:04:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07f403740 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1abb170 0x55bba1abd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07f403740 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abac30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07f403740 1 --2- 172.21.15.60:0/1908375963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aae8c0 0x55bba1aaeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.437+0000 7ff07f403740 1 -- 172.21.15.60:0/1908375963 >> 172.21.15.60:0/1908375963 conn(0x55bba1aac6f0 msgr2=0x55bba1aad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.439+0000 7ff07f403740 1 -- 172.21.15.60:0/1908375963 shutdown_connections 2024-07-24T22:04:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.439+0000 7ff07f403740 1 -- 172.21.15.60:0/1908375963 wait complete. 2024-07-24T22:04:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.440+0000 7ff07f403740 1 -- start start 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.440+0000 7ff07f403740 1 Processor -- start 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.440+0000 7ff07f403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1aba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 0x55bba1ab82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bba194dab0 con 0x55bba1abb170 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff079c44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff079c44640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:50024/0 (socket says 172.21.15.60:50024) 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff079c44640 1 -- 172.21.15.60:0/4038689111 learned_addr learned my addr 172.21.15.60:0/4038689111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07ac46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 0x55bba1ab82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bba199b9e0 con 0x55bba1aae8c0 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bba1abde50 con 0x55bba1aaf1e0 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.441+0000 7ff07f403740 10 client.0 _ll_register_callbacks cb 0x55bba1723a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 -- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 msgr2=0x55bba1ab7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 -- 172.21.15.60:0/4038689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 msgr2=0x55bba1aba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1aba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba17f0620 con 0x55bba1abb170 2024-07-24T22:04:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff07ac46640 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 0x55bba1ab82c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0045b0 tx=0x7ff06c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a6c220618d24f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.443+0000 7ff05dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.444+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00db90 con 0x55bba1abb170 2024-07-24T22:04:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.444+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bba19c2ce0 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.444+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7ff06c00dd30 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.444+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff06c01a420 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.445+0000 7ff07f403740 10 client.19062 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.445+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bba1ac96e0 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.445+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c01f030 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.445+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff06c00b040 con 0x55bba1abb170 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 10 client.19062 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 10 client.19062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 20 client.19062 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1acb040 0x55bba1acd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 4 client.19062 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bba19c2ce0 con 0x55bba1acb040 2024-07-24T22:04:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.446+0000 7ff07a445640 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1acb040 0x55bba1acd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.450+0000 7ff07a445640 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1acb040 0x55bba1acd4a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a21bedd326d9585 server_cookie=60f9e323e779852 in_seq=0 out_seq=0 2024-07-24T22:04:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.450+0000 7ff05dffb640 10 client.19062 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.453+0000 7ff05dffb640 1 -- 172.21.15.60:0/4038689111 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bba19c2ce0 con 0x55bba1acb040 2024-07-24T22:04:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.453+0000 7ff05dffb640 10 client.19062 handle_command_reply: tid=0 2024-07-24T22:04:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.453+0000 7ff07f403740 1 client.19062 shutdown 2024-07-24T22:04:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.461+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 msgr2=0x55bba1ab82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.462+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 0x55bba1ab82c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0045b0 tx=0x7ff06c004ab0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 shutdown_connections 2024-07-24T22:04:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1acb040 0x55bba1acd4a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1abb170 0x55bba1ab82c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 --2- 172.21.15.60:0/4038689111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1aba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.464+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 >> 172.21.15.60:0/4038689111 conn(0x55bba1aac6f0 msgr2=0x55bba1ab2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.465+0000 7ff05dffb640 0 client.19062 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.470+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 shutdown_connections 2024-07-24T22:04:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.471+0000 7ff07f403740 1 -- 172.21.15.60:0/4038689111 wait complete. 2024-07-24T22:04:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.475+0000 7ff07f403740 20 client.19062 trim_cache size 0 max 16384 2024-07-24T22:04:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.509+0000 7ff07f403740 1 Processor -- start 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 -- start start 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aae8c0 0x55bba1aadf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ab8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bfa90 con 0x55bba1aae8c0 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bd2a0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff07f403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bb110 con 0x55bba1abb170 2024-07-24T22:04:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff079c44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff079c44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35474/0 (socket says 172.21.15.60:35474) 2024-07-24T22:04:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.510+0000 7ff079c44640 1 -- 172.21.15.60:0/3497493834 learned_addr learned my addr 172.21.15.60:0/3497493834 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.511+0000 7ff07a445640 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aae8c0 0x55bba1aadf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.512+0000 7ff079c44640 1 -- 172.21.15.60:0/3497493834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 msgr2=0x55bba1ab8290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.512+0000 7ff079c44640 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ab8290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.512+0000 7ff079c44640 1 -- 172.21.15.60:0/3497493834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aae8c0 msgr2=0x55bba1aadf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.512+0000 7ff079c44640 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aae8c0 0x55bba1aadf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.512+0000 7ff079c44640 1 -- 172.21.15.60:0/3497493834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba1acd7e0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.516+0000 7ff079c44640 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff064031000 tx=0x7ff064032e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e42c196a8a1fa server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.517+0000 7ff05ffff640 1 -- 172.21.15.60:0/3497493834 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640ca1a0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.517+0000 7ff05ffff640 1 -- 172.21.15.60:0/3497493834 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0640328f0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.517+0000 7ff05ffff640 1 -- 172.21.15.60:0/3497493834 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640c9fc0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.518+0000 7ff07f403740 1 -- 172.21.15.60:0/3497493834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 msgr2=0x55bba1ab7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.518+0000 7ff07f403740 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff064031000 tx=0x7ff064032e10 comp rx=0 tx=0).stop 2024-07-24T22:04:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 -- 172.21.15.60:0/3497493834 shutdown_connections 2024-07-24T22:04:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ab8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aaf1e0 0x55bba1ab7d50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 --2- 172.21.15.60:0/3497493834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aae8c0 0x55bba1aadf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 -- 172.21.15.60:0/3497493834 >> 172.21.15.60:0/3497493834 conn(0x55bba1aac6f0 msgr2=0x55bba1aad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.519+0000 7ff07f403740 1 -- 172.21.15.60:0/3497493834 shutdown_connections 2024-07-24T22:04:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 -- 172.21.15.60:0/3497493834 wait complete. 2024-07-24T22:04:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 -- start start 2024-07-24T22:04:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 Processor -- start 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1ac37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ac01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bb760 con 0x55bba1aaf1e0 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.521+0000 7ff07f403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bcbd0 con 0x55bba1aae8c0 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.522+0000 7ff07f403740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741bbe30 con 0x55bba1abb170 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.522+0000 7ff07f403740 10 client.0 _ll_register_callbacks cb 0x7ff06c0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.522+0000 7ff07a445640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1ac37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.522+0000 7ff07a445640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1ac37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:35486/0 (socket says 172.21.15.60:35486) 2024-07-24T22:04:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.522+0000 7ff07a445640 1 -- 172.21.15.60:0/2775173308 learned_addr learned my addr 172.21.15.60:0/2775173308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.523+0000 7ff07ac46640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ac01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.523+0000 7ff079c44640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.524+0000 7ff079c44640 1 -- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 msgr2=0x55bba1ac01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.525+0000 7ff079c44640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ac01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.525+0000 7ff079c44640 1 -- 172.21.15.60:0/2775173308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 msgr2=0x55bba1ac37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.525+0000 7ff079c44640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1ac37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.525+0000 7ff079c44640 1 -- 172.21.15.60:0/2775173308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba1ac0a10 con 0x55bba1aaf1e0 2024-07-24T22:04:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.533+0000 7ff079c44640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abfca0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff0640c7ac0 tx=0x7ff0640cace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c858b2134e6706 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.534+0000 7ff05dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.538+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640c9fc0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.538+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0640328f0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.538+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7ff064003e00 con 0x55bba1aaf1e0 2024-07-24T22:04:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.539+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff0640ca1a0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.540+0000 7ff07f403740 10 client.19068 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.540+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bba1b34df0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.544+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0640328f0 con 0x55bba1aaf1e0 2024-07-24T22:04:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.544+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7ff0640cbb80 con 0x55bba1aaf1e0 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 10 client.19068 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 10 client.19068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 20 client.19068 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1b352d0 0x55bba1b376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 4 client.19068 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:04:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.546+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bba1a261d0 con 0x55bba1b352d0 2024-07-24T22:04:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.547+0000 7ff07a445640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1b352d0 0x55bba1b376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.552+0000 7ff07a445640 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1b352d0 0x55bba1b376f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c20ed1e5b08134 server_cookie=ac34eff25e19316e in_seq=0 out_seq=0 2024-07-24T22:04:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.552+0000 7ff05dffb640 10 client.19068 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.672+0000 7ff05dffb640 1 -- 172.21.15.60:0/2775173308 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bba1a261d0 con 0x55bba1b352d0 2024-07-24T22:04:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff05dffb640 10 client.19068 handle_command_reply: tid=0 2024-07-24T22:04:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 client.19068 shutdown 2024-07-24T22:04:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 msgr2=0x55bba1abfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abfca0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff0640c7ac0 tx=0x7ff0640cace0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 shutdown_connections 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55bba1b352d0 0x55bba1b376f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bba1abb170 0x55bba1ac01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bba1aaf1e0 0x55bba1abfca0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 --2- 172.21.15.60:0/2775173308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bba1aae8c0 0x55bba1ac37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 >> 172.21.15.60:0/2775173308 conn(0x55bba1aac6f0 msgr2=0x55bba1ab7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff05dffb640 0 client.19068 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 shutdown_connections 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 1 -- 172.21.15.60:0/2775173308 wait complete. 2024-07-24T22:04:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.673+0000 7ff07f403740 20 client.19068 trim_cache size 0 max 16384 2024-07-24T22:04:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:04:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:04:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:04:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:04:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 msgr2=0x7ff0600b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 0x7ff0600b6410 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff06c005e30 tx=0x7ff06c005d80 comp rx=0 tx=0).stop 2024-07-24T22:04:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 msgr2=0x7ff074110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff064006010 tx=0x7ff064004270 comp rx=0 tx=0).stop 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 shutdown_connections 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff0600b3f50 0x7ff0600b6410 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.674+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff074107910 0x7ff074110ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.675+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff074106f90 0x7ff074110980 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.675+0000 7ff07c6a9640 1 --2- 172.21.15.60:0/3510036985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff074101590 0x7ff074110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.675+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 >> 172.21.15.60:0/3510036985 conn(0x7ff0740fd010 msgr2=0x7ff0740ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.675+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 shutdown_connections 2024-07-24T22:04:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:04.675+0000 7ff07c6a9640 1 -- 172.21.15.60:0/3510036985 wait complete. 2024-07-24T22:04:05.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:05 smithi123 ceph-mon[37143]: osdmap e113: 12 total, 12 up, 12 in 2024-07-24T22:04:05.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:05 smithi123 ceph-mon[37143]: pgmap v925: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 82 KiB/s wr, 549 op/s 2024-07-24T22:04:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:05 smithi086 ceph-mon[37128]: osdmap e113: 12 total, 12 up, 12 in 2024-07-24T22:04:05.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:05 smithi086 ceph-mon[37128]: pgmap v925: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 82 KiB/s wr, 549 op/s 2024-07-24T22:04:05.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:05 smithi060 ceph-mon[33000]: osdmap e113: 12 total, 12 up, 12 in 2024-07-24T22:04:05.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:05 smithi060 ceph-mon[33000]: pgmap v925: 97 pgs: 97 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 82 KiB/s wr, 549 op/s 2024-07-24T22:04:07.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:04:07.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:07 smithi086 ceph-mon[37128]: pgmap v926: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 70 KiB/s wr, 124 op/s 2024-07-24T22:04:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:07 smithi060 ceph-mon[33000]: pgmap v926: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 70 KiB/s wr, 124 op/s 2024-07-24T22:04:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:07 smithi123 ceph-mon[37143]: pgmap v926: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 70 KiB/s wr, 124 op/s 2024-07-24T22:04:09.795 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:09 smithi123 ceph-mon[37143]: pgmap v927: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-07-24T22:04:09.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:09 smithi086 ceph-mon[37128]: pgmap v927: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-07-24T22:04:09.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:09 smithi060 ceph-mon[33000]: pgmap v927: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 79 KiB/s wr, 39 op/s 2024-07-24T22:04:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.951+0000 7f29018b9640 1 -- 172.21.15.60:0/3008106037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 msgr2=0x7f28f40966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.951+0000 7f29018b9640 1 --2- 172.21.15.60:0/3008106037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 0x7f28f40966d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f28f00074e0 tx=0x7f28f002b080 comp rx=0 tx=0).stop 2024-07-24T22:04:09.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 -- 172.21.15.60:0/3008106037 shutdown_connections 2024-07-24T22:04:09.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 --2- 172.21.15.60:0/3008106037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4096c10 0x7f28f40a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:09.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 --2- 172.21.15.60:0/3008106037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 0x7f28f40966d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:09.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 --2- 172.21.15.60:0/3008106037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28f4093e40 0x7f28f4094220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:09.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 -- 172.21.15.60:0/3008106037 >> 172.21.15.60:0/3008106037 conn(0x7f28f4009490 msgr2=0x7f28f400bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:09.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.959+0000 7f29018b9640 1 -- 172.21.15.60:0/3008106037 shutdown_connections 2024-07-24T22:04:09.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.963+0000 7f29018b9640 1 -- 172.21.15.60:0/3008106037 wait complete. 2024-07-24T22:04:09.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.963+0000 7f29018b9640 1 Processor -- start 2024-07-24T22:04:09.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 -- start start 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 0x7f28f40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28f4096c10 0x7f28f40aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40a3450 con 0x7f28f4096270 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40a32d0 con 0x7f28f4093e40 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.969+0000 7f29018b9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40a35d0 con 0x7f28f4096c10 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42034/0 (socket says 172.21.15.60:42034) 2024-07-24T22:04:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 -- 172.21.15.60:0/2136686718 learned_addr learned my addr 172.21.15.60:0/2136686718 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 -- 172.21.15.60:0/2136686718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28f4096c10 msgr2=0x7f28f40aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28f4096c10 0x7f28f40aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 -- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 msgr2=0x7f28f40a5310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:04:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 0x7f28f40a5310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.970+0000 7f29008b7640 1 -- 172.21.15.60:0/2136686718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40aade0 con 0x7f28f4093e40 2024-07-24T22:04:09.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.979+0000 7f29008b7640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b4a0 tx=0x7f28e800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:09.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.981+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8014070 con 0x7f28f4093e40 2024-07-24T22:04:09.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.981+0000 7f29018b9640 1 -- 172.21.15.60:0/2136686718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f40ab0d0 con 0x7f28f4093e40 2024-07-24T22:04:09.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.981+0000 7f29018b9640 1 -- 172.21.15.60:0/2136686718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f40af8e0 con 0x7f28f4093e40 2024-07-24T22:04:09.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.982+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e8010ac0 con 0x7f28f4093e40 2024-07-24T22:04:09.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.982+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8017c30 con 0x7f28f4093e40 2024-07-24T22:04:09.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.983+0000 7f29018b9640 1 -- 172.21.15.60:0/2136686718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f4094220 con 0x7f28f4093e40 2024-07-24T22:04:09.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.990+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f28e8017070 con 0x7f28f4093e40 2024-07-24T22:04:09.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.993+0000 7f28f9ffb640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 0x7f28ec0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:09.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.993+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f28e80e4c20 con 0x7f28f4093e40 2024-07-24T22:04:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.995+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f28e80ac000 con 0x7f28f4093e40 2024-07-24T22:04:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:09.998+0000 7f28fbfff640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 0x7f28ec0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:10.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.024+0000 7f28fbfff640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 0x7f28ec0b6410 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007320 tx=0x7f28f0005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:10.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.464+0000 7f29018b9640 1 -- 172.21.15.60:0/2136686718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28f40a5f00 con 0x7f28f4093e40 2024-07-24T22:04:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.470+0000 7f28f9ffb640 1 -- 172.21.15.60:0/2136686718 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f28e80b42d0 con 0x7f28f4093e40 2024-07-24T22:04:10.474 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:04:10.474 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:04:10.475 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:04:10.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.479+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 msgr2=0x7f28ec0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:10.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.479+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 0x7f28ec0b6410 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007320 tx=0x7f28f0005fd0 comp rx=0 tx=0).stop 2024-07-24T22:04:10.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.479+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 msgr2=0x7f28f40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:10.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.479+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b4a0 tx=0x7f28e800b960 comp rx=0 tx=0).stop 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 shutdown_connections 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f28ec0b3f50 0x7f28ec0b6410 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28f4096c10 0x7f28f40aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f4096270 0x7f28f40a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 --2- 172.21.15.60:0/2136686718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28f4093e40 0x7f28f40a4dd0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b4a0 tx=0x7f28e800b960 comp rx=0 tx=0).stop 2024-07-24T22:04:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.481+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 >> 172.21.15.60:0/2136686718 conn(0x7f28f4009490 msgr2=0x7f28f4092bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:10.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.482+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 shutdown_connections 2024-07-24T22:04:10.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:10.482+0000 7f28e77fe640 1 -- 172.21.15.60:0/2136686718 wait complete. 2024-07-24T22:04:10.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:10 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2136686718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:10.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:10 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2136686718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:11.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:10 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2136686718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --set-id 874f7a0d --release --await' 2024-07-24T22:04:11.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --set-id 874f7a0d --release --await 2024-07-24T22:04:11.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:11 smithi060 ceph-mon[33000]: pgmap v928: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-07-24T22:04:11.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:11 smithi086 ceph-mon[37128]: pgmap v928: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-07-24T22:04:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:11 smithi123 ceph-mon[37143]: pgmap v928: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-07-24T22:04:13.813 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:13 smithi060 ceph-mon[33000]: pgmap v929: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-07-24T22:04:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:13 smithi123 ceph-mon[37143]: pgmap v929: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-07-24T22:04:14.050 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:13 smithi086 ceph-mon[37128]: pgmap v929: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-07-24T22:04:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.108+0000 7f659fa81640 1 -- 172.21.15.60:0/2668775099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659811ba90 msgr2=0x7f659811de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.109+0000 7f659c81b640 1 -- 172.21.15.60:0/2668775099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590034020 con 0x7f659811ba90 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.109+0000 7f659fa81640 1 --2- 172.21.15.60:0/2668775099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659811ba90 0x7f659811de80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6590009ca0 tx=0x7f659002a970 comp rx=0 tx=0).stop 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 -- 172.21.15.60:0/2668775099 shutdown_connections 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 --2- 172.21.15.60:0/2668775099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f659811ba90 0x7f659811de80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 --2- 172.21.15.60:0/2668775099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6598109f20 0x7f659810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 --2- 172.21.15.60:0/2668775099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109b40 0x7f6598110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 -- 172.21.15.60:0/2668775099 >> 172.21.15.60:0/2668775099 conn(0x7f659806d980 msgr2=0x7f659806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.110+0000 7f659fa81640 1 -- 172.21.15.60:0/2668775099 shutdown_connections 2024-07-24T22:04:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.111+0000 7f659fa81640 1 -- 172.21.15.60:0/2668775099 wait complete. 2024-07-24T22:04:14.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.111+0000 7f659fa81640 1 Processor -- start 2024-07-24T22:04:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.111+0000 7f659fa81640 1 -- start start 2024-07-24T22:04:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 0x7f65981a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659811ba90 0x7f65981ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65981215e0 con 0x7f6598109b40 2024-07-24T22:04:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6598121460 con 0x7f659811ba90 2024-07-24T22:04:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.112+0000 7f659fa81640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6598121760 con 0x7f6598109f20 2024-07-24T22:04:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.115+0000 7f659d01c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.115+0000 7f659d01c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58632/0 (socket says 172.21.15.60:58632) 2024-07-24T22:04:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.115+0000 7f659d01c640 1 -- 172.21.15.60:0/2222143194 learned_addr learned my addr 172.21.15.60:0/2222143194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.115+0000 7f659e01e640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659811ba90 0x7f65981ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d81d640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 0x7f65981a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d01c640 1 -- 172.21.15.60:0/2222143194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659811ba90 msgr2=0x7f65981ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d01c640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659811ba90 0x7f65981ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d01c640 1 -- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 msgr2=0x7f65981a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d01c640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 0x7f65981a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d01c640 1 -- 172.21.15.60:0/2222143194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981ada80 con 0x7f6598109f20 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.116+0000 7f659d81d640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 0x7f65981a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.117+0000 7f659d01c640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f658800d950 tx=0x7f658800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.118+0000 7f658effd640 1 -- 172.21.15.60:0/2222143194 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588017070 con 0x7f6598109f20 2024-07-24T22:04:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.118+0000 7f658effd640 1 -- 172.21.15.60:0/2222143194 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6588009c30 con 0x7f6598109f20 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.118+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981add70 con 0x7f6598109f20 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.118+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6598111270 con 0x7f6598109f20 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.122+0000 7f65a1e32740 1 Processor -- start 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.122+0000 7f65a1e32740 1 -- start start 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfc1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 0x55f8bcfd06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f8bcf38dd0 con 0x55f8bcfc2350 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8bcf282d0 con 0x55f8bcfc1a30 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.123+0000 7f65a1e32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f8bcf2c310 con 0x55f8bcfce2e0 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.124+0000 7f658effd640 1 -- 172.21.15.60:0/2222143194 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658800f620 con 0x7f6598109f20 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.124+0000 7f659d81d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.125+0000 7f659d81d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36422/0 (socket says 172.21.15.60:36422) 2024-07-24T22:04:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.125+0000 7f659d81d640 1 -- 172.21.15.60:0/2528000247 learned_addr learned my addr 172.21.15.60:0/2528000247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.125+0000 7f659d01c640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 0x55f8bcfd06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.128+0000 7f658effd640 1 -- 172.21.15.60:0/2222143194 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6588005040 con 0x7f6598109f20 2024-07-24T22:04:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.128+0000 7f659e01e640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfc1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d01c640 1 -- 172.21.15.60:0/2528000247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 msgr2=0x55f8bcfc1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d01c640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfc1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d01c640 1 -- 172.21.15.60:0/2528000247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 msgr2=0x55f8bcfcdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d01c640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d01c640 1 -- 172.21.15.60:0/2528000247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8bcc88b00 con 0x55f8bcfce2e0 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659e01e640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfc1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f659d81d640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f658effd640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 0x7f65840b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.129+0000 7f658effd640 1 -- 172.21.15.60:0/2222143194 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f65880ea940 con 0x7f6598109f20 2024-07-24T22:04:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.130+0000 7f659e01e640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 0x7f65840b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.130+0000 7f659e01e640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 0x7f65840b62e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6590004af0 tx=0x7f6590005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.134+0000 7f659d01c640 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 0x55f8bcfd06d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65880e7fd0 tx=0x7f65880eb540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280cd01c22332879 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.135+0000 7f6583fff640 1 -- 172.21.15.60:0/2528000247 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588020020 con 0x55f8bcfce2e0 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.135+0000 7f6583fff640 1 -- 172.21.15.60:0/2528000247 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65880b0fa0 con 0x55f8bcfce2e0 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.135+0000 7f6583fff640 1 -- 172.21.15.60:0/2528000247 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588009de0 con 0x55f8bcfce2e0 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.136+0000 7f65a1e32740 1 -- 172.21.15.60:0/2528000247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 msgr2=0x55f8bcfd06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.136+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 0x55f8bcfd06d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65880e7fd0 tx=0x7f65880eb540 comp rx=0 tx=0).stop 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.137+0000 7f65a1e32740 1 -- 172.21.15.60:0/2528000247 shutdown_connections 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.137+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfce2e0 0x55f8bcfd06d0 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65880e7fd0 tx=0x7f65880eb540 comp rx=0 tx=0).stop 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.137+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.137+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2528000247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfc1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.137+0000 7f65a1e32740 1 -- 172.21.15.60:0/2528000247 >> 172.21.15.60:0/2528000247 conn(0x55f8bcfbf860 msgr2=0x55f8bcfc0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.141+0000 7f65a1e32740 1 -- 172.21.15.60:0/2528000247 shutdown_connections 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.141+0000 7f65a1e32740 1 -- 172.21.15.60:0/2528000247 wait complete. 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.141+0000 7f65a1e32740 1 -- start start 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 Processor -- start 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfcb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfda2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f8bcd8bbb0 con 0x55f8bcfcbf20 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8bceca900 con 0x55f8bcfc2350 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f8bcfd0f60 con 0x55f8bcfc1a30 2024-07-24T22:04:14.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.142+0000 7f65a1e32740 10 client.0 _ll_register_callbacks cb 0x55f8bced4c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.143+0000 7f659d01c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.143+0000 7f659d01c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:42068/0 (socket says 172.21.15.60:42068) 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.143+0000 7f659d01c640 1 -- 172.21.15.60:0/90410722 learned_addr learned my addr 172.21.15.60:0/90410722 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.143+0000 7f659d81d640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfcb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.144+0000 7f659d81d640 1 -- 172.21.15.60:0/90410722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 msgr2=0x55f8bcfcb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.144+0000 7f659d81d640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.144+0000 7f659d81d640 1 -- 172.21.15.60:0/90410722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 msgr2=0x55f8bcfda2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.144+0000 7f659d81d640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfda2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.144+0000 7f659d81d640 1 -- 172.21.15.60:0/90410722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8bccff960 con 0x55f8bcfc1a30 2024-07-24T22:04:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.147+0000 7f659d81d640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfcb4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65940058c0 tx=0x7f6594010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=274e83bc0a1bf3f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65817fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659401d060 con 0x55f8bcfc1a30 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65880b0fa0 con 0x55f8bcfc1a30 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f659400ecf0 con 0x55f8bcfc1a30 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6594018430 con 0x55f8bcfc1a30 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65a1e32740 10 client.28300 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.149+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f8bcfdd150 con 0x55f8bcfc1a30 2024-07-24T22:04:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.152+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65880b0fa0 con 0x55f8bcfc1a30 2024-07-24T22:04:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.152+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6594020780 con 0x55f8bcfc1a30 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 10 client.28300 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 10 client.28300 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 20 client.28300 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bcfdea60 0x55f8bcfe0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 4 client.28300 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.153+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8bcf3ad20 con 0x55f8bcfdea60 2024-07-24T22:04:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.155+0000 7f659d01c640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bcfdea60 0x55f8bcfe0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.167+0000 7f659d01c640 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bcfdea60 0x55f8bcfe0ee0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1adf90b22be05bf server_cookie=c56d562004b0a040 in_seq=0 out_seq=0 2024-07-24T22:04:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.167+0000 7f65817fa640 10 client.28300 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:14.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65817fa640 1 -- 172.21.15.60:0/90410722 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8bcf3ad20 con 0x55f8bcfdea60 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65817fa640 10 client.28300 handle_command_reply: tid=0 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 client.28300 shutdown 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 msgr2=0x55f8bcfcb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfcb4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65940058c0 tx=0x7f6594010c80 comp rx=0 tx=0).stop 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 shutdown_connections 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bcfdea60 0x55f8bcfe0ee0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfda2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc2350 0x55f8bcfcb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 --2- 172.21.15.60:0/90410722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfcb4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.170+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 >> 172.21.15.60:0/90410722 conn(0x55f8bcfbf860 msgr2=0x55f8bcfc5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.171+0000 7f65817fa640 0 client.28300 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.172+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 shutdown_connections 2024-07-24T22:04:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.172+0000 7f65a1e32740 1 -- 172.21.15.60:0/90410722 wait complete. 2024-07-24T22:04:14.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.176+0000 7f65a1e32740 20 client.28300 trim_cache size 0 max 16384 2024-07-24T22:04:14.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.219+0000 7f65a1e32740 1 Processor -- start 2024-07-24T22:04:14.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.219+0000 7f65a1e32740 1 -- start start 2024-07-24T22:04:14.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.219+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfe67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f65a1e32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0113f0 con 0x55f8bcfc1a30 2024-07-24T22:04:14.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f65a1e32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f657c00ec00 con 0x55f8bcfcbf20 2024-07-24T22:04:14.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f659d81d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f65a1e32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f657c00ca70 con 0x55f8bcfc2350 2024-07-24T22:04:14.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.221+0000 7f659d01c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.221+0000 7f659d01c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:58660/0 (socket says 172.21.15.60:58660) 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.221+0000 7f659d01c640 1 -- 172.21.15.60:0/2715938166 learned_addr learned my addr 172.21.15.60:0/2715938166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.220+0000 7f659d81d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfbf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36444/0 (socket says 172.21.15.60:36444) 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.224+0000 7f659d01c640 1 -- 172.21.15.60:0/2715938166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfcbf20 msgr2=0x55f8bcfe67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.224+0000 7f659d01c640 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfe67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.224+0000 7f659d01c640 1 -- 172.21.15.60:0/2715938166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 msgr2=0x55f8bcfbf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.224+0000 7f659d01c640 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfbf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.224+0000 7f659d01c640 1 -- 172.21.15.60:0/2715938166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8bcfc99e0 con 0x55f8bcfc2350 2024-07-24T22:04:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.225+0000 7f659d01c640 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f65880ad480 tx=0x7f65880e8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4b7d7ffb0e99e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.227+0000 7f65837fe640 1 -- 172.21.15.60:0/2715938166 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588020040 con 0x55f8bcfc2350 2024-07-24T22:04:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.227+0000 7f65837fe640 1 -- 172.21.15.60:0/2715938166 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65880b0fa0 con 0x55f8bcfc2350 2024-07-24T22:04:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.227+0000 7f65837fe640 1 -- 172.21.15.60:0/2715938166 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880b1490 con 0x55f8bcfc2350 2024-07-24T22:04:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.228+0000 7f65a1e32740 1 -- 172.21.15.60:0/2715938166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 msgr2=0x55f8bcfc5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.228+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f65880ad480 tx=0x7f65880e8000 comp rx=0 tx=0).stop 2024-07-24T22:04:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 -- 172.21.15.60:0/2715938166 shutdown_connections 2024-07-24T22:04:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfe67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfc5b80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 --2- 172.21.15.60:0/2715938166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfbf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 -- 172.21.15.60:0/2715938166 >> 172.21.15.60:0/2715938166 conn(0x55f8bcfbf860 msgr2=0x55f8bcfcd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 -- 172.21.15.60:0/2715938166 shutdown_connections 2024-07-24T22:04:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.229+0000 7f65a1e32740 1 -- 172.21.15.60:0/2715938166 wait complete. 2024-07-24T22:04:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.230+0000 7f65a1e32740 1 -- start start 2024-07-24T22:04:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.230+0000 7f65a1e32740 1 Processor -- start 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfd6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f657c00d0c0 con 0x55f8bcfcbf20 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f657c00e530 con 0x55f8bcfc1a30 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f657c00d790 con 0x55f8bcfc2350 2024-07-24T22:04:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f65a1e32740 10 client.0 _ll_register_callbacks cb 0x7f6594010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f659e01e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f659e01e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36454/0 (socket says 172.21.15.60:36454) 2024-07-24T22:04:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.231+0000 7f659e01e640 1 -- 172.21.15.60:0/867031772 learned_addr learned my addr 172.21.15.60:0/867031772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659d01c640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfd6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659e01e640 1 -- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 msgr2=0x55f8bcfd6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659e01e640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfd6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659e01e640 1 -- 172.21.15.60:0/867031772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 msgr2=0x55f8bcfd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659e01e640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.232+0000 7f659e01e640 1 -- 172.21.15.60:0/867031772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8bcfd3760 con 0x55f8bcfcbf20 2024-07-24T22:04:14.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.242+0000 7f659e01e640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6590035000 tx=0x7f6590003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83fb4f17c054034 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.242+0000 7f65817fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.243+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590034040 con 0x55f8bcfcbf20 2024-07-24T22:04:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.243+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f6590030940 con 0x55f8bcfcbf20 2024-07-24T22:04:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.243+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65880b0fa0 con 0x55f8bcfcbf20 2024-07-24T22:04:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.245+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6590030b40 con 0x55f8bcfcbf20 2024-07-24T22:04:14.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.247+0000 7f65a1e32740 10 client.19083 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:14.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.247+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f8bd049ec0 con 0x55f8bcfcbf20 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6590030ce0 con 0x55f8bcfcbf20 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f659005d020 con 0x55f8bcfcbf20 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 10 client.19083 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 10 client.19083 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 20 client.19083 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bd04a700 0x55f8bd04cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 4 client.19083 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "874f7a0d", "release": true, "await": true}] 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.251+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "set_id": "874f7a0d", "release": true, "await": true}) -- 0x7f65880b0fa0 con 0x55f8bd04a700 2024-07-24T22:04:14.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.254+0000 7f659d81d640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bd04a700 0x55f8bd04cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:14.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.255+0000 7f659d81d640 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bd04a700 0x55f8bd04cb20 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717df03a40571d12 server_cookie=592f2ee1eca29568 in_seq=0 out_seq=0 2024-07-24T22:04:14.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.257+0000 7f65817fa640 10 client.19083 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:14.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.803+0000 7f65817fa640 1 -- 172.21.15.60:0/867031772 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f65880b0fa0 con 0x55f8bd04a700 2024-07-24T22:04:14.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.803+0000 7f65817fa640 10 client.19083 handle_command_reply: tid=0 2024-07-24T22:04:14.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.803+0000 7f65a1e32740 1 client.19083 shutdown 2024-07-24T22:04:14.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.803+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 msgr2=0x55f8bcfd3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.803+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6590035000 tx=0x7f6590003ef0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 shutdown_connections 2024-07-24T22:04:14.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55f8bd04a700 0x55f8bd04cb20 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8bcfcbf20 0x55f8bcfd3220 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65817fa640 0 client.19083 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8bcfc2350 0x55f8bcfd6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 --2- 172.21.15.60:0/867031772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8bcfc1a30 0x55f8bcfd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 >> 172.21.15.60:0/867031772 conn(0x55f8bcfbf860 msgr2=0x55f8bcfd90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 shutdown_connections 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 1 -- 172.21.15.60:0/867031772 wait complete. 2024-07-24T22:04:14.807 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 46, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 56, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": { 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "874f7a0d": { 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 56, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "age_ref": 0.0, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "timeout": 90.0, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "expiration": 147.3, 2024-07-24T22:04:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "members": { 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "file:/": { 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "excluded": false, 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "state": { 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RELEASED", 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:14.809 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.804+0000 7f65a1e32740 20 client.19083 trim_cache size 0 max 16384 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.805+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 msgr2=0x7f65840b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.805+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 0x7f65840b62e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6590004af0 tx=0x7f6590005d80 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 msgr2=0x7f65981a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f658800d950 tx=0x7f658800de10 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 shutdown_connections 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f65840b3e20 0x7f65840b62e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f659811ba90 0x7f65981ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598109f20 0x7f65981a7f10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 --2- 172.21.15.60:0/2222143194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6598109b40 0x7f65981a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:14.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 >> 172.21.15.60:0/2222143194 conn(0x7f659806d980 msgr2=0x7f659811be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 shutdown_connections 2024-07-24T22:04:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:14.806+0000 7f659fa81640 1 -- 172.21.15.60:0/2222143194 wait complete. 2024-07-24T22:04:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:04:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:04:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:04:14.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:14 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:04:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:04:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:04:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:04:14.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:14 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:04:15.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:04:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:04:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:04:15.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:14 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:04:15.540 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '874f7a0d', total seconds elapsed: 35.6 2024-07-24T22:04:15.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:15 smithi060 ceph-mon[33000]: pgmap v930: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-24T22:04:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:15 smithi123 ceph-mon[37143]: pgmap v930: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-24T22:04:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:15 smithi086 ceph-mon[37128]: pgmap v930: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-07-24T22:04:17.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:17 smithi060 ceph-mon[33000]: pgmap v931: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-24T22:04:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.827+0000 7f5f07265640 1 -- 172.21.15.60:0/829096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f00103ef0 msgr2=0x7f5f0010c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.827+0000 7f5f07265640 1 --2- 172.21.15.60:0/829096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f00103ef0 0x7f5f0010c3d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000fb10 tx=0x7f5ef002e570 comp rx=0 tx=0).stop 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 -- 172.21.15.60:0/829096657 shutdown_connections 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 --2- 172.21.15.60:0/829096657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0010c9f0 0x7f5f0010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 --2- 172.21.15.60:0/829096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f00103ef0 0x7f5f0010c3d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 --2- 172.21.15.60:0/829096657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 0x7f5f001039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 -- 172.21.15.60:0/829096657 >> 172.21.15.60:0/829096657 conn(0x7f5f000782e0 msgr2=0x7f5f000ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 -- 172.21.15.60:0/829096657 shutdown_connections 2024-07-24T22:04:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 -- 172.21.15.60:0/829096657 wait complete. 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 Processor -- start 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.828+0000 7f5f07265640 1 -- start start 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 0x7f5f0006c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 0x7f5f0006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 0x7f5f00072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00118ca0 con 0x7f5f00103ef0 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00118b20 con 0x7f5f001035d0 2024-07-24T22:04:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f07265640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00118e20 con 0x7f5f0010c9f0 2024-07-24T22:04:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f05802640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 0x7f5f00072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f04800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 0x7f5f0006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f05802640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 0x7f5f00072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44642/0 (socket says 172.21.15.60:44642) 2024-07-24T22:04:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f05802640 1 -- 172.21.15.60:0/1508928744 learned_addr learned my addr 172.21.15.60:0/1508928744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.829+0000 7f5f05001640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 0x7f5f0006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 -- 172.21.15.60:0/1508928744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 msgr2=0x7f5f00072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 0x7f5f00072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 -- 172.21.15.60:0/1508928744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 msgr2=0x7f5f0006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 0x7f5f0006c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 -- 172.21.15.60:0/1508928744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00072900 con 0x7f5f00103ef0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f04800640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 0x7f5f0006cda0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0037040 tx=0x7f5ef0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5ef67fc640 1 -- 172.21.15.60:0/1508928744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0004030 con 0x7f5f00103ef0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5ef67fc640 1 -- 172.21.15.60:0/1508928744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef000e970 con 0x7f5f00103ef0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5ef67fc640 1 -- 172.21.15.60:0/1508928744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0003a10 con 0x7f5f00103ef0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.830+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f00072b90 con 0x7f5f00103ef0 2024-07-24T22:04:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.831+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f000732a0 con 0x7f5f00103ef0 2024-07-24T22:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.834+0000 7f5ef67fc640 1 -- 172.21.15.60:0/1508928744 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5ef000c070 con 0x7f5f00103ef0 2024-07-24T22:04:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.835+0000 7f5f09fc3740 1 Processor -- start 2024-07-24T22:04:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.836+0000 7f5ef67fc640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 0x7f5eec0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.836+0000 7f5ef67fc640 1 -- 172.21.15.60:0/1508928744 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5ef0104860 con 0x7f5f00103ef0 2024-07-24T22:04:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 -- start start 2024-07-24T22:04:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x5583835f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558383557280 con 0x5583835e21e0 2024-07-24T22:04:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.837+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558383546780 con 0x5583835e18c0 2024-07-24T22:04:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.838+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55838354a7c0 con 0x5583835ee170 2024-07-24T22:04:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:38916/0 (socket says 172.21.15.60:38916) 2024-07-24T22:04:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 -- 172.21.15.60:0/2670247842 learned_addr learned my addr 172.21.15.60:0/2670247842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f04800640 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05802640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 0x7f5eec0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 -- 172.21.15.60:0/2670247842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 msgr2=0x5583835f0560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x5583835f0560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 -- 172.21.15.60:0/2670247842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 msgr2=0x5583835edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.840+0000 7f5f05001640 1 -- 172.21.15.60:0/2670247842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583832aa790 con 0x5583835e18c0 2024-07-24T22:04:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.841+0000 7f5f05001640 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00cbf0 tx=0x7f5efc00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42220276854fc6a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.842+0000 7f5eeb7fe640 1 -- 172.21.15.60:0/2670247842 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc010630 con 0x5583835e18c0 2024-07-24T22:04:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.842+0000 7f5eeb7fe640 1 -- 172.21.15.60:0/2670247842 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5efc010c80 con 0x5583835e18c0 2024-07-24T22:04:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.842+0000 7f5eeb7fe640 1 -- 172.21.15.60:0/2670247842 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc00fb80 con 0x5583835e18c0 2024-07-24T22:04:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.842+0000 7f5f05802640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 0x7f5eec0b6300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80049d0 tx=0x7f5ef8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.843+0000 7f5f09fc3740 1 -- 172.21.15.60:0/2670247842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 msgr2=0x5583835e1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.843+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00cbf0 tx=0x7f5efc00e720 comp rx=0 tx=0).stop 2024-07-24T22:04:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 -- 172.21.15.60:0/2670247842 shutdown_connections 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x5583835f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/2670247842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835e1ca0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 -- 172.21.15.60:0/2670247842 >> 172.21.15.60:0/2670247842 conn(0x5583835df6f0 msgr2=0x5583835e0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 -- 172.21.15.60:0/2670247842 shutdown_connections 2024-07-24T22:04:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 -- 172.21.15.60:0/2670247842 wait complete. 2024-07-24T22:04:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 -- start start 2024-07-24T22:04:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.845+0000 7f5f09fc3740 1 Processor -- start 2024-07-24T22:04:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 0x5583835ed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835eae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835ee170 0x5583835eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558383480ab0 con 0x5583835e21e0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583834ce9e0 con 0x5583835ee170 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583835f0e50 con 0x5583835e18c0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f09fc3740 10 client.0 _ll_register_callbacks cb 0x558383256a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f05001640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 0x5583835ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f05001640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 0x5583835ed960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f05001640 1 -- 172.21.15.60:0/1563767984 learned_addr learned my addr 172.21.15.60:0/1563767984 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f04800640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.846+0000 7f5f05802640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835ee170 0x5583835eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 -- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 msgr2=0x5583835ed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 0x5583835ed960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 -- 172.21.15.60:0/1563767984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835ee170 msgr2=0x5583835eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835ee170 0x5583835eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 -- 172.21.15.60:0/1563767984 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558383323620 con 0x5583835e21e0 2024-07-24T22:04:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5f04800640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835eae40 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0101ef0 tx=0x7f5ef00cabe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bcb82b3ace98bf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.847+0000 7f5ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00cf9b0 con 0x5583835e21e0 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5ef00c79a0 con 0x5583835e21e0 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583834f5ce0 con 0x5583835e21e0 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5ef00ea050 con 0x5583835e21e0 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5f09fc3740 10 client.19095 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.848+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5583835fc680 con 0x5583835e21e0 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583834f5ce0 con 0x5583835e21e0 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5ef004f060 con 0x5583835e21e0 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 10 client.19095 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 10 client.19095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 20 client.19095 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5583835fdae0 0x5583835fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 4 client.19095 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583835591d0 con 0x5583835fdae0 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.849+0000 7f5f05001640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5583835fdae0 0x5583835fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.850+0000 7f5ee8ff9640 10 client.19095 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.850+0000 7f5f05001640 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5583835fdae0 0x5583835fff50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0db64f5644d2aa server_cookie=94df4c4e4c9a9e71 in_seq=0 out_seq=0 2024-07-24T22:04:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1563767984 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583835591d0 con 0x5583835fdae0 2024-07-24T22:04:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5ee8ff9640 10 client.19095 handle_command_reply: tid=0 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 client.19095 shutdown 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 msgr2=0x5583835eae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835eae40 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0101ef0 tx=0x7f5ef00cabe0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 shutdown_connections 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5583835fdae0 0x5583835fff50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835ee170 0x5583835eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835eae40 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1563767984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835e18c0 0x5583835ed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 >> 172.21.15.60:0/1563767984 conn(0x5583835df6f0 msgr2=0x5583835e5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.851+0000 7f5ee8ff9640 0 client.19095 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.852+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 shutdown_connections 2024-07-24T22:04:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.852+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1563767984 wait complete. 2024-07-24T22:04:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.852+0000 7f5f09fc3740 20 client.19095 trim_cache size 0 max 16384 2024-07-24T22:04:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.881+0000 7f5f09fc3740 1 Processor -- start 2024-07-24T22:04:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.881+0000 7f5f09fc3740 1 -- start start 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e18c0 0x5583835f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e21e0 0x558383605860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bfa10 con 0x5583835e18c0 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bd220 con 0x5583835e21e0 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bb090 con 0x5583835ee170 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f05802640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f05802640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44680/0 (socket says 172.21.15.60:44680) 2024-07-24T22:04:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.882+0000 7f5f05802640 1 -- 172.21.15.60:0/261523999 learned_addr learned my addr 172.21.15.60:0/261523999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05001640 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e18c0 0x5583835f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f04800640 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e21e0 0x558383605860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05802640 1 -- 172.21.15.60:0/261523999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e21e0 msgr2=0x558383605860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05802640 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e21e0 0x558383605860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05802640 1 -- 172.21.15.60:0/261523999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e18c0 msgr2=0x5583835f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05802640 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e18c0 0x5583835f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.883+0000 7f5f05802640 1 -- 172.21.15.60:0/261523999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583835e2010 con 0x5583835ee170 2024-07-24T22:04:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.884+0000 7f5f05802640 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800f6b0 tx=0x7f5ef8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637167b0c6d8f8c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.884+0000 7f5eeaffd640 1 -- 172.21.15.60:0/261523999 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef801f5d0 con 0x5583835ee170 2024-07-24T22:04:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.884+0000 7f5eeaffd640 1 -- 172.21.15.60:0/261523999 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef801fc20 con 0x5583835ee170 2024-07-24T22:04:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.885+0000 7f5eeaffd640 1 -- 172.21.15.60:0/261523999 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8027ce0 con 0x5583835ee170 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 -- 172.21.15.60:0/261523999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 msgr2=0x558383605da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800f6b0 tx=0x7f5ef8014000 comp rx=0 tx=0).stop 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 -- 172.21.15.60:0/261523999 shutdown_connections 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383605da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e21e0 0x558383605860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/261523999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e18c0 0x5583835f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 -- 172.21.15.60:0/261523999 >> 172.21.15.60:0/261523999 conn(0x5583835df6f0 msgr2=0x5583835eae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 -- 172.21.15.60:0/261523999 shutdown_connections 2024-07-24T22:04:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.886+0000 7f5f09fc3740 1 -- 172.21.15.60:0/261523999 wait complete. 2024-07-24T22:04:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.887+0000 7f5f09fc3740 1 -- start start 2024-07-24T22:04:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.887+0000 7f5f09fc3740 1 Processor -- start 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383666830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bb6e0 con 0x5583835e21e0 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bcb50 con 0x5583835e18c0 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001bbdb0 con 0x5583835ee170 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f09fc3740 10 client.0 _ll_register_callbacks cb 0x7f5ef00c72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f04800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f04800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59078/0 (socket says 172.21.15.60:59078) 2024-07-24T22:04:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f04800640 1 -- 172.21.15.60:0/1437327504 learned_addr learned my addr 172.21.15.60:0/1437327504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 -- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 msgr2=0x558383666830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383666830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 -- 172.21.15.60:0/1437327504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 msgr2=0x5583835f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f05001640 1 -- 172.21.15.60:0/1437327504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558383666f70 con 0x5583835e18c0 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.888+0000 7f5f04800640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.889+0000 7f5f05001640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835f3820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x558383600800 tx=0x7f5efc00f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ec21f277a233de server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.889+0000 7f5ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.889+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02b020 con 0x5583835e18c0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.889+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef801fc20 con 0x5583835e18c0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.889+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5efc005590 con 0x5583835e18c0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.890+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5efc036050 con 0x5583835e18c0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.890+0000 7f5f09fc3740 10 client.28442 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.890+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558383668280 con 0x5583835e18c0 2024-07-24T22:04:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.890+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef801fc20 con 0x5583835e18c0 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5efc004ad0 con 0x5583835e18c0 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 10 client.28442 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 10 client.28442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 20 client.28442 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558383668a20 0x55838366ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 4 client.28442 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.891+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558383554c70 con 0x558383668a20 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.892+0000 7f5f04800640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558383668a20 0x55838366ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.892+0000 7f5ee8ff9640 10 client.28442 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.893+0000 7f5f04800640 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558383668a20 0x55838366ae40 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b436fa1939e21deb server_cookie=ebf5d31280153f05 in_seq=0 out_seq=0 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5ee8ff9640 1 -- 172.21.15.60:0/1437327504 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558383554c70 con 0x558383668a20 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5ee8ff9640 10 client.28442 handle_command_reply: tid=0 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 client.28442 shutdown 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 msgr2=0x5583835f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835f3820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x558383600800 tx=0x7f5efc00f6c0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 shutdown_connections 2024-07-24T22:04:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558383668a20 0x55838366ae40 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583835ee170 0x558383666830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583835e21e0 0x5583835f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 --2- 172.21.15.60:0/1437327504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583835e18c0 0x5583835f3820 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 >> 172.21.15.60:0/1437327504 conn(0x5583835df6f0 msgr2=0x5583835ec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5ee8ff9640 0 client.28442 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 shutdown_connections 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.951+0000 7f5f09fc3740 1 -- 172.21.15.60:0/1437327504 wait complete. 2024-07-24T22:04:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.952+0000 7f5f09fc3740 20 client.28442 trim_cache size 0 max 16384 2024-07-24T22:04:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:04:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:04:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:04:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:04:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.955+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 msgr2=0x7f5eec0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.955+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 0x7f5eec0b6300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80049d0 tx=0x7f5ef8009500 comp rx=0 tx=0).stop 2024-07-24T22:04:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.955+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 msgr2=0x7f5f0006cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.955+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 0x7f5f0006cda0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0037040 tx=0x7f5ef0004970 comp rx=0 tx=0).stop 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 shutdown_connections 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5eec0b3e40 0x7f5eec0b6300 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f0010c9f0 0x7f5f00072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f00103ef0 0x7f5f0006cda0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 --2- 172.21.15.60:0/1508928744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f001035d0 0x7f5f0006c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.957+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 >> 172.21.15.60:0/1508928744 conn(0x7f5f000782e0 msgr2=0x7f5f000ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.958+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 shutdown_connections 2024-07-24T22:04:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:17.958+0000 7f5f07265640 1 -- 172.21.15.60:0/1508928744 wait complete. 2024-07-24T22:04:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:17 smithi123 ceph-mon[37143]: pgmap v931: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-24T22:04:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:17 smithi086 ceph-mon[37128]: pgmap v931: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-07-24T22:04:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:19 smithi123 ceph-mon[37143]: pgmap v932: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 329 KiB/s wr, 24 op/s 2024-07-24T22:04:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:19 smithi086 ceph-mon[37128]: pgmap v932: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 329 KiB/s wr, 24 op/s 2024-07-24T22:04:20.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:19 smithi060 ceph-mon[33000]: pgmap v932: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 329 KiB/s wr, 24 op/s 2024-07-24T22:04:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:21 smithi123 ceph-mon[37143]: pgmap v933: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 324 KiB/s wr, 24 op/s 2024-07-24T22:04:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:21 smithi086 ceph-mon[37128]: pgmap v933: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 324 KiB/s wr, 24 op/s 2024-07-24T22:04:22.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:21 smithi060 ceph-mon[33000]: pgmap v933: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 324 KiB/s wr, 24 op/s 2024-07-24T22:04:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:23 smithi123 ceph-mon[37143]: pgmap v934: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-07-24T22:04:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:23 smithi086 ceph-mon[37128]: pgmap v934: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-07-24T22:04:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:23 smithi060 ceph-mon[33000]: pgmap v934: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 572 KiB/s wr, 44 op/s 2024-07-24T22:04:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:25 smithi123 ceph-mon[37143]: pgmap v935: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 738 KiB/s wr, 63 op/s 2024-07-24T22:04:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:25 smithi086 ceph-mon[37128]: pgmap v935: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 738 KiB/s wr, 63 op/s 2024-07-24T22:04:26.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:25 smithi060 ceph-mon[33000]: pgmap v935: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 738 KiB/s wr, 63 op/s 2024-07-24T22:04:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:27 smithi123 ceph-mon[37143]: pgmap v936: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 870 KiB/s wr, 85 op/s 2024-07-24T22:04:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:27 smithi086 ceph-mon[37128]: pgmap v936: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 870 KiB/s wr, 85 op/s 2024-07-24T22:04:28.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:27 smithi060 ceph-mon[33000]: pgmap v936: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 870 KiB/s wr, 85 op/s 2024-07-24T22:04:28.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:04:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:29.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:29 smithi060 ceph-mon[33000]: pgmap v937: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 868 KiB/s wr, 93 op/s 2024-07-24T22:04:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:29 smithi123 ceph-mon[37143]: pgmap v937: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 868 KiB/s wr, 93 op/s 2024-07-24T22:04:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:29 smithi086 ceph-mon[37128]: pgmap v937: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 868 KiB/s wr, 93 op/s 2024-07-24T22:04:30.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.939+0000 7f4a36d0d640 1 -- 172.21.15.60:0/1190944695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 msgr2=0x7f4a3010c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:30.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.941+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/1190944695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a3010c6c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009860 tx=0x7f4a1c02a360 comp rx=0 tx=0).stop 2024-07-24T22:04:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.942+0000 7f4a36d0d640 1 -- 172.21.15.60:0/1190944695 shutdown_connections 2024-07-24T22:04:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.942+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/1190944695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a3010cc00 0x7f4a3010eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.942+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/1190944695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a3010c6c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.942+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/1190944695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a301015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.942+0000 7f4a36d0d640 1 -- 172.21.15.60:0/1190944695 >> 172.21.15.60:0/1190944695 conn(0x7f4a300778d0 msgr2=0x7f4a30079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:30.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.943+0000 7f4a36d0d640 1 -- 172.21.15.60:0/1190944695 shutdown_connections 2024-07-24T22:04:30.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.943+0000 7f4a36d0d640 1 -- 172.21.15.60:0/1190944695 wait complete. 2024-07-24T22:04:30.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.944+0000 7f4a36d0d640 1 Processor -- start 2024-07-24T22:04:30.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.945+0000 7f4a36d0d640 1 -- start start 2024-07-24T22:04:30.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.945+0000 7f4a36d0d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a30198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.946+0000 7f4a36d0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a30198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.946+0000 7f4a34aa9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a30198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.946+0000 7f4a34aa9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a30198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54792/0 (socket says 172.21.15.60:54792) 2024-07-24T22:04:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.946+0000 7f4a34aa9640 1 -- 172.21.15.60:0/3013920435 learned_addr learned my addr 172.21.15.60:0/3013920435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.947+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a30198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.947+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a3010cc00 0x7f4a3019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.947+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112700 con 0x7f4a3010cc00 2024-07-24T22:04:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.947+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112580 con 0x7f4a3010a2d0 2024-07-24T22:04:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112880 con 0x7f4a301011c0 2024-07-24T22:04:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a2ffff640 1 -- 172.21.15.60:0/3013920435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 msgr2=0x7f4a30198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a30198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a2ffff640 1 -- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a3010cc00 msgr2=0x7f4a3019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a3010cc00 0x7f4a3019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.948+0000 7f4a2ffff640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3019eae0 con 0x7f4a3010a2d0 2024-07-24T22:04:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.949+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a30198f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a880 tx=0x7f4a1c032df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.950+0000 7f4a2dffb640 1 -- 172.21.15.60:0/3013920435 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c037040 con 0x7f4a3010a2d0 2024-07-24T22:04:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.950+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3019ed70 con 0x7f4a3010a2d0 2024-07-24T22:04:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.950+0000 7f4a2dffb640 1 -- 172.21.15.60:0/3013920435 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c041070 con 0x7f4a3010a2d0 2024-07-24T22:04:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.950+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a30102070 con 0x7f4a3010a2d0 2024-07-24T22:04:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.950+0000 7f4a2dffb640 1 -- 172.21.15.60:0/3013920435 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c03d8d0 con 0x7f4a3010a2d0 2024-07-24T22:04:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 Processor -- start 2024-07-24T22:04:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 -- start start 2024-07-24T22:04:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6808ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f6808f360 0x557f6808f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557f68002960 con 0x557f6808f360 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f67ff1e60 con 0x557f68093e90 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.955+0000 7f4a390be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f67ff5ea0 con 0x557f680947b0 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.956+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.956+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45420/0 (socket says 172.21.15.60:45420) 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.956+0000 7f4a352aa640 1 -- 172.21.15.60:0/2692419697 learned_addr learned my addr 172.21.15.60:0/2692419697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f6808f360 0x557f6808f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a352aa640 1 -- 172.21.15.60:0/2692419697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 msgr2=0x557f6808ee20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a352aa640 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6808ee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a352aa640 1 -- 172.21.15.60:0/2692419697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f6808f360 msgr2=0x557f6808f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a352aa640 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f6808f360 0x557f6808f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.957+0000 7f4a352aa640 1 -- 172.21.15.60:0/2692419697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f67f7b2d0 con 0x557f68093e90 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.958+0000 7f4a352aa640 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4a28030430 tx=0x7f4a280308f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abaad9640cc9b43 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.958+0000 7f4a2dffb640 1 -- 172.21.15.60:0/3013920435 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4a1c03da70 con 0x7f4a3010a2d0 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.958+0000 7f4a1affd640 1 -- 172.21.15.60:0/2692419697 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28004b20 con 0x557f68093e90 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.958+0000 7f4a1affd640 1 -- 172.21.15.60:0/2692419697 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28003e20 con 0x557f68093e90 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 -- 172.21.15.60:0/2692419697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 msgr2=0x557f68094270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4a28030430 tx=0x7f4a280308f0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 -- 172.21.15.60:0/2692419697 shutdown_connections 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f6808f360 0x557f6808f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6808ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 --2- 172.21.15.60:0/2692419697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f68094270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.959+0000 7f4a390be740 1 -- 172.21.15.60:0/2692419697 >> 172.21.15.60:0/2692419697 conn(0x557f68091cc0 msgr2=0x557f68092e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:30.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.962+0000 7f4a2dffb640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 0x7f4a200b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.963+0000 7f4a2dffb640 1 -- 172.21.15.60:0/3013920435 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f4a1c103c70 con 0x7f4a3010a2d0 2024-07-24T22:04:30.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.963+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 0x7f4a200b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.964+0000 7f4a390be740 1 -- 172.21.15.60:0/2692419697 shutdown_connections 2024-07-24T22:04:30.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.965+0000 7f4a390be740 1 -- 172.21.15.60:0/2692419697 wait complete. 2024-07-24T22:04:30.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.965+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 0x7f4a200b61c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x557f68093970 tx=0x7f4a2400a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:30.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.965+0000 7f4a390be740 1 -- start start 2024-07-24T22:04:30.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.965+0000 7f4a390be740 1 Processor -- start 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f68098320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f680988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557f67e55700 con 0x557f68093e90 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f67f945a0 con 0x557f6808f360 2024-07-24T22:04:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f68090060 con 0x557f680947b0 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a390be740 10 client.0 _ll_register_callbacks cb 0x557f67fb4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54822/0 (socket says 172.21.15.60:54822) 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.966+0000 7f4a352aa640 1 -- 172.21.15.60:0/696386419 learned_addr learned my addr 172.21.15.60:0/696386419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 -- 172.21.15.60:0/696386419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 msgr2=0x557f68098320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f68098320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 -- 172.21.15.60:0/696386419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 msgr2=0x557f680988a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f680988a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 -- 172.21.15.60:0/696386419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f67f87630 con 0x557f680947b0 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.967+0000 7f4a352aa640 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4a2803a040 tx=0x7f4a28004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851ef37ad18c6500 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.968+0000 7f4a0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:04:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.972+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28047020 con 0x557f680947b0 2024-07-24T22:04:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.972+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f4a28003a30 con 0x557f680947b0 2024-07-24T22:04:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.973+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a28004810 con 0x557f680947b0 2024-07-24T22:04:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.975+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4a2803c040 con 0x557f680947b0 2024-07-24T22:04:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.975+0000 7f4a390be740 10 client.28324 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.975+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557f67d52650 con 0x557f680947b0 2024-07-24T22:04:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.975+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28004810 con 0x557f680947b0 2024-07-24T22:04:30.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.976+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f4a2800b040 con 0x557f680947b0 2024-07-24T22:04:30.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 10 client.28324 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:30.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 10 client.28324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:30.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 20 client.28324 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:30.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f680a7ec0 0x557f680aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:30.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 4 client.28324 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:30.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.977+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f680048b0 con 0x557f680a7ec0 2024-07-24T22:04:30.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.983+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f680a7ec0 0x557f680aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:30.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.984+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f680a7ec0 0x557f680aa340 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d462d229f62b745 server_cookie=c5a705bb1a39b449 in_seq=0 out_seq=0 2024-07-24T22:04:30.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.984+0000 7f4a0ffff640 10 client.28324 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:30.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.984+0000 7f4a0ffff640 1 -- 172.21.15.60:0/696386419 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f680048b0 con 0x557f680a7ec0 2024-07-24T22:04:30.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a0ffff640 10 client.28324 handle_command_reply: tid=0 2024-07-24T22:04:30.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 client.28324 shutdown 2024-07-24T22:04:30.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 msgr2=0x557f6809a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:30.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4a2803a040 tx=0x7f4a28004e90 comp rx=0 tx=0).stop 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 shutdown_connections 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f680a7ec0 0x557f680aa340 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809a490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f680988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 --2- 172.21.15.60:0/696386419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f68098320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 >> 172.21.15.60:0/696386419 conn(0x557f68091cc0 msgr2=0x557f680977f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a0ffff640 0 client.28324 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.985+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 shutdown_connections 2024-07-24T22:04:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.986+0000 7f4a390be740 1 -- 172.21.15.60:0/696386419 wait complete. 2024-07-24T22:04:30.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:30.986+0000 7f4a390be740 20 client.28324 trim_cache size 0 max 16384 2024-07-24T22:04:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 Processor -- start 2024-07-24T22:04:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 -- start start 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f6809a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f6808d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301ae850 con 0x557f68093e90 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301ac060 con 0x557f6808f360 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a390be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301a9f60 con 0x557f680947b0 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a34aa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f6809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.020+0000 7f4a34aa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f6809a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45430/0 (socket says 172.21.15.60:45430) 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a34aa9640 1 -- 172.21.15.60:0/1776136950 learned_addr learned my addr 172.21.15.60:0/1776136950 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f6808d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 -- 172.21.15.60:0/1776136950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 msgr2=0x557f6809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 -- 172.21.15.60:0/1776136950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 msgr2=0x557f6809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f6809a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.021+0000 7f4a2ffff640 1 -- 172.21.15.60:0/1776136950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6808dfa0 con 0x557f68093e90 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.022+0000 7f4a2ffff640 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f6808d8c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c101410 tx=0x7f4a1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0e0b456c02288 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.022+0000 7f4a1a7fc640 1 -- 172.21.15.60:0/1776136950 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0ca040 con 0x557f68093e90 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.022+0000 7f4a1a7fc640 1 -- 172.21.15.60:0/1776136950 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c00b0b0 con 0x557f68093e90 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.023+0000 7f4a1a7fc640 1 -- 172.21.15.60:0/1776136950 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0c6060 con 0x557f68093e90 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.024+0000 7f4a390be740 1 -- 172.21.15.60:0/1776136950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 msgr2=0x557f6808d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.024+0000 7f4a390be740 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f6808d8c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c101410 tx=0x7f4a1c000bc0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.025+0000 7f4a390be740 1 -- 172.21.15.60:0/1776136950 shutdown_connections 2024-07-24T22:04:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.025+0000 7f4a390be740 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f680947b0 0x557f6809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.025+0000 7f4a390be740 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f68093e90 0x557f6808d8c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.025+0000 7f4a390be740 1 --2- 172.21.15.60:0/1776136950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f6808f360 0x557f6809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.025+0000 7f4a390be740 1 -- 172.21.15.60:0/1776136950 >> 172.21.15.60:0/1776136950 conn(0x557f68091cc0 msgr2=0x557f6809b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.030+0000 7f4a390be740 1 -- 172.21.15.60:0/1776136950 shutdown_connections 2024-07-24T22:04:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.030+0000 7f4a390be740 1 -- 172.21.15.60:0/1776136950 wait complete. 2024-07-24T22:04:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.031+0000 7f4a390be740 1 -- start start 2024-07-24T22:04:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.031+0000 7f4a390be740 1 Processor -- start 2024-07-24T22:04:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f6808f360 0x557f680a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f6808bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301aa520 con 0x557f680947b0 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301ab990 con 0x557f68093e90 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301aabf0 con 0x557f6808f360 2024-07-24T22:04:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a390be740 10 client.0 _ll_register_callbacks cb 0x7f4a28009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a352aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43798/0 (socket says 172.21.15.60:43798) 2024-07-24T22:04:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.032+0000 7f4a352aa640 1 -- 172.21.15.60:0/119389526 learned_addr learned my addr 172.21.15.60:0/119389526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 -- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f6808f360 msgr2=0x557f680a1a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f6808f360 0x557f680a1a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 -- 172.21.15.60:0/119389526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 msgr2=0x557f6808bc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:31.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f6808bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 -- 172.21.15.60:0/119389526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6808c820 con 0x557f680947b0 2024-07-24T22:04:31.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.033+0000 7f4a352aa640 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006010 tx=0x7f4a28009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a16cac9b477044 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.034+0000 7f4a0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:31.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.035+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28047070 con 0x557f680947b0 2024-07-24T22:04:31.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.035+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f4a28042a40 con 0x557f680947b0 2024-07-24T22:04:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.038+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c00b0b0 con 0x557f680947b0 2024-07-24T22:04:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.039+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4a28003a30 con 0x557f680947b0 2024-07-24T22:04:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.039+0000 7f4a390be740 10 client.19113 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.039+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557f68112aa0 con 0x557f680947b0 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.039+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28003bd0 con 0x557f680947b0 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.039+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f4a2805d030 con 0x557f680947b0 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 10 client.19113 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 20 client.19113 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f681131c0 0x557f681155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 4 client.19113 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:04:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.040+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4a1c00b0b0 con 0x557f681131c0 2024-07-24T22:04:31.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.043+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f681131c0 0x557f681155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a0ffff640 10 client.19113 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a34aa9640 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f681131c0 0x557f681155e0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad868e000391cb2f server_cookie=7409c3bf89ff5320 in_seq=0 out_seq=0 2024-07-24T22:04:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a0ffff640 1 -- 172.21.15.60:0/119389526 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f4a1c00b0b0 con 0x557f681131c0 2024-07-24T22:04:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a0ffff640 10 client.19113 handle_command_reply: tid=0 2024-07-24T22:04:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 client.19113 shutdown 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 msgr2=0x557f6808c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006010 tx=0x7f4a28009a10 comp rx=0 tx=0).stop 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 shutdown_connections 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x557f681131c0 0x557f681155e0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557f680947b0 0x557f6808c1b0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f68093e90 0x557f6808bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 --2- 172.21.15.60:0/119389526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f6808f360 0x557f680a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 >> 172.21.15.60:0/119389526 conn(0x557f68091cc0 msgr2=0x557f68090360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.053+0000 7f4a0ffff640 0 client.19113 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.058+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 shutdown_connections 2024-07-24T22:04:31.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.058+0000 7f4a390be740 1 -- 172.21.15.60:0/119389526 wait complete. 2024-07-24T22:04:31.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.059+0000 7f4a390be740 20 client.19113 trim_cache size 0 max 16384 2024-07-24T22:04:31.061 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:04:31.061 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:04:31.061 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:04:31.062 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 msgr2=0x7f4a200b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 0x7f4a200b61c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x557f68093970 tx=0x7f4a2400a4c0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 msgr2=0x7f4a30198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a30198f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a880 tx=0x7f4a1c032df0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 shutdown_connections 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4a200b3d00 0x7f4a200b61c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a3010cc00 0x7f4a3019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a3010a2d0 0x7f4a30198f70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 --2- 172.21.15.60:0/3013920435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a301011c0 0x7f4a30198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 >> 172.21.15.60:0/3013920435 conn(0x7f4a300778d0 msgr2=0x7f4a3010d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 shutdown_connections 2024-07-24T22:04:31.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:31.061+0000 7f4a36d0d640 1 -- 172.21.15.60:0/3013920435 wait complete. 2024-07-24T22:04:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.113+0000 7f023625e640 1 -- 172.21.15.60:0/970571451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0230108d00 msgr2=0x7f02301090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.113+0000 7f023625e640 1 --2- 172.21.15.60:0/970571451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0230108d00 0x7f02301090e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0220009900 tx=0x7f022002a440 comp rx=0 tx=0).stop 2024-07-24T22:04:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.115+0000 7f023625e640 1 -- 172.21.15.60:0/970571451 shutdown_connections 2024-07-24T22:04:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.115+0000 7f023625e640 1 --2- 172.21.15.60:0/970571451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010f080 0x7f0230115880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.115+0000 7f023625e640 1 --2- 172.21.15.60:0/970571451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f023010e700 0x7f023010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.115+0000 7f023625e640 1 --2- 172.21.15.60:0/970571451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0230108d00 0x7f02301090e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.115+0000 7f023625e640 1 -- 172.21.15.60:0/970571451 >> 172.21.15.60:0/970571451 conn(0x7f0230078330 msgr2=0x7f023007a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.117+0000 7f023625e640 1 -- 172.21.15.60:0/970571451 shutdown_connections 2024-07-24T22:04:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.118+0000 7f023625e640 1 -- 172.21.15.60:0/970571451 wait complete. 2024-07-24T22:04:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.118+0000 7f023625e640 1 Processor -- start 2024-07-24T22:04:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 -- start start 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f023010e700 0x7f023019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f023010f080 0x7f02301a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0230118e30 con 0x7f023010e700 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0230118cb0 con 0x7f0230108d00 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f023625e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0230118fb0 con 0x7f023010f080 2024-07-24T22:04:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45448/0 (socket says 172.21.15.60:45448) 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.119+0000 7f022f7fe640 1 -- 172.21.15.60:0/789589794 learned_addr learned my addr 172.21.15.60:0/789589794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.120+0000 7f022f7fe640 1 -- 172.21.15.60:0/789589794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f023010f080 msgr2=0x7f02301a4aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.120+0000 7f022f7fe640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f023010f080 0x7f02301a4aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.120+0000 7f022f7fe640 1 -- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f023010e700 msgr2=0x7f023019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.120+0000 7f022f7fe640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f023010e700 0x7f023019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.120+0000 7f022f7fe640 1 -- 172.21.15.60:0/789589794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301a5270 con 0x7f0230108d00 2024-07-24T22:04:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.124+0000 7f022f7fe640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f022002aa30 tx=0x7f0220030730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.126+0000 7f022cff9640 1 -- 172.21.15.60:0/789589794 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022003f020 con 0x7f0230108d00 2024-07-24T22:04:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.126+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a5500 con 0x7f0230108d00 2024-07-24T22:04:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.126+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023006c560 con 0x7f0230108d00 2024-07-24T22:04:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.126+0000 7f022cff9640 1 -- 172.21.15.60:0/789589794 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0220030950 con 0x7f0230108d00 2024-07-24T22:04:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.127+0000 7f022cff9640 1 -- 172.21.15.60:0/789589794 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220035070 con 0x7f0230108d00 2024-07-24T22:04:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.130+0000 7f022cff9640 1 -- 172.21.15.60:0/789589794 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0220042020 con 0x7f0230108d00 2024-07-24T22:04:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.133+0000 7f022cff9640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 0x7f02180b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.133+0000 7f022cff9640 1 -- 172.21.15.60:0/789589794 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0220105ac0 con 0x7f0230108d00 2024-07-24T22:04:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.134+0000 7f0238fb8740 1 Processor -- start 2024-07-24T22:04:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.134+0000 7f0238fb8740 1 -- start start 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.134+0000 7f022effd640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 0x7f02180b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56154b54e280 con 0x56154b5d88c0 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56154b53d780 con 0x56154b5e5170 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f0238fb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56154b5417c0 con 0x56154b5d91e0 2024-07-24T22:04:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45452/0 (socket says 172.21.15.60:45452) 2024-07-24T22:04:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022ffff640 1 -- 172.21.15.60:0/894851829 learned_addr learned my addr 172.21.15.60:0/894851829 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022f7fe640 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022effd640 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.135+0000 7f022effd640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 0x7f02180b6410 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f023007a920 tx=0x7f0224006c40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 -- 172.21.15.60:0/894851829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 msgr2=0x56154b5e4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 -- 172.21.15.60:0/894851829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 msgr2=0x56154b5e7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 -- 172.21.15.60:0/894851829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154b2a1790 con 0x56154b5d88c0 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f022f7fe640 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5d8ca0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0220103150 tx=0x7f02200cbe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ba3fc07e86109 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f0215ffb640 1 -- 172.21.15.60:0/894851829 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200cc610 con 0x56154b5d88c0 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f0215ffb640 1 -- 172.21.15.60:0/894851829 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02200d0c10 con 0x56154b5d88c0 2024-07-24T22:04:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.136+0000 7f0215ffb640 1 -- 172.21.15.60:0/894851829 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200ccad0 con 0x56154b5d88c0 2024-07-24T22:04:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 -- 172.21.15.60:0/894851829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 msgr2=0x56154b5d8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5d8ca0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0220103150 tx=0x7f02200cbe40 comp rx=0 tx=0).stop 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 -- 172.21.15.60:0/894851829 shutdown_connections 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.137+0000 7f0238fb8740 1 --2- 172.21.15.60:0/894851829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5d8ca0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.138+0000 7f0238fb8740 1 -- 172.21.15.60:0/894851829 >> 172.21.15.60:0/894851829 conn(0x56154b5d66f0 msgr2=0x56154b5d7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.138+0000 7f0238fb8740 1 -- 172.21.15.60:0/894851829 shutdown_connections 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.138+0000 7f0238fb8740 1 -- 172.21.15.60:0/894851829 wait complete. 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.138+0000 7f0238fb8740 1 -- start start 2024-07-24T22:04:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.139+0000 7f0238fb8740 1 Processor -- start 2024-07-24T22:04:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.139+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43836/0 (socket says 172.21.15.60:43836) 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022f7fe640 1 -- 172.21.15.60:0/1649078212 learned_addr learned my addr 172.21.15.60:0/1649078212 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56154b477ab0 con 0x56154b5d88c0 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56154b4c59e0 con 0x56154b5e5170 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56154b54e280 con 0x56154b5d91e0 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f0238fb8740 10 client.0 _ll_register_callbacks cb 0x56154b24da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022ffff640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 -- 172.21.15.60:0/1649078212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 msgr2=0x56154b5e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 -- 172.21.15.60:0/1649078212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 msgr2=0x56154b5e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.140+0000 7f022effd640 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154b31a620 con 0x56154b5d91e0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.141+0000 7f022effd640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e1d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02240094d0 tx=0x7f022400e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9e079b8d947e22 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.141+0000 7f020effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.141+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224026060 con 0x56154b5d91e0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.141+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56154b4ecce0 con 0x56154b5d91e0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.141+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0224021680 con 0x56154b5d91e0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.142+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f02240218c0 con 0x56154b5d91e0 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.142+0000 7f0238fb8740 10 client.28333 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.142+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56154b5f3290 con 0x56154b5d91e0 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.142+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0224013400 con 0x56154b5d91e0 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0224012070 con 0x56154b5d91e0 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 10 client.28333 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 10 client.28333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 20 client.28333 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b5f47c0 0x56154b5f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 4 client.28333 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56154b4ecce0 con 0x56154b5f47c0 2024-07-24T22:04:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.143+0000 7f022f7fe640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b5f47c0 0x56154b5f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.144+0000 7f022f7fe640 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b5f47c0 0x56154b5f6bd0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1c70e8e4034f98 server_cookie=33291cea53af1310 in_seq=0 out_seq=0 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.144+0000 7f020effd640 10 client.28333 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f020effd640 1 -- 172.21.15.60:0/1649078212 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56154b4ecce0 con 0x56154b5f47c0 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f020effd640 10 client.28333 handle_command_reply: tid=0 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 client.28333 shutdown 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 msgr2=0x56154b5e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e1d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02240094d0 tx=0x7f022400e000 comp rx=0 tx=0).stop 2024-07-24T22:04:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 shutdown_connections 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b5f47c0 0x56154b5f6bd0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5e1d80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 --2- 172.21.15.60:0/1649078212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 >> 172.21.15.60:0/1649078212 conn(0x56154b5d66f0 msgr2=0x56154b5dc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.145+0000 7f020effd640 0 client.28333 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.146+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 shutdown_connections 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.146+0000 7f0238fb8740 1 -- 172.21.15.60:0/1649078212 wait complete. 2024-07-24T22:04:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.146+0000 7f0238fb8740 20 client.28333 trim_cache size 0 max 16384 2024-07-24T22:04:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.175+0000 7f0238fb8740 1 Processor -- start 2024-07-24T22:04:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.175+0000 7f0238fb8740 1 -- start start 2024-07-24T22:04:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5d91e0 0x56154b5e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f021c0112d0 con 0x56154b5d88c0 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00eae0 con 0x56154b5d91e0 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f022ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f022ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54870/0 (socket says 172.21.15.60:54870) 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f022ffff640 1 -- 172.21.15.60:0/2988031880 learned_addr learned my addr 172.21.15.60:0/2988031880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.176+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00c950 con 0x56154b5e5170 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 -- 172.21.15.60:0/2988031880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5d91e0 msgr2=0x56154b5e2290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5d91e0 0x56154b5e2290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 -- 172.21.15.60:0/2988031880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 msgr2=0x56154b5e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 -- 172.21.15.60:0/2988031880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154b5d8fd0 con 0x56154b5e5170 2024-07-24T22:04:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.177+0000 7f022ffff640 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d70 tx=0x7f0228007720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0806a1fdb70218d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.180+0000 7f02157fa640 1 -- 172.21.15.60:0/2988031880 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228016480 con 0x56154b5e5170 2024-07-24T22:04:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.180+0000 7f02157fa640 1 -- 172.21.15.60:0/2988031880 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022801b070 con 0x56154b5e5170 2024-07-24T22:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.181+0000 7f02157fa640 1 -- 172.21.15.60:0/2988031880 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280159d0 con 0x56154b5e5170 2024-07-24T22:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.181+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 msgr2=0x56154b5fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.181+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0228009d70 tx=0x7f0228007720 comp rx=0 tx=0).stop 2024-07-24T22:04:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.182+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 shutdown_connections 2024-07-24T22:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.182+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5e5170 0x56154b5fc4d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.182+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5d91e0 0x56154b5e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.182+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2988031880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.182+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 >> 172.21.15.60:0/2988031880 conn(0x56154b5d66f0 msgr2=0x56154b5e0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.183+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 shutdown_connections 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.183+0000 7f0238fb8740 1 -- 172.21.15.60:0/2988031880 wait complete. 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.184+0000 7f0238fb8740 1 -- start start 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.184+0000 7f0238fb8740 1 Processor -- start 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.184+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.184+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f0238fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f0238fb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00cfa0 con 0x56154b5d88c0 2024-07-24T22:04:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f0238fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00e410 con 0x56154b5e5170 2024-07-24T22:04:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f0238fb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f021c00d670 con 0x56154b5d91e0 2024-07-24T22:04:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f0238fb8740 10 client.0 _ll_register_callbacks cb 0x7f022400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f022ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f022ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45492/0 (socket says 172.21.15.60:45492) 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f022ffff640 1 -- 172.21.15.60:0/2324331142 learned_addr learned my addr 172.21.15.60:0/2324331142 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f022effd640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.185+0000 7f022f7fe640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 -- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 msgr2=0x56154b5ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 -- 172.21.15.60:0/2324331142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 msgr2=0x56154b5ecce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5ecce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 -- 172.21.15.60:0/2324331142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154b5dec50 con 0x56154b5e5170 2024-07-24T22:04:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022f7fe640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5ecce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.186+0000 7f022ffff640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0228009890 tx=0x7f0228015910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb79d4c8091ca74e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.190+0000 7f020effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:04:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.190+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228025070 con 0x56154b5e5170 2024-07-24T22:04:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.192+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0228015c10 con 0x56154b5e5170 2024-07-24T22:04:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.193+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022801b070 con 0x56154b5e5170 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.194+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f022802f050 con 0x56154b5e5170 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.196+0000 7f0238fb8740 10 client.28466 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.196+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56154b65ee50 con 0x56154b5e5170 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022801b070 con 0x56154b5e5170 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f022802a600 con 0x56154b5e5170 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 10 client.28466 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 10 client.28466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 20 client.28466 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b65f5a0 0x56154b6619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 4 client.28466 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:04:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.197+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56154b5501d0 con 0x56154b65f5a0 2024-07-24T22:04:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.198+0000 7f022f7fe640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b65f5a0 0x56154b6619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.201+0000 7f022f7fe640 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b65f5a0 0x56154b6619c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ad60f786bf769d server_cookie=a0487d28ed0298ce in_seq=0 out_seq=0 2024-07-24T22:04:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.202+0000 7f020effd640 10 client.28466 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.262+0000 7f020effd640 1 -- 172.21.15.60:0/2324331142 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56154b5501d0 con 0x56154b65f5a0 2024-07-24T22:04:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.262+0000 7f020effd640 10 client.28466 handle_command_reply: tid=0 2024-07-24T22:04:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.262+0000 7f0238fb8740 1 client.28466 shutdown 2024-07-24T22:04:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.263+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 msgr2=0x56154b5de5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.263+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0228009890 tx=0x7f0228015910 comp rx=0 tx=0).stop 2024-07-24T22:04:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 shutdown_connections 2024-07-24T22:04:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56154b65f5a0 0x56154b6619c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56154b5e5170 0x56154b5de5b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56154b5d91e0 0x56154b5ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 --2- 172.21.15.60:0/2324331142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56154b5d88c0 0x56154b5ecce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 >> 172.21.15.60:0/2324331142 conn(0x56154b5d66f0 msgr2=0x56154b5e0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f020effd640 0 client.28466 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 shutdown_connections 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 1 -- 172.21.15.60:0/2324331142 wait complete. 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.264+0000 7f0238fb8740 20 client.28466 trim_cache size 0 max 16384 2024-07-24T22:04:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:04:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:04:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:04:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:04:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 msgr2=0x7f02180b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 0x7f02180b6410 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f023007a920 tx=0x7f0224006c40 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 msgr2=0x7f023019f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f022002aa30 tx=0x7f0220030730 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 shutdown_connections 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02180b3f50 0x7f02180b6410 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f023010f080 0x7f02301a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f023010e700 0x7f023019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 --2- 172.21.15.60:0/789589794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d00 0x7f023019f1d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 >> 172.21.15.60:0/789589794 conn(0x7f0230078330 msgr2=0x7f023010ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 shutdown_connections 2024-07-24T22:04:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:31.265+0000 7f023625e640 1 -- 172.21.15.60:0/789589794 wait complete. 2024-07-24T22:04:31.809 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:31 smithi060 ceph-mon[33000]: pgmap v938: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 736 KiB/s wr, 91 op/s 2024-07-24T22:04:31.858 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:04:31.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:04:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:31 smithi123 ceph-mon[37143]: pgmap v938: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 736 KiB/s wr, 91 op/s 2024-07-24T22:04:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:31 smithi086 ceph-mon[37128]: pgmap v938: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 736 KiB/s wr, 91 op/s 2024-07-24T22:04:32.362 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:33.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:33 smithi060 ceph-mon[33000]: pgmap v939: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-07-24T22:04:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:33 smithi123 ceph-mon[37143]: pgmap v939: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-07-24T22:04:34.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:33 smithi086 ceph-mon[37128]: pgmap v939: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-07-24T22:04:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 -- 172.21.15.60:0/3508032301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 msgr2=0x7f77e010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 --2- 172.21.15.60:0/3508032301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e010ef20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009be0 tx=0x7f77d802a6d0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 -- 172.21.15.60:0/3508032301 shutdown_connections 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 --2- 172.21.15.60:0/3508032301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e010ef20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 --2- 172.21.15.60:0/3508032301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e0101bc0 0x7f77e010c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 --2- 172.21.15.60:0/3508032301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77e01011c0 0x7f77e01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 -- 172.21.15.60:0/3508032301 >> 172.21.15.60:0/3508032301 conn(0x7f77e0076890 msgr2=0x7f77e0076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 -- 172.21.15.60:0/3508032301 shutdown_connections 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.378+0000 7f77e76fd640 1 -- 172.21.15.60:0/3508032301 wait complete. 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.379+0000 7f77e76fd640 1 Processor -- start 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.379+0000 7f77e76fd640 1 -- start start 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77e01011c0 0x7f77e006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e0101bc0 0x7f77e006cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f77e0112720 con 0x7f77e0101bc0 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01125a0 con 0x7f77e010ca80 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e76fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77e01128a0 con 0x7f77e01011c0 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e5c9a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e5c9a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:45500/0 (socket says 172.21.15.60:45500) 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.380+0000 7f77e5c9a640 1 -- 172.21.15.60:0/1146878354 learned_addr learned my addr 172.21.15.60:0/1146878354 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.383+0000 7f77e5c9a640 1 -- 172.21.15.60:0/1146878354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77e01011c0 msgr2=0x7f77e006c850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.383+0000 7f77e5c9a640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77e01011c0 0x7f77e006c850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.383+0000 7f77e5c9a640 1 -- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e0101bc0 msgr2=0x7f77e006cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.383+0000 7f77e5c9a640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e0101bc0 0x7f77e006cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.383+0000 7f77e5c9a640 1 -- 172.21.15.60:0/1146878354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00728b0 con 0x7f77e010ca80 2024-07-24T22:04:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.384+0000 7f77e5c9a640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77d8034040 tx=0x7f77d8032f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.384+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8036040 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.384+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e0072b40 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.384+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e00732a0 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.385+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d8030e20 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.385+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d803cdf0 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.386+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c40052c0 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.389+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f77d803c670 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.390+0000 7f77d67fc640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 0x7f77cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.390+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f77d8108ed0 con 0x7f77e010ca80 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.390+0000 7f77e5499640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 0x7f77cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.391+0000 7f77e5499640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 0x7f77cc0b6280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005fd0 tx=0x7f77dc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:34.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.396+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f77d80d0290 con 0x7f77e010ca80 2024-07-24T22:04:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.822+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c40055b0 con 0x7f77e010ca80 2024-07-24T22:04:34.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:04:34.826 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:04:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.824+0000 7f77d67fc640 1 -- 172.21.15.60:0/1146878354 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f77d80cfb90 con 0x7f77e010ca80 2024-07-24T22:04:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.827+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 msgr2=0x7f77cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.827+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 0x7f77cc0b6280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005fd0 tx=0x7f77dc009260 comp rx=0 tx=0).stop 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.827+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 msgr2=0x7f77e0072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.827+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77d8034040 tx=0x7f77d8032f70 comp rx=0 tx=0).stop 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 shutdown_connections 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f77cc0b3dc0 0x7f77cc0b6280 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e010ca80 0x7f77e0072130 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e0101bc0 0x7f77e006cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 --2- 172.21.15.60:0/1146878354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77e01011c0 0x7f77e006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 >> 172.21.15.60:0/1146878354 conn(0x7f77e0076890 msgr2=0x7f77e0104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 shutdown_connections 2024-07-24T22:04:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:34.828+0000 7f77e76fd640 1 -- 172.21.15.60:0/1146878354 wait complete. 2024-07-24T22:04:35.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:04:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:35 smithi123 ceph-mon[37143]: pgmap v940: 97 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 54 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 125 op/s 2024-07-24T22:04:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1146878354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:35 smithi086 ceph-mon[37128]: pgmap v940: 97 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 54 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 125 op/s 2024-07-24T22:04:36.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1146878354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:36.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:35 smithi060 ceph-mon[33000]: pgmap v940: 97 pgs: 12 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 54 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 125 op/s 2024-07-24T22:04:36.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1146878354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:37.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:37 smithi060 ceph-mon[33000]: pgmap v941: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-07-24T22:04:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:37 smithi123 ceph-mon[37143]: pgmap v941: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-07-24T22:04:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:37 smithi086 ceph-mon[37128]: pgmap v941: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 -- 172.21.15.60:0/3451695887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 msgr2=0x7f6d64103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 --2- 172.21.15.60:0/3451695887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64103a40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009900 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 -- 172.21.15.60:0/3451695887 shutdown_connections 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 --2- 172.21.15.60:0/3451695887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6410cac0 0x7f6d6410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 --2- 172.21.15.60:0/3451695887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d64103f80 0x7f6d6410c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 --2- 172.21.15.60:0/3451695887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64103a40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 -- 172.21.15.60:0/3451695887 >> 172.21.15.60:0/3451695887 conn(0x7f6d640fd0c0 msgr2=0x7f6d640ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 -- 172.21.15.60:0/3451695887 shutdown_connections 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.398+0000 7f6d6a849640 1 -- 172.21.15.60:0/3451695887 wait complete. 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.399+0000 7f6d6a849640 1 Processor -- start 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.399+0000 7f6d6a849640 1 -- start start 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.399+0000 7f6d6a849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.399+0000 7f6d6a849640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 0x7f6d64199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d6a849640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d6410cac0 0x7f6d6419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d6a849640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64112750 con 0x7f6d64103660 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d6a849640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641125d0 con 0x7f6d64103f80 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d6a849640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641128d0 con 0x7f6d6410cac0 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59960/0 (socket says 172.21.15.60:59960) 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 -- 172.21.15.60:0/1607696750 learned_addr learned my addr 172.21.15.60:0/1607696750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 0x7f6d64199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 -- 172.21.15.60:0/1607696750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d6410cac0 msgr2=0x7f6d6419e3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d6410cac0 0x7f6d6419e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 -- 172.21.15.60:0/1607696750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 msgr2=0x7f6d64199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 0x7f6d64199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 -- 172.21.15.60:0/1607696750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6419ebb0 con 0x7f6d64103660 2024-07-24T22:04:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 0x7f6d64199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.400+0000 7f6d63fff640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009850 tx=0x7f6d54030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.401+0000 7f6d617fa640 1 -- 172.21.15.60:0/1607696750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5403f070 con 0x7f6d64103660 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.401+0000 7f6d617fa640 1 -- 172.21.15.60:0/1607696750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d54004460 con 0x7f6d64103660 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.401+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6419ee40 con 0x7f6d64103660 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.401+0000 7f6d617fa640 1 -- 172.21.15.60:0/1607696750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54035030 con 0x7f6d64103660 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.401+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641a5560 con 0x7f6d64103660 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 Processor -- start 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 -- start start 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e06a70 0x565176e06eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e073f0 0x565176e10de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x565176d709b0 con 0x565176df7150 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565176d5feb0 con 0x565176e073f0 2024-07-24T22:04:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565176d63ef0 con 0x565176e06a70 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59966/0 (socket says 172.21.15.60:59966) 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.405+0000 7f6d68de6640 1 -- 172.21.15.60:0/1866429875 learned_addr learned my addr 172.21.15.60:0/1866429875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e073f0 0x565176e10de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d617fa640 1 -- 172.21.15.60:0/1607696750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f6d54042070 con 0x7f6d64103660 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 -- 172.21.15.60:0/1866429875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e06a70 msgr2=0x565176e06eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e06a70 0x565176e06eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 -- 172.21.15.60:0/1866429875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e073f0 msgr2=0x565176e10de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e073f0 0x565176e10de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 -- 172.21.15.60:0/1866429875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565176ce9320 con 0x565176df7150 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.406+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009db0 tx=0x7f6d5c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352a16cfe7f1ea2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.407+0000 7f6d4a7fc640 1 -- 172.21.15.60:0/1866429875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c004b20 con 0x565176df7150 2024-07-24T22:04:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.407+0000 7f6d4a7fc640 1 -- 172.21.15.60:0/1866429875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5c004cc0 con 0x565176df7150 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.407+0000 7f6d617fa640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 0x7f6d4c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d617fa640 1 -- 172.21.15.60:0/1607696750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f6d54104ab0 con 0x7f6d64103660 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1866429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 msgr2=0x565176df7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009db0 tx=0x7f6d5c00e720 comp rx=0 tx=0).stop 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1866429875 shutdown_connections 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e073f0 0x565176e10de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e06a70 0x565176e06eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1866429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176df7150 0x565176df7530 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.408+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1866429875 >> 172.21.15.60:0/1866429875 conn(0x565176df9670 msgr2=0x565176df9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.409+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1866429875 shutdown_connections 2024-07-24T22:04:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.410+0000 7f6d63fff640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 0x7f6d4c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.410+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1866429875 wait complete. 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.410+0000 7f6d63fff640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 0x7f6d4c0b6280 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6d540c7680 tx=0x7f6d5402a410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.410+0000 7f6d6cbfa740 1 -- start start 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 Processor -- start 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e10c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x565176bc3700 con 0x565176e0e6b0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565176d025f0 con 0x565176e06a70 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565176e114d0 con 0x565176e073f0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d6cbfa740 10 client.0 _ll_register_callbacks cb 0x565176d22960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e10c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.684 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54140/0 (socket says 172.21.15.60:54140) 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 -- 172.21.15.60:0/1545610038 learned_addr learned my addr 172.21.15.60:0/1545610038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 -- 172.21.15.60:0/1545610038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 msgr2=0x565176e10c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e10c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 -- 172.21.15.60:0/1545610038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 msgr2=0x565176e0eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.411+0000 7f6d637fe640 1 -- 172.21.15.60:0/1545610038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565176cf5680 con 0x565176e073f0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800bf90 tx=0x7f6d580128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae0b7f6e31a4846 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d580109c0 con 0x565176e073f0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565176d72900 con 0x565176e073f0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f6d58010b60 con 0x565176e073f0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d58010d60 con 0x565176e073f0 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d6cbfa740 10 client.28348 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565176ac0650 con 0x565176e073f0 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.412+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565176d72900 con 0x565176e073f0 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6d580233c0 con 0x565176e073f0 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 10 client.28348 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 10 client.28348 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 20 client.28348 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 4 client.28348 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565176d6e3a0 con 0x565176e1e710 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d437fe640 10 client.28348 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.413+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e20b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2616389be16499d0 server_cookie=a9c3278f6faa994e in_seq=0 out_seq=0 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d437fe640 1 -- 172.21.15.60:0/1545610038 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565176d6e3a0 con 0x565176e1e710 2024-07-24T22:04:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d437fe640 10 client.28348 handle_command_reply: tid=0 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 client.28348 shutdown 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 msgr2=0x565176e0e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800bf90 tx=0x7f6d580128e0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 shutdown_connections 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e20b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d437fe640 0 client.28348 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0e0d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1545610038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e10c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.414+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 >> 172.21.15.60:0/1545610038 conn(0x565176df9670 msgr2=0x565176df8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.415+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 shutdown_connections 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.415+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1545610038 wait complete. 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.415+0000 7f6d6cbfa740 20 client.28348 trim_cache size 0 max 16384 2024-07-24T22:04:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.438+0000 7f6d6cbfa740 1 Processor -- start 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 -- start start 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176df70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641aea80 con 0x565176e0e6b0 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641ac290 con 0x565176e06a70 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.439+0000 7f6d6cbfa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641aa100 con 0x565176e073f0 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.441+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.442+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59980/0 (socket says 172.21.15.60:59980) 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.442+0000 7f6d68de6640 1 -- 172.21.15.60:0/3186976514 learned_addr learned my addr 172.21.15.60:0/3186976514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.442+0000 7f6d68de6640 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176df70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 -- 172.21.15.60:0/3186976514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 msgr2=0x565176df70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176df70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 -- 172.21.15.60:0/3186976514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 msgr2=0x565176e0fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.443+0000 7f6d637fe640 1 -- 172.21.15.60:0/3186976514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565176e20e70 con 0x565176e073f0 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.444+0000 7f6d637fe640 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0f700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d58023a20 tx=0x7f6d5800b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d2c0909775ad8d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.444+0000 7f6d49ffb640 1 -- 172.21.15.60:0/3186976514 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58023ca0 con 0x565176e073f0 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.444+0000 7f6d49ffb640 1 -- 172.21.15.60:0/3186976514 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58023e70 con 0x565176e073f0 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.444+0000 7f6d49ffb640 1 -- 172.21.15.60:0/3186976514 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005470 con 0x565176e073f0 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.446+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/3186976514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 msgr2=0x565176e0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.447+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0f700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d58023a20 tx=0x7f6d5800b4e0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.449+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/3186976514 shutdown_connections 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.449+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e0e6b0 0x565176e0fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.449+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e073f0 0x565176e0f700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.449+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/3186976514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176df70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.449+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/3186976514 >> 172.21.15.60:0/3186976514 conn(0x565176df9670 msgr2=0x565176e05fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.451+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/3186976514 shutdown_connections 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.451+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/3186976514 wait complete. 2024-07-24T22:04:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.452+0000 7f6d6cbfa740 1 -- start start 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.452+0000 7f6d6cbfa740 1 Processor -- start 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e073f0 0x565176e16cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e0e6b0 0x565176e11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d68de6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44648/0 (socket says 172.21.15.60:44648) 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d68de6640 1 -- 172.21.15.60:0/1383261317 learned_addr learned my addr 172.21.15.60:0/1383261317 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641aa750 con 0x565176e073f0 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641abbc0 con 0x565176e06a70 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.453+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641aae20 con 0x565176e0e6b0 2024-07-24T22:04:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d6cbfa740 10 client.0 _ll_register_callbacks cb 0x7f6d5800b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e0e6b0 0x565176e11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 -- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e0e6b0 msgr2=0x565176e11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e0e6b0 0x565176e11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 -- 172.21.15.60:0/1383261317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e073f0 msgr2=0x565176e16cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e073f0 0x565176e16cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565176e11cd0 con 0x565176e06a70 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d68de6640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ec20 tx=0x7f6d5800bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80ab8174f83a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565176d6bae0 con 0x565176e06a70 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.454+0000 7f6d437fe640 1 -- 172.21.15.60:0/1383261317 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c004d60 con 0x565176e06a70 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.455+0000 7f6d437fe640 1 -- 172.21.15.60:0/1383261317 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f6d5c018990 con 0x565176e06a70 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.455+0000 7f6d437fe640 1 -- 172.21.15.60:0/1383261317 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c018bf0 con 0x565176e06a70 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.455+0000 7f6d6cbfa740 10 client.28490 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.455+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=46}) -- 0x565176e89b20 con 0x565176e06a70 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d437fe640 1 -- 172.21.15.60:0/1383261317 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f6d5c033040 con 0x565176e06a70 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 10 client.28490 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 10 client.28490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 20 client.28490 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e8bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 4 client.28490 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565176d6bae0 con 0x565176e1e710 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.456+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d637fe640 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e8bf00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9744962bd5432c server_cookie=dfba1df0e92270a4 in_seq=0 out_seq=0 2024-07-24T22:04:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d437fe640 10 client.28490 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d437fe640 1 -- 172.21.15.60:0/1383261317 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565176d6bae0 con 0x565176e1e710 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d437fe640 10 client.28490 handle_command_reply: tid=0 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d6cbfa740 1 client.28490 shutdown 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 msgr2=0x565176e167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.457+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ec20 tx=0x7f6d5800bb10 comp rx=0 tx=0).stop 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 shutdown_connections 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x565176e1e710 0x565176e8bf00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d437fe640 0 client.28490 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565176e0e6b0 0x565176e11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x565176e073f0 0x565176e16cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 --2- 172.21.15.60:0/1383261317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565176e06a70 0x565176e167b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 >> 172.21.15.60:0/1383261317 conn(0x565176df9670 msgr2=0x565176e0bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 shutdown_connections 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 1 -- 172.21.15.60:0/1383261317 wait complete. 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.458+0000 7f6d6cbfa740 20 client.28490 trim_cache size 0 max 16384 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.459+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 msgr2=0x7f6d4c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.459+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 0x7f6d4c0b6280 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6d540c7680 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.459+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 msgr2=0x7f6d64198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.459+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009850 tx=0x7f6d54030650 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 shutdown_connections 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f6d4c0b3dc0 0x7f6d4c0b6280 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d6410cac0 0x7f6d6419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d64103f80 0x7f6d64199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 --2- 172.21.15.60:0/1607696750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d64103660 0x7f6d64198b00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 >> 172.21.15.60:0/1607696750 conn(0x7f6d640fd0c0 msgr2=0x7f6d64104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 shutdown_connections 2024-07-24T22:04:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:38.460+0000 7f6d6a849640 1 -- 172.21.15.60:0/1607696750 wait complete. 2024-07-24T22:04:39.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:04:39.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:39 smithi123 ceph-mon[37143]: pgmap v942: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-07-24T22:04:40.003 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:40.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:39 smithi086 ceph-mon[37128]: pgmap v942: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-07-24T22:04:40.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:39 smithi060 ceph-mon[33000]: pgmap v942: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-07-24T22:04:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 -- 172.21.15.60:0/400644510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 msgr2=0x7f5f64107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 --2- 172.21.15.60:0/400644510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64107310 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d910 tx=0x7f5f58032990 comp rx=0 tx=0).stop 2024-07-24T22:04:41.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:41 smithi060 ceph-mon[33000]: pgmap v943: 97 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-07-24T22:04:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 -- 172.21.15.60:0/400644510 shutdown_connections 2024-07-24T22:04:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 --2- 172.21.15.60:0/400644510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64107850 0x7f5f6410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 --2- 172.21.15.60:0/400644510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64107310 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 --2- 172.21.15.60:0/400644510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f641014c0 0x7f5f641018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 -- 172.21.15.60:0/400644510 >> 172.21.15.60:0/400644510 conn(0x7f5f640fcf10 msgr2=0x7f5f640ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.801+0000 7f5f693dd640 1 -- 172.21.15.60:0/400644510 shutdown_connections 2024-07-24T22:04:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.802+0000 7f5f693dd640 1 -- 172.21.15.60:0/400644510 wait complete. 2024-07-24T22:04:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.802+0000 7f5f693dd640 1 Processor -- start 2024-07-24T22:04:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.802+0000 7f5f693dd640 1 -- start start 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 0x7f5f64196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f64107850 0x7f5f6419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64110450 con 0x7f5f64106ed0 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641102d0 con 0x7f5f64107850 2024-07-24T22:04:41.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f693dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641105d0 con 0x7f5f641014c0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59994/0 (socket says 172.21.15.60:59994) 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190616751 learned_addr learned my addr 172.21.15.60:0/3190616751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f68bdc640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f64107850 0x7f5f6419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.803+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 0x7f5f64196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190616751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 msgr2=0x7f5f64196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 0x7f5f64196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190616751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f64107850 msgr2=0x7f5f6419c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f64107850 0x7f5f6419c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190616751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6419c910 con 0x7f5f64106ed0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 0x7f5f64196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.804+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f5800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.805+0000 7f5f617fa640 1 -- 172.21.15.60:0/3190616751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005460 con 0x7f5f64106ed0 2024-07-24T22:04:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.805+0000 7f5f617fa640 1 -- 172.21.15.60:0/3190616751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58046070 con 0x7f5f64106ed0 2024-07-24T22:04:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.805+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6419cba0 con 0x7f5f64106ed0 2024-07-24T22:04:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.805+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f641a32c0 con 0x7f5f64106ed0 2024-07-24T22:04:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.805+0000 7f5f617fa640 1 -- 172.21.15.60:0/3190616751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5804b760 con 0x7f5f64106ed0 2024-07-24T22:04:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.808+0000 7f5f6c9f0740 1 Processor -- start 2024-07-24T22:04:41.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 -- start start 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c87bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c973bf29b0 con 0x55c973c7c170 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c973be1eb0 con 0x55c973c7b850 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.809+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c973be5ef0 con 0x55c973c88100 2024-07-24T22:04:41.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f68bdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f617fa640 1 -- 172.21.15.60:0/3190616751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5f58036070 con 0x7f5f64106ed0 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f68bdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f68bdc640 1 -- 172.21.15.60:0/3279241416 learned_addr learned my addr 172.21.15.60:0/3279241416 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c87bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 -- 172.21.15.60:0/3279241416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 msgr2=0x55c973c8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 -- 172.21.15.60:0/3279241416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 msgr2=0x55c973c7bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 -- 172.21.15.60:0/3279241416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c973b6b320 con 0x55c973c7c170 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f68bdc640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.810+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c87bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009d70 tx=0x7f5f5400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1425bdd5230d34e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.811+0000 7f5f617fa640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 0x7f5f4c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.812+0000 7f5f617fa640 1 -- 172.21.15.60:0/3190616751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5f5810cf10 con 0x7f5f64106ed0 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.812+0000 7f5f68bdc640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 0x7f5f4c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f4a7fc640 1 -- 172.21.15.60:0/3279241416 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017070 con 0x55c973c7c170 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f4a7fc640 1 -- 172.21.15.60:0/3279241416 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54013920 con 0x55c973c7c170 2024-07-24T22:04:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f4a7fc640 1 -- 172.21.15.60:0/3279241416 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f540127f0 con 0x55c973c7c170 2024-07-24T22:04:41.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3279241416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 msgr2=0x55c973c87bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c87bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009d70 tx=0x7f5f5400e5a0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.813+0000 7f5f68bdc640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 0x7f5f4c0b63f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c540 tx=0x7f5f5c00f040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3279241416 shutdown_connections 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c87bc0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3279241416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7b850 0x55c973c7bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3279241416 >> 172.21.15.60:0/3279241416 conn(0x55c973c79680 msgr2=0x55c973c7a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3279241416 shutdown_connections 2024-07-24T22:04:41.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.814+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3279241416 wait complete. 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.815+0000 7f5f6c9f0740 1 -- start start 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.815+0000 7f5f6c9f0740 1 Processor -- start 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 0x55c973c85aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c973a45700 con 0x55c973c88100 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c973b845f0 con 0x55c973c7c170 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c973c8ad80 con 0x55c973c7b850 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f6c9f0740 10 client.0 _ll_register_callbacks cb 0x55c973ba4960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f68bdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 0x55c973c85aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f68bdc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 0x55c973c85aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60016/0 (socket says 172.21.15.60:60016) 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f68bdc640 1 -- 172.21.15.60:0/3190332144 learned_addr learned my addr 172.21.15.60:0/3190332144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.816+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190332144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 msgr2=0x55c973c85020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c85020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190332144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 msgr2=0x55c973c85aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 0x55c973c85aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 -- 172.21.15.60:0/3190332144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c973b77680 con 0x55c973c7c170 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c85560 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f5810a6b0 tx=0x7f5f581108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac0c2dbff72aa87 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580d3940 con 0x55c973c7c170 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.817+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54013920 con 0x55c973c7c170 2024-07-24T22:04:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.818+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5f5800d500 con 0x55c973c7c170 2024-07-24T22:04:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.818+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5f580cf270 con 0x55c973c7c170 2024-07-24T22:04:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.818+0000 7f5f6c9f0740 10 client.28496 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.818+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c973942650 con 0x55c973c7c170 2024-07-24T22:04:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.818+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54013920 con 0x55c973c7c170 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5f58053030 con 0x55c973c7c170 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 10 client.28496 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 10 client.28496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 20 client.28496 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973c97da0 0x55c973c9a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 4 client.28496 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.819+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c973bf4900 con 0x55c973c97da0 2024-07-24T22:04:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.825+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973c97da0 0x55c973c9a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.826+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973c97da0 0x55c973c9a200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=328e7386024bade4 server_cookie=dee057abb9b9d0d3 in_seq=0 out_seq=0 2024-07-24T22:04:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.826+0000 7f5f437fe640 10 client.28496 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:04:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.827+0000 7f5f437fe640 1 -- 172.21.15.60:0/3190332144 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c973bf4900 con 0x55c973c97da0 2024-07-24T22:04:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.827+0000 7f5f437fe640 10 client.28496 handle_command_reply: tid=0 2024-07-24T22:04:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.827+0000 7f5f6c9f0740 1 client.28496 shutdown 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 msgr2=0x55c973c85560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c85560 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f5810a6b0 tx=0x7f5f581108d0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 shutdown_connections 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973c97da0 0x55c973c9a200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c88100 0x55c973c85aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c85560 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3190332144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c85020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 >> 172.21.15.60:0/3190332144 conn(0x55c973c79680 msgr2=0x55c973c7f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.828+0000 7f5f437fe640 0 client.28496 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.829+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 shutdown_connections 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.829+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3190332144 wait complete. 2024-07-24T22:04:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.829+0000 7f5f6c9f0740 20 client.28496 trim_cache size 0 max 16384 2024-07-24T22:04:41.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.849+0000 7f5f6c9f0740 1 Processor -- start 2024-07-24T22:04:41.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.849+0000 7f5f6c9f0740 1 -- start start 2024-07-24T22:04:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c86d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641ae7c0 con 0x55c973c7b850 2024-07-24T22:04:41.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641abfd0 con 0x55c973c7c170 2024-07-24T22:04:41.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f6c9f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641a9e40 con 0x55c973c88100 2024-07-24T22:04:41.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60030/0 (socket says 172.21.15.60:60030) 2024-07-24T22:04:41.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.850+0000 7f5f63fff640 1 -- 172.21.15.60:0/3554201170 learned_addr learned my addr 172.21.15.60:0/3554201170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c86d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f63fff640 1 -- 172.21.15.60:0/3554201170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 msgr2=0x55c973c9fb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c9fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f63fff640 1 -- 172.21.15.60:0/3554201170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 msgr2=0x55c973c86d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c86d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f63fff640 1 -- 172.21.15.60:0/3554201170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c973c83520 con 0x55c973c7b850 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.851+0000 7f5f637fe640 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c86d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.852+0000 7f5f63fff640 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009b00 tx=0x7f5f54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e715d1ecf81f50 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.856+0000 7f5f49ffb640 1 -- 172.21.15.60:0/3554201170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017040 con 0x55c973c7b850 2024-07-24T22:04:41.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.856+0000 7f5f49ffb640 1 -- 172.21.15.60:0/3554201170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54013d50 con 0x55c973c7b850 2024-07-24T22:04:41.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.856+0000 7f5f49ffb640 1 -- 172.21.15.60:0/3554201170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f540062b0 con 0x55c973c7b850 2024-07-24T22:04:41.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.857+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3554201170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 msgr2=0x55c973c7f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.857+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009b00 tx=0x7f5f54008040 comp rx=0 tx=0).stop 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3554201170 shutdown_connections 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c88100 0x55c973c9fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c7c170 0x55c973c86d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/3554201170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7b850 0x55c973c7f9a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3554201170 >> 172.21.15.60:0/3554201170 conn(0x55c973c79680 msgr2=0x55c973c86650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3554201170 shutdown_connections 2024-07-24T22:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.858+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/3554201170 wait complete. 2024-07-24T22:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.859+0000 7f5f6c9f0740 1 -- start start 2024-07-24T22:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.859+0000 7f5f6c9f0740 1 Processor -- start 2024-07-24T22:04:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c81b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c88100 0x55c973c82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f63fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f63fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:54208/0 (socket says 172.21.15.60:54208) 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f63fff640 1 -- 172.21.15.60:0/2012188106 learned_addr learned my addr 172.21.15.60:0/2012188106 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641aa490 con 0x55c973c7c170 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f637fe640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c81b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641ab900 con 0x55c973c88100 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.860+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641aab60 con 0x55c973c7b850 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f6c9f0740 10 client.0 _ll_register_callbacks cb 0x7f5f58005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 -- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c88100 msgr2=0x55c973c82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c88100 0x55c973c82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 -- 172.21.15.60:0/2012188106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 msgr2=0x55c973c81b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c81b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c973c82680 con 0x55c973c7b850 2024-07-24T22:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54013d50 con 0x55c973c7b850 2024-07-24T22:04:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f63fff640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009810 tx=0x7f5f54013ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d949eccda38f4cec server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.861+0000 7f5f437fe640 10 client.28369 ms_handle_connect on v2:172.21.15.123:3300/0 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.862+0000 7f5f437fe640 1 -- 172.21.15.60:0/2012188106 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54046020 con 0x55c973c7b850 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.862+0000 7f5f437fe640 1 -- 172.21.15.60:0/2012188106 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5f54005dd0 con 0x55c973c7b850 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.862+0000 7f5f437fe640 1 -- 172.21.15.60:0/2012188106 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5f54017050 con 0x55c973c7b850 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.862+0000 7f5f6c9f0740 10 client.28369 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.862+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55c973c8ecc0 con 0x55c973c7b850 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f437fe640 1 -- 172.21.15.60:0/2012188106 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5f54006050 con 0x55c973c7b850 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 10 client.28369 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 10 client.28369 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 20 client.28369 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973d02ae0 0x55c973d04ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 4 client.28369 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:04:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.863+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c973bf03a0 con 0x55c973d02ae0 2024-07-24T22:04:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.864+0000 7f5f637fe640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973d02ae0 0x55c973d04ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.865+0000 7f5f637fe640 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973d02ae0 0x55c973d04ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb3c040e55706b88 server_cookie=9f04547eae7fd85e in_seq=0 out_seq=0 2024-07-24T22:04:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.865+0000 7f5f437fe640 10 client.28369 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:04:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f437fe640 1 -- 172.21.15.60:0/2012188106 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c973bf03a0 con 0x55c973d02ae0 2024-07-24T22:04:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f437fe640 10 client.28369 handle_command_reply: tid=0 2024-07-24T22:04:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f6c9f0740 1 client.28369 shutdown 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 msgr2=0x55c973c90490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009810 tx=0x7f5f54013ad0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.866+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 shutdown_connections 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55c973d02ae0 0x55c973d04ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c973c88100 0x55c973c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f437fe640 0 client.28369 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:04:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c973c7c170 0x55c973c81b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 --2- 172.21.15.60:0/2012188106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c973c7b850 0x55c973c90490 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 >> 172.21.15.60:0/2012188106 conn(0x55c973c79680 msgr2=0x55c973c86e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 shutdown_connections 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 1 -- 172.21.15.60:0/2012188106 wait complete. 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.867+0000 7f5f6c9f0740 20 client.28369 trim_cache size 0 max 16384 2024-07-24T22:04:41.870 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:04:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 msgr2=0x7f5f4c0b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 0x7f5f4c0b63f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c540 tx=0x7f5f5c00f040 comp rx=0 tx=0).stop 2024-07-24T22:04:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 msgr2=0x7f5f64196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f5800d340 comp rx=0 tx=0).stop 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 shutdown_connections 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5f4c0b3f30 0x7f5f4c0b63f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f64107850 0x7f5f6419c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f64106ed0 0x7f5f64196da0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 --2- 172.21.15.60:0/3190616751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f641014c0 0x7f5f64196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 >> 172.21.15.60:0/3190616751 conn(0x7f5f640fcf10 msgr2=0x7f5f640ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 shutdown_connections 2024-07-24T22:04:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:41.868+0000 7f5f693dd640 1 -- 172.21.15.60:0/3190616751 wait complete. 2024-07-24T22:04:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:41 smithi123 ceph-mon[37143]: pgmap v943: 97 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-07-24T22:04:42.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:41 smithi086 ceph-mon[37128]: pgmap v943: 97 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 86 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-07-24T22:04:43.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:04:43.702 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:43.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:43 smithi060 ceph-mon[33000]: pgmap v944: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-07-24T22:04:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:43 smithi123 ceph-mon[37143]: pgmap v944: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-07-24T22:04:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:43 smithi086 ceph-mon[37128]: pgmap v944: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-07-24T22:04:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 -- 172.21.15.60:0/3769495750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c106fc0 msgr2=0x7f5c9c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/3769495750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c107400 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009900 tx=0x7f5c8402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 -- 172.21.15.60:0/3769495750 shutdown_connections 2024-07-24T22:04:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/3769495750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c107940 0x7f5c9c10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/3769495750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c107400 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/3769495750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 -- 172.21.15.60:0/3769495750 >> 172.21.15.60:0/3769495750 conn(0x7f5c9c0fcfc0 msgr2=0x7f5c9c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 -- 172.21.15.60:0/3769495750 shutdown_connections 2024-07-24T22:04:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.363+0000 7f5ca17a2640 1 -- 172.21.15.60:0/3769495750 wait complete. 2024-07-24T22:04:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 Processor -- start 2024-07-24T22:04:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 -- start start 2024-07-24T22:04:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c111560 con 0x7f5c9c107940 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1113e0 con 0x7f5c9c1015c0 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5ca17a2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1116e0 con 0x7f5c9c106fc0 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60052/0 (socket says 172.21.15.60:60052) 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 -- 172.21.15.60:0/934246042 learned_addr learned my addr 172.21.15.60:0/934246042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9affd640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c93fff640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 -- 172.21.15.60:0/934246042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c106fc0 msgr2=0x7f5c9c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 -- 172.21.15.60:0/934246042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 msgr2=0x7f5c9c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9b7fe640 1 -- 172.21.15.60:0/934246042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c19ec80 con 0x7f5c9c107940 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.364+0000 7f5c9affd640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400efd0 tx=0x7f5c9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5c98ff9640 1 -- 172.21.15.60:0/934246042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94003b70 con 0x7f5c9c107940 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5c98ff9640 1 -- 172.21.15.60:0/934246042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c94003d10 con 0x7f5c9c107940 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c19ef70 con 0x7f5c9c107940 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5c98ff9640 1 -- 172.21.15.60:0/934246042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940108a0 con 0x7f5c9c107940 2024-07-24T22:04:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.365+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a5630 con 0x7f5c9c107940 2024-07-24T22:04:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.367+0000 7f5ca45d4740 1 Processor -- start 2024-07-24T22:04:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c98ff9640 1 -- 172.21.15.60:0/934246042 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5c94004020 con 0x7f5c9c107940 2024-07-24T22:04:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 -- start start 2024-07-24T22:04:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e887972ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88797ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e8878e8280 con 0x55e88797f170 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8878d7780 con 0x55e8879728c0 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8878db7c0 con 0x55e8879731e0 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60060/0 (socket says 172.21.15.60:60060) 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c93fff640 1 -- 172.21.15.60:0/1909691831 learned_addr learned my addr 172.21.15.60:0/1909691831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c93fff640 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e887972ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.368+0000 7f5c9affd640 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88797ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 -- 172.21.15.60:0/1909691831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 msgr2=0x55e88797ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88797ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 -- 172.21.15.60:0/1909691831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 msgr2=0x55e887972ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e887972ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 -- 172.21.15.60:0/1909691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88763b790 con 0x55e88797f170 2024-07-24T22:04:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c93fff640 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5c8402a930 tx=0x7f5c84005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfb0e5498e96c04 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c98ff9640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 0x7f5c880b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c9affd640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 0x7f5c880b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c917fa640 1 -- 172.21.15.60:0/1909691831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84035070 con 0x55e88797f170 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c98ff9640 1 -- 172.21.15.60:0/934246042 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5c940e4490 con 0x7f5c9c107940 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c917fa640 1 -- 172.21.15.60:0/1909691831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c840035f0 con 0x55e88797f170 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.369+0000 7f5c917fa640 1 -- 172.21.15.60:0/1909691831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840044e0 con 0x55e88797f170 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- 172.21.15.60:0/1909691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 msgr2=0x55e887981560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5c9affd640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 0x7f5c880b6200 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55e887975a40 tx=0x7f5c8c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5c8402a930 tx=0x7f5c84005b50 comp rx=0 tx=0).stop 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- 172.21.15.60:0/1909691831 shutdown_connections 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e88797f170 0x55e887981560 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88797ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/1909691831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e887972ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- 172.21.15.60:0/1909691831 >> 172.21.15.60:0/1909691831 conn(0x55e8879706f0 msgr2=0x55e887971890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- 172.21.15.60:0/1909691831 shutdown_connections 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- 172.21.15.60:0/1909691831 wait complete. 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 -- start start 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.370+0000 7f5ca45d4740 1 Processor -- start 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879728c0 0x55e88797e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88797c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55e887811ab0 con 0x55e8879731e0 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e88785f9e0 con 0x55e88797f170 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e887981df0 con 0x55e8879728c0 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 10 client.0 _ll_register_callbacks cb 0x55e8875e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60070/0 (socket says 172.21.15.60:60070) 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 -- 172.21.15.60:0/3602411195 learned_addr learned my addr 172.21.15.60:0/3602411195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88797c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879728c0 0x55e88797e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 -- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879728c0 msgr2=0x55e88797e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879728c0 0x55e88797e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 -- 172.21.15.60:0/3602411195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 msgr2=0x55e88797c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88797c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 -- 172.21.15.60:0/3602411195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8876b4620 con 0x55e8879731e0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5c840319a0 tx=0x7f5c840319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c453db59ad5a14 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c827fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84035040 con 0x55e8879731e0 2024-07-24T22:04:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5c84003fc0 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e887886ce0 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.371+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5c84058050 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 10 client.19173 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e88798d540 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e887886ce0 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5c840035f0 con 0x55e8879731e0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 10 client.19173 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 10 client.19173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 20 client.19173 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e88798e890 0x55e887990c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 4 client.19173 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8878ea1d0 con 0x55e88798e890 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.372+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e88798e890 0x55e887990c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e88798e890 0x55e887990c80 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0c0ad5c30922f5 server_cookie=7577deab4b99e0d0 in_seq=0 out_seq=0 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5c827fc640 10 client.19173 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5c827fc640 1 -- 172.21.15.60:0/3602411195 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8878ea1d0 con 0x55e88798e890 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5c827fc640 10 client.19173 handle_command_reply: tid=0 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5ca45d4740 1 client.19173 shutdown 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 msgr2=0x55e88797bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5c840319a0 tx=0x7f5c840319d0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.373+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 shutdown_connections 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e88798e890 0x55e887990c80 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88797c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e88797bd00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3602411195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879728c0 0x55e88797e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5c827fc640 0 client.19173 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 >> 172.21.15.60:0/3602411195 conn(0x55e8879706f0 msgr2=0x55e887976290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 shutdown_connections 2024-07-24T22:04:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3602411195 wait complete. 2024-07-24T22:04:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.374+0000 7f5ca45d4740 20 client.19173 trim_cache size 0 max 16384 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.390+0000 7f5ca45d4740 1 Processor -- start 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.390+0000 7f5ca45d4740 1 -- start start 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e88797c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88797f170 0x55e887978ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1b0a60 con 0x55e8879731e0 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1ae270 con 0x55e8879728c0 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1ac0e0 con 0x55e88797f170 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e88797c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60074/0 (socket says 172.21.15.60:60074) 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 -- 172.21.15.60:0/2980132685 learned_addr learned my addr 172.21.15.60:0/2980132685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88797f170 0x55e887978ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 -- 172.21.15.60:0/2980132685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88797f170 msgr2=0x55e887978ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88797f170 0x55e887978ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 -- 172.21.15.60:0/2980132685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 msgr2=0x55e88797c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e88797c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c93fff640 1 -- 172.21.15.60:0/2980132685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e887973010 con 0x55e8879731e0 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.391+0000 7f5c9affd640 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e88797c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.392+0000 7f5c93fff640 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5c84033000 tx=0x7f5c84005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d905068acdd07447 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.392+0000 7f5c90ff9640 1 -- 172.21.15.60:0/2980132685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840035f0 con 0x55e8879731e0 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.392+0000 7f5c90ff9640 1 -- 172.21.15.60:0/2980132685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c84009520 con 0x55e8879731e0 2024-07-24T22:04:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.392+0000 7f5c90ff9640 1 -- 172.21.15.60:0/2980132685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840042f0 con 0x55e8879731e0 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- 172.21.15.60:0/2980132685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 msgr2=0x55e8879772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5c84033000 tx=0x7f5c84005b50 comp rx=0 tx=0).stop 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- 172.21.15.60:0/2980132685 shutdown_connections 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e88797f170 0x55e887978ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879731e0 0x55e8879772f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/2980132685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8879728c0 0x55e88797c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- 172.21.15.60:0/2980132685 >> 172.21.15.60:0/2980132685 conn(0x55e8879706f0 msgr2=0x55e887976be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- 172.21.15.60:0/2980132685 shutdown_connections 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- 172.21.15.60:0/2980132685 wait complete. 2024-07-24T22:04:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 -- start start 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 Processor -- start 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.393+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88798ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1ac730 con 0x55e8879728c0 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1adba0 con 0x55e88797f170 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1ace00 con 0x55e8879731e0 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5ca45d4740 10 client.0 _ll_register_callbacks cb 0x7f5c84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60090/0 (socket says 172.21.15.60:60090) 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 -- 172.21.15.60:0/3315047556 learned_addr learned my addr 172.21.15.60:0/3315047556 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:44746/0 (socket says 172.21.15.60:44746) 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88798ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 -- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 msgr2=0x55e88798ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88798ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 -- 172.21.15.60:0/3315047556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 msgr2=0x55e88798f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 -- 172.21.15.60:0/3315047556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88798fa80 con 0x55e8879728c0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9b7fe640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c9affd640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c012ea0 tx=0x7f5c8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d1fb071829db47 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c827fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.394+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c005020 con 0x55e8879728c0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f5c8c0051c0 con 0x55e8879728c0 2024-07-24T22:04:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c84009520 con 0x55e8879728c0 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c0053c0 con 0x55e8879728c0 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 10 client.19179 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e8879fc730 con 0x55e8879728c0 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c84009520 con 0x55e8879728c0 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f5c8c02b860 con 0x55e8879728c0 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 10 client.19179 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 10 client.19179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 20 client.19179 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8879fcec0 0x55e8879ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 4 client.19179 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:04:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8878e5c70 con 0x55e8879fcec0 2024-07-24T22:04:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.395+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8879fcec0 0x55e8879ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.396+0000 7f5c93fff640 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8879fcec0 0x55e8879ff2e0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a68791ab82c3ff5 server_cookie=635d207ee08dce3b in_seq=0 out_seq=0 2024-07-24T22:04:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.396+0000 7f5c827fc640 10 client.19179 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5c827fc640 1 -- 172.21.15.60:0/3315047556 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8878e5c70 con 0x55e8879fcec0 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5c827fc640 10 client.19179 handle_command_reply: tid=0 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 client.19179 shutdown 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 msgr2=0x55e8879943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c012ea0 tx=0x7f5c8c017000 comp rx=0 tx=0).stop 2024-07-24T22:04:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 shutdown_connections 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55e8879fcec0 0x55e8879ff2e0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e88797f170 0x55e88798f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8879731e0 0x55e88798ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5c827fc640 0 client.19179 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 --2- 172.21.15.60:0/3315047556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e8879728c0 0x55e8879943f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 >> 172.21.15.60:0/3315047556 conn(0x55e8879706f0 msgr2=0x55e887975a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 shutdown_connections 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 1 -- 172.21.15.60:0/3315047556 wait complete. 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.406+0000 7f5ca45d4740 20 client.19179 trim_cache size 0 max 16384 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:04:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:04:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:04:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:04:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 msgr2=0x7f5c880b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 0x7f5c880b6200 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55e887975a40 tx=0x7f5c8c005f90 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 msgr2=0x7f5c9c19e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400efd0 tx=0x7f5c9400c6a0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 shutdown_connections 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5c880b3d40 0x7f5c880b6200 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c9c107940 0x7f5c9c19e4b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c106fc0 0x7f5c9c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 --2- 172.21.15.60:0/934246042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c1015c0 0x7f5c9c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 >> 172.21.15.60:0/934246042 conn(0x7f5c9c0fcfc0 msgr2=0x7f5c9c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 shutdown_connections 2024-07-24T22:04:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:44.407+0000 7f5ca17a2640 1 -- 172.21.15.60:0/934246042 wait complete. 2024-07-24T22:04:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:45 smithi060 ceph-mon[33000]: pgmap v945: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.827+0000 7f0bea00d640 1 -- 172.21.15.60:0/688210357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be41011e0 msgr2=0x7f0be41015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.827+0000 7f0bea00d640 1 --2- 172.21.15.60:0/688210357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be41011e0 0x7f0be41015c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009900 tx=0x7f0bd402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 -- 172.21.15.60:0/688210357 shutdown_connections 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 --2- 172.21.15.60:0/688210357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 0x7f0be410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 --2- 172.21.15.60:0/688210357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be410a2f0 0x7f0be410c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 --2- 172.21.15.60:0/688210357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be41011e0 0x7f0be41015c0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 -- 172.21.15.60:0/688210357 >> 172.21.15.60:0/688210357 conn(0x7f0be4076940 msgr2=0x7f0be4076d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 -- 172.21.15.60:0/688210357 shutdown_connections 2024-07-24T22:04:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 -- 172.21.15.60:0/688210357 wait complete. 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 Processor -- start 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.828+0000 7f0bea00d640 1 -- start start 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be41011e0 0x7f0be41021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 0x7f0be4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4112730 con 0x7f0be410a2f0 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41125b0 con 0x7f0be41011e0 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41128b0 con 0x7f0be410cc20 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44300/0 (socket says 172.21.15.60:44300) 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 -- 172.21.15.60:0/1634057311 learned_addr learned my addr 172.21.15.60:0/1634057311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 0x7f0be4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be37fe640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be41011e0 0x7f0be41021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 -- 172.21.15.60:0/1634057311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 msgr2=0x7f0be4102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 0x7f0be4102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 -- 172.21.15.60:0/1634057311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be41011e0 msgr2=0x7f0be41021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be41011e0 0x7f0be41021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 -- 172.21.15.60:0/1634057311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be41a7060 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9f0 tx=0x7f0bd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be0ff9640 1 -- 172.21.15.60:0/1634057311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800cd80 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be0ff9640 1 -- 172.21.15.60:0/1634057311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd80108e0 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be41a7350 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.829+0000 7f0be0ff9640 1 -- 172.21.15.60:0/1634057311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8018980 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.830+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be41a7890 con 0x7f0be410a2f0 2024-07-24T22:04:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 Processor -- start 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 -- start start 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be0ff9640 1 -- 172.21.15.60:0/1634057311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0bd8010430 con 0x7f0be410a2f0 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 0x5572a0616ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 0x5572a0622a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a0622f80 0x5572a0625370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5572a058d9b0 con 0x5572a0616ff0 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572a057ceb0 con 0x5572a06166d0 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0bec3be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572a0580ef0 con 0x5572a0622f80 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 0x5572a0622a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 0x5572a0616ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 0x5572a0622a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44314/0 (socket says 172.21.15.60:44314) 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be37fe640 1 -- 172.21.15.60:0/1695823807 learned_addr learned my addr 172.21.15.60:0/1695823807 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be3fff640 1 -- 172.21.15.60:0/1695823807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a0622f80 msgr2=0x5572a0625370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a0622f80 0x5572a0625370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be3fff640 1 -- 172.21.15.60:0/1695823807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 msgr2=0x5572a0622a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.833+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 0x5572a0622a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0be3fff640 1 -- 172.21.15.60:0/1695823807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a0506320 con 0x5572a06166d0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 0x5572a0616ab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005fd0 tx=0x7f0bdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc9a09f7d7cce1c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0bc9ffb640 1 -- 172.21.15.60:0/1695823807 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc004d20 con 0x5572a06166d0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0be0ff9640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 0x7f0bcc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0bc9ffb640 1 -- 172.21.15.60:0/1695823807 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc004030 con 0x5572a06166d0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0bc9ffb640 1 -- 172.21.15.60:0/1695823807 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc005220 con 0x5572a06166d0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0be37fe640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 0x7f0bcc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.834+0000 7f0be0ff9640 1 -- 172.21.15.60:0/1634057311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0bd80e4370 con 0x7f0be410a2f0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- 172.21.15.60:0/1695823807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 msgr2=0x5572a0616ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0be37fe640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 0x7f0bcc0b6200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005c00 tx=0x7f0bd4005b90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 0x5572a0616ab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005fd0 tx=0x7f0bdc011810 comp rx=0 tx=0).stop 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- 172.21.15.60:0/1695823807 shutdown_connections 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a0622f80 0x5572a0625370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0616ff0 0x5572a0622a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1695823807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a06166d0 0x5572a0616ab0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- 172.21.15.60:0/1695823807 >> 172.21.15.60:0/1695823807 conn(0x5572a0614500 msgr2=0x5572a06156a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- 172.21.15.60:0/1695823807 shutdown_connections 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- 172.21.15.60:0/1695823807 wait complete. 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 -- start start 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.835+0000 7f0bec3be740 1 Processor -- start 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a06204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5572a03e0700 con 0x5572a0622f80 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572a051f5f0 con 0x5572a0616ff0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572a0625c00 con 0x5572a06166d0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bec3be740 10 client.0 _ll_register_callbacks cb 0x5572a053f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44328/0 (socket says 172.21.15.60:44328) 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 -- 172.21.15.60:0/2735453828 learned_addr learned my addr 172.21.15.60:0/2735453828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a06204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be37fe640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 -- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 msgr2=0x5572a061ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 -- 172.21.15.60:0/2735453828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 msgr2=0x5572a06204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a06204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 -- 172.21.15.60:0/2735453828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a0512680 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0045e0 tx=0x7f0bdc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a33fc300d4a1f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.836+0000 7f0bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc027020 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0bdc003d30 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bdc004030 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc032050 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bec3be740 10 client.19191 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5572a02dd650 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.837+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc004030 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0bdc018070 con 0x5572a0622f80 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 10 client.19191 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 20 client.19191 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a0634dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 4 client.19191 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572a058f900 con 0x5572a06329d0 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0be37fe640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a0634dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0be37fe640 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a0634dc0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=814b053ec363e39 server_cookie=f312a704cedfa1ea in_seq=0 out_seq=0 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.838+0000 7f0bc2ffd640 10 client.19191 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2735453828 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572a058f900 con 0x5572a06329d0 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bc2ffd640 10 client.19191 handle_command_reply: tid=0 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 client.19191 shutdown 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 msgr2=0x5572a0620a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0045e0 tx=0x7f0bdc004610 comp rx=0 tx=0).stop 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 shutdown_connections 2024-07-24T22:04:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a0634dc0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bc2ffd640 0 client.19191 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0620a00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a06204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2735453828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 >> 172.21.15.60:0/2735453828 conn(0x5572a0614500 msgr2=0x5572a061a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 shutdown_connections 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 1 -- 172.21.15.60:0/2735453828 wait complete. 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.839+0000 7f0bec3be740 20 client.19191 trim_cache size 0 max 16384 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 Processor -- start 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 -- start start 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a063a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41b2dc0 con 0x5572a0622f80 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41b05d0 con 0x5572a0616ff0 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0bec3be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41ae440 con 0x5572a06166d0 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a063a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44338/0 (socket says 172.21.15.60:44338) 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.855+0000 7f0be3fff640 1 -- 172.21.15.60:0/1627777761 learned_addr learned my addr 172.21.15.60:0/1627777761 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be37fe640 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 -- 172.21.15.60:0/1627777761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 msgr2=0x5572a061fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 -- 172.21.15.60:0/1627777761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 msgr2=0x5572a063a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a063a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 -- 172.21.15.60:0/1627777761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a061ad60 con 0x5572a0622f80 2024-07-24T22:04:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a063a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0be3fff640 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009270 tx=0x7f0bdc004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5797beebf8c397d server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0bc97fa640 1 -- 172.21.15.60:0/1627777761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc027070 con 0x5572a0622f80 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0bc97fa640 1 -- 172.21.15.60:0/1627777761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc00fe80 con 0x5572a0622f80 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.856+0000 7f0bc97fa640 1 -- 172.21.15.60:0/1627777761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01e040 con 0x5572a0622f80 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- 172.21.15.60:0/1627777761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 msgr2=0x5572a063aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009270 tx=0x7f0bdc004930 comp rx=0 tx=0).stop 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- 172.21.15.60:0/1627777761 shutdown_connections 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a063aaf0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a063a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 --2- 172.21.15.60:0/1627777761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a061fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- 172.21.15.60:0/1627777761 >> 172.21.15.60:0/1627777761 conn(0x5572a0614500 msgr2=0x5572a0613570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- 172.21.15.60:0/1627777761 shutdown_connections 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- 172.21.15.60:0/1627777761 wait complete. 2024-07-24T22:04:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 -- start start 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.857+0000 7f0bec3be740 1 Processor -- start 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a062b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0626960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41aea90 con 0x5572a0622f80 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41aff00 con 0x5572a0616ff0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41af160 con 0x5572a06166d0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bec3be740 10 client.0 _ll_register_callbacks cb 0x7f0bdc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a062b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0626960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a062b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:46932/0 (socket says 172.21.15.60:46932) 2024-07-24T22:04:45.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52318/0 (socket says 172.21.15.60:52318) 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be37fe640 1 -- 172.21.15.60:0/2685763847 learned_addr learned my addr 172.21.15.60:0/2685763847 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 -- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 msgr2=0x5572a062b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a062b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 -- 172.21.15.60:0/2685763847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 msgr2=0x5572a062b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 -- 172.21.15.60:0/2685763847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a0627190 con 0x5572a0622f80 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be2ffd640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0be3fff640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0626960 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00f760 tx=0x7f0bdc00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef68578b98f97888 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.858+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc03b070 con 0x5572a0622f80 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8609+0+0 (secure 0 0 0) 0x7f0bdc003e40 con 0x5572a0622f80 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572a058b3a0 con 0x5572a0622f80 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc047050 con 0x5572a0622f80 2024-07-24T22:04:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 10 client.19197 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5572a069d290 con 0x5572a0622f80 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572a058b3a0 con 0x5572a0622f80 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0bdc018040 con 0x5572a0622f80 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 10 client.19197 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 10 client.19197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 20 client.19197 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a069f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 4 client.19197 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572a0588ae0 con 0x5572a06329d0 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.859+0000 7f0be37fe640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a069f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bc2ffd640 10 client.19197 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0be37fe640 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a069f940 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c629f21a3f06a545 server_cookie=c964272c7702776e in_seq=0 out_seq=0 2024-07-24T22:04:45.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bc2ffd640 1 -- 172.21.15.60:0/2685763847 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572a0588ae0 con 0x5572a06329d0 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bc2ffd640 10 client.19197 handle_command_reply: tid=0 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 client.19197 shutdown 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 msgr2=0x5572a0626960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0626960 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00f760 tx=0x7f0bdc00fbe0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 shutdown_connections 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5572a06329d0 0x5572a069f940 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572a0622f80 0x5572a0626960 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572a0616ff0 0x5572a062b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 --2- 172.21.15.60:0/2685763847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572a06166d0 0x5572a062b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 >> 172.21.15.60:0/2685763847 conn(0x5572a0614500 msgr2=0x5572a0619810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bc2ffd640 0 client.19197 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 shutdown_connections 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 1 -- 172.21.15.60:0/2685763847 wait complete. 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.860+0000 7f0bec3be740 20 client.19197 trim_cache size 0 max 16384 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 msgr2=0x7f0bcc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 0x7f0bcc0b6200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005c00 tx=0x7f0bd4005b90 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 msgr2=0x7f0be41026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9f0 tx=0x7f0bd800eeb0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 shutdown_connections 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0bcc0b3d40 0x7f0bcc0b6200 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0be410cc20 0x7f0be4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0be410a2f0 0x7f0be41026f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 --2- 172.21.15.60:0/1634057311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be41011e0 0x7f0be41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 >> 172.21.15.60:0/1634057311 conn(0x7f0be4076940 msgr2=0x7f0be40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 shutdown_connections 2024-07-24T22:04:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:45.861+0000 7f0bea00d640 1 -- 172.21.15.60:0/1634057311 wait complete. 2024-07-24T22:04:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:45 smithi123 ceph-mon[37143]: pgmap v945: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-07-24T22:04:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:45 smithi086 ceph-mon[37128]: pgmap v945: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-07-24T22:04:47.761 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-24T22:04:47.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-24T22:04:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:47 smithi123 ceph-mon[37143]: pgmap v946: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-07-24T22:04:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:47 smithi086 ceph-mon[37128]: pgmap v946: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-07-24T22:04:48.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:47 smithi060 ceph-mon[33000]: pgmap v946: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-07-24T22:04:48.111 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:49 smithi060 ceph-mon[33000]: osdmap e114: 12 total, 12 up, 12 in 2024-07-24T22:04:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:49 smithi060 ceph-mon[33000]: pgmap v948: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-07-24T22:04:49.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:49 smithi086 ceph-mon[37128]: osdmap e114: 12 total, 12 up, 12 in 2024-07-24T22:04:49.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:49 smithi086 ceph-mon[37128]: pgmap v948: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-07-24T22:04:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:49 smithi123 ceph-mon[37143]: osdmap e114: 12 total, 12 up, 12 in 2024-07-24T22:04:49.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:49 smithi123 ceph-mon[37143]: pgmap v948: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 -- 172.21.15.60:0/3170265432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 msgr2=0x7f18e41074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 --2- 172.21.15.60:0/3170265432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41074a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009900 tx=0x7f18d402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 -- 172.21.15.60:0/3170265432 shutdown_connections 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 --2- 172.21.15.60:0/3170265432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 --2- 172.21.15.60:0/3170265432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41074a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 --2- 172.21.15.60:0/3170265432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 0x7f18e4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 -- 172.21.15.60:0/3170265432 >> 172.21.15.60:0/3170265432 conn(0x7f18e40fd0c0 msgr2=0x7f18e40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 -- 172.21.15.60:0/3170265432 shutdown_connections 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.473+0000 7f18ebae0640 1 -- 172.21.15.60:0/3170265432 wait complete. 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 Processor -- start 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 -- start start 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 0x7f18e4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4111600 con 0x7f18e4107060 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4111480 con 0x7f18e41079e0 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ebae0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4111780 con 0x7f18e4101650 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44384/0 (socket says 172.21.15.60:44384) 2024-07-24T22:04:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 -- 172.21.15.60:0/1838367057 learned_addr learned my addr 172.21.15.60:0/1838367057 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e987c640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 0x7f18e4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 -- 172.21.15.60:0/1838367057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 msgr2=0x7f18e4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 0x7f18e4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 -- 172.21.15.60:0/1838367057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 msgr2=0x7f18e419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18e907b640 1 -- 172.21.15.60:0/1838367057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e419ed20 con 0x7f18e4107060 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.474+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18e907b640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005940 tx=0x7f18d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18daffd640 1 -- 172.21.15.60:0/1838367057 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d403f030 con 0x7f18e4107060 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e419efb0 con 0x7f18e4107060 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18daffd640 1 -- 172.21.15.60:0/1838367057 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d4003aa0 con 0x7f18e4107060 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18daffd640 1 -- 172.21.15.60:0/1838367057 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4035070 con 0x7f18e4107060 2024-07-24T22:04:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.475+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a56d0 con 0x7f18e4107060 2024-07-24T22:04:50.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18daffd640 1 -- 172.21.15.60:0/1838367057 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f18d4042030 con 0x7f18e4107060 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 Processor -- start 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 -- start start 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5335290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5337bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558cd52a09f0 con 0x558cd5328f20 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558cd528fef0 con 0x558cd53357d0 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.478+0000 7f18ede91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558cd5293f30 con 0x558cd5329840 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18ea07d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18ea07d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44392/0 (socket says 172.21.15.60:44392) 2024-07-24T22:04:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18ea07d640 1 -- 172.21.15.60:0/2918911159 learned_addr learned my addr 172.21.15.60:0/2918911159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e907b640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5337bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5335290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18daffd640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 0x7f18d00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 -- 172.21.15.60:0/2918911159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 msgr2=0x558cd5337bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5337bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 -- 172.21.15.60:0/2918911159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 msgr2=0x558cd5329300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 -- 172.21.15.60:0/2918911159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd4ff0930 con 0x558cd5329840 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18ea07d640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18daffd640 1 -- 172.21.15.60:0/1838367057 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f18d4104660 con 0x7f18e4107060 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 0x7f18d00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18e987c640 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5335290 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004630 tx=0x7f18e000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b81011f36dd794e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.479+0000 7f18cffff640 1 -- 172.21.15.60:0/2918911159 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0003bd0 con 0x558cd5329840 2024-07-24T22:04:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18cffff640 1 -- 172.21.15.60:0/2918911159 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e0015070 con 0x558cd5329840 2024-07-24T22:04:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18cffff640 1 -- 172.21.15.60:0/2918911159 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0004c70 con 0x558cd5329840 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 -- 172.21.15.60:0/2918911159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 msgr2=0x558cd5335290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5335290 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004630 tx=0x7f18e000bb20 comp rx=0 tx=0).stop 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 0x7f18d00b6200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009fd0 tx=0x7f18dc004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 -- 172.21.15.60:0/2918911159 shutdown_connections 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5337bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5335290 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 --2- 172.21.15.60:0/2918911159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd5329300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.480+0000 7f18ede91740 1 -- 172.21.15.60:0/2918911159 >> 172.21.15.60:0/2918911159 conn(0x558cd5326d50 msgr2=0x558cd5327ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.481+0000 7f18ede91740 1 -- 172.21.15.60:0/2918911159 shutdown_connections 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.481+0000 7f18ede91740 1 -- 172.21.15.60:0/2918911159 wait complete. 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.481+0000 7f18ede91740 1 -- start start 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.481+0000 7f18ede91740 1 Processor -- start 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5332c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5333170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558cd50f3890 con 0x558cd5328f20 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558cd5232390 con 0x558cd53357d0 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558cd5338450 con 0x558cd5329840 2024-07-24T22:04:50.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 10 client.0 _ll_register_callbacks cb 0x558cd523c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44396/0 (socket says 172.21.15.60:44396) 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 -- 172.21.15.60:0/1949286429 learned_addr learned my addr 172.21.15.60:0/1949286429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5333170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e907b640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5332c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 -- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 msgr2=0x558cd5332c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5332c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 -- 172.21.15.60:0/1949286429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 msgr2=0x558cd5333170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5333170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 -- 172.21.15.60:0/1949286429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd522cd60 con 0x558cd5328f20 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ea07d640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5333170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18e987c640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005cd0 tx=0x7f18e000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f52ba54634b2d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0003de0 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f18e0011440 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.482+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0015070 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f18e0011660 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 10 client.19209 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558cd5344320 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e0015070 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f18e0025990 con 0x558cd5328f20 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 10 client.19209 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 20 client.19209 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd5345720 0x558cd5347b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 4 client.19209 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cd52a2940 con 0x558cd5345720 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.483+0000 7f18e907b640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd5345720 0x558cd5347b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.484+0000 7f18e907b640 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd5345720 0x558cd5347b10 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d48bacf4ea9e72 server_cookie=2ea53785872c103c in_seq=0 out_seq=0 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.484+0000 7f18cd7fa640 10 client.19209 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.484+0000 7f18cd7fa640 1 -- 172.21.15.60:0/1949286429 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cd52a2940 con 0x558cd5345720 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.484+0000 7f18cd7fa640 10 client.19209 handle_command_reply: tid=0 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 client.19209 shutdown 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 msgr2=0x558cd53326f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005cd0 tx=0x7f18e000bfa0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 shutdown_connections 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd5345720 0x558cd5347b10 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5333170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5332c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 --2- 172.21.15.60:0/1949286429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd53326f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18cd7fa640 0 client.19209 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 >> 172.21.15.60:0/1949286429 conn(0x558cd5326d50 msgr2=0x558cd532c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 shutdown_connections 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 1 -- 172.21.15.60:0/1949286429 wait complete. 2024-07-24T22:04:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.485+0000 7f18ede91740 20 client.19209 trim_cache size 0 max 16384 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 Processor -- start 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 -- start start 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 0x558cd5334430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd53357d0 0x558cd534d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41b0af0 con 0x558cd5329840 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ae300 con 0x558cd5328f20 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ede91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ac170 con 0x558cd53357d0 2024-07-24T22:04:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18e907b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18e907b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44400/0 (socket says 172.21.15.60:44400) 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18e907b640 1 -- 172.21.15.60:0/3557255666 learned_addr learned my addr 172.21.15.60:0/3557255666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18e987c640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 0x558cd5334430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.504+0000 7f18ea07d640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd53357d0 0x558cd534d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 -- 172.21.15.60:0/3557255666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd53357d0 msgr2=0x558cd534d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd53357d0 0x558cd534d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 -- 172.21.15.60:0/3557255666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 msgr2=0x558cd534d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 -- 172.21.15.60:0/3557255666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd5329670 con 0x558cd5328f20 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e907b640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18e987c640 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 0x558cd5334430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004580 tx=0x7f18e000b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408e8c4b9b01c17 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18cf7fe640 1 -- 172.21.15.60:0/3557255666 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0003ea0 con 0x558cd5328f20 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18cf7fe640 1 -- 172.21.15.60:0/3557255666 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e0015070 con 0x558cd5328f20 2024-07-24T22:04:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.505+0000 7f18cf7fe640 1 -- 172.21.15.60:0/3557255666 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0005630 con 0x558cd5328f20 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- 172.21.15.60:0/3557255666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 msgr2=0x558cd5334430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 0x558cd5334430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f18e0004580 tx=0x7f18e000b4a0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- 172.21.15.60:0/3557255666 shutdown_connections 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd53357d0 0x558cd534d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5329840 0x558cd534d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 --2- 172.21.15.60:0/3557255666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd5328f20 0x558cd5334430 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- 172.21.15.60:0/3557255666 >> 172.21.15.60:0/3557255666 conn(0x558cd5326d50 msgr2=0x558cd5333d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- 172.21.15.60:0/3557255666 shutdown_connections 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- 172.21.15.60:0/3557255666 wait complete. 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 -- start start 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.506+0000 7f18ede91740 1 Processor -- start 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5339050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5339590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ac7c0 con 0x558cd5328f20 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41adc30 con 0x558cd53357d0 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18e41ace90 con 0x558cd5329840 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ede91740 10 client.0 _ll_register_callbacks cb 0x7f18e00044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44408/0 (socket says 172.21.15.60:44408) 2024-07-24T22:04:50.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 -- 172.21.15.60:0/2896526493 learned_addr learned my addr 172.21.15.60:0/2896526493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e907b640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5339050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18ea07d640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5339590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 -- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 msgr2=0x558cd5339050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5339050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 -- 172.21.15.60:0/2896526493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 msgr2=0x558cd5339590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5339590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 -- 172.21.15.60:0/2896526493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd5339bd0 con 0x558cd5328f20 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e907b640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5339050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18e987c640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f18e000bb60 tx=0x7f18e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fc1519c1ab264b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.507+0000 7f18cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0044060 con 0x558cd5328f20 2024-07-24T22:04:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f18e0005390 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0015070 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f18e0011800 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 10 client.19215 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558cd53b0160 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e0015070 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f18e0004630 con 0x558cd5328f20 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 10 client.19215 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 10 client.19215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 20 client.19215 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd53b07b0 0x558cd53b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 4 client.19215 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.508+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558cd529e3e0 con 0x558cd53b07b0 2024-07-24T22:04:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18e907b640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd53b07b0 0x558cd53b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18cd7fa640 10 client.19215 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18e907b640 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd53b07b0 0x558cd53b2bd0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff6e0a152536be0 server_cookie=cfbec7d3ad6a6338 in_seq=0 out_seq=0 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18cd7fa640 1 -- 172.21.15.60:0/2896526493 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558cd529e3e0 con 0x558cd53b07b0 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18cd7fa640 10 client.19215 handle_command_reply: tid=0 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.509+0000 7f18ede91740 1 client.19215 shutdown 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 msgr2=0x558cd533db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f18e000bb60 tx=0x7f18e0005800 comp rx=0 tx=0).stop 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 shutdown_connections 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x558cd53b07b0 0x558cd53b2bd0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558cd53357d0 0x558cd5339590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558cd5329840 0x558cd5339050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 --2- 172.21.15.60:0/2896526493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd5328f20 0x558cd533db80 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18cd7fa640 0 client.19215 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 >> 172.21.15.60:0/2896526493 conn(0x558cd5326d50 msgr2=0x558cd5325e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 shutdown_connections 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 1 -- 172.21.15.60:0/2896526493 wait complete. 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ede91740 20 client.19215 trim_cache size 0 max 16384 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "22962021-c0ae-4d68-940f-e9f650868ef2", 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-07-24T22:04:50.517 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 msgr2=0x7f18d00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 0x7f18d00b6200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009fd0 tx=0x7f18dc004a40 comp rx=0 tx=0).stop 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 msgr2=0x7f18e41991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.510+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005940 tx=0x7f18d4004270 comp rx=0 tx=0).stop 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 shutdown_connections 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18d00b3d40 0x7f18d00b6200 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e41079e0 0x7f18e419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18e4107060 0x7f18e41991b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 --2- 172.21.15.60:0/1838367057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18e4101650 0x7f18e4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 >> 172.21.15.60:0/1838367057 conn(0x7f18e40fd0c0 msgr2=0x7f18e406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 shutdown_connections 2024-07-24T22:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:50.511+0000 7f18ebae0640 1 -- 172.21.15.60:0/1838367057 wait complete. 2024-07-24T22:04:51.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:04:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: pgmap v949: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 313 op/s 2024-07-24T22:04:51.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: scrub queued for path: / 2024-07-24T22:04:51.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:51.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:04:51.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: scrub summary: active paths [/] 2024-07-24T22:04:51.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:51 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: pgmap v949: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 313 op/s 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: scrub queued for path: / 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: scrub summary: active paths [/] 2024-07-24T22:04:51.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:51 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: pgmap v949: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 313 op/s 2024-07-24T22:04:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: scrub queued for path: / 2024-07-24T22:04:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:04:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: scrub summary: active paths [/] 2024-07-24T22:04:51.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:51 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:52.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:52 smithi060 ceph-mon[33000]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:52.798 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-07-24T22:04:52.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-24T22:04:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:52 smithi123 ceph-mon[37143]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:53.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:52 smithi086 ceph-mon[37128]: scrub summary: idle+waiting paths [/] 2024-07-24T22:04:53.118 DEBUG:teuthology.parallel:result is None 2024-07-24T22:04:53.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-07-24T22:04:53.154 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-07-24T22:04:53.155 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-07-24T22:04:53.165 INFO:tasks.quiescer:joining Quiescers 2024-07-24T22:04:53.165 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-07-24T22:04:53.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:04:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 -- 172.21.15.60:0/44022740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 msgr2=0x7f0298101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 --2- 172.21.15.60:0/44022740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298101a30 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0288009ca0 tx=0x7f028802a8c0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 -- 172.21.15.60:0/44022740 shutdown_connections 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 --2- 172.21.15.60:0/44022740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 --2- 172.21.15.60:0/44022740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 0x7f02981074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 --2- 172.21.15.60:0/44022740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298101a30 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 -- 172.21.15.60:0/44022740 >> 172.21.15.60:0/44022740 conn(0x7f02980fd0c0 msgr2=0x7f02980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 -- 172.21.15.60:0/44022740 shutdown_connections 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.515+0000 7f029cb07640 1 -- 172.21.15.60:0/44022740 wait complete. 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 Processor -- start 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 -- start start 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 0x7f0298199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02981115f0 con 0x7f0298101650 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0298111470 con 0x7f02981079e0 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029cb07640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0298111770 con 0x7f0298107060 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44424/0 (socket says 172.21.15.60:44424) 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 -- 172.21.15.60:0/231877219 learned_addr learned my addr 172.21.15.60:0/231877219 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f0295d9b640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 0x7f0298199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f0296d9d640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 -- 172.21.15.60:0/231877219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 msgr2=0x7f0298199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 0x7f0298199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.516+0000 7f029659c640 1 -- 172.21.15.60:0/231877219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 msgr2=0x7f029819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f029659c640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f029659c640 1 -- 172.21.15.60:0/231877219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02981a4e20 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f0296d9d640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f029659c640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0288006010 tx=0x7f02880319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f02877fe640 1 -- 172.21.15.60:0/231877219 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288003bf0 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02981a50b0 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f02877fe640 1 -- 172.21.15.60:0/231877219 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0288003d90 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f02877fe640 1 -- 172.21.15.60:0/231877219 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028802fc20 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.517+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02981a5640 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.519+0000 7f02877fe640 1 -- 172.21.15.60:0/231877219 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f02880046a0 con 0x7f0298101650 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 Processor -- start 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 -- start start 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 0x563e7f987a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f997e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563e7f8fe990 con 0x563e7f998370 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e7f8ede90 con 0x563e7f9876b0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029eeb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563e7f8f1ed0 con 0x563e7f987fd0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f0296d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 0x563e7f987a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44426/0 (socket says 172.21.15.60:44426) 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f0295d9b640 1 -- 172.21.15.60:0/537429062 learned_addr learned my addr 172.21.15.60:0/537429062 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f0296d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 0x563e7f987a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52404/0 (socket says 172.21.15.60:52404) 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.520+0000 7f029659c640 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f997e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f02877fe640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 0x7f02800b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0296d9d640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 0x7f02800b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 -- 172.21.15.60:0/537429062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 msgr2=0x563e7f997e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f997e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 -- 172.21.15.60:0/537429062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 msgr2=0x563e7f987a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 0x563e7f987a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 -- 172.21.15.60:0/537429062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7f877300 con 0x563e7f998370 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f02877fe640 1 -- 172.21.15.60:0/231877219 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f0288105520 con 0x7f0298101650 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0295d9b640 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f028c00bcb0 tx=0x7f028c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a31759441780b7b server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0296d9d640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 0x7f02800b6200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f029819aa20 tx=0x7f0290004730 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0284ff9640 1 -- 172.21.15.60:0/537429062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c015070 con 0x563e7f998370 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0284ff9640 1 -- 172.21.15.60:0/537429062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c01a910 con 0x563e7f998370 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.521+0000 7f0284ff9640 1 -- 172.21.15.60:0/537429062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0118b0 con 0x563e7f998370 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 -- 172.21.15.60:0/537429062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 msgr2=0x563e7f99a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f028c00bcb0 tx=0x7f028c00f730 comp rx=0 tx=0).stop 2024-07-24T22:04:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 -- 172.21.15.60:0/537429062 shutdown_connections 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f99a760 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f997e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 --2- 172.21.15.60:0/537429062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f9876b0 0x563e7f987a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 -- 172.21.15.60:0/537429062 >> 172.21.15.60:0/537429062 conn(0x563e7f9854e0 msgr2=0x563e7f986680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 -- 172.21.15.60:0/537429062 shutdown_connections 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.522+0000 7f029eeb8740 1 -- 172.21.15.60:0/537429062 wait complete. 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 -- start start 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 Processor -- start 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f995810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f995d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563e7f751700 con 0x563e7f9876b0 2024-07-24T22:04:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e7f8905d0 con 0x563e7f987fd0 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563e7f99b020 con 0x563e7f998370 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029eeb8740 10 client.0 _ll_register_callbacks cb 0x563e7f8b0940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44428/0 (socket says 172.21.15.60:44428) 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 -- 172.21.15.60:0/3438591488 learned_addr learned my addr 172.21.15.60:0/3438591488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f0296d9d640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f995d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f995810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 -- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 msgr2=0x563e7f995d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f995d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 -- 172.21.15.60:0/3438591488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 msgr2=0x563e7f995810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f995810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f029659c640 1 -- 172.21.15.60:0/3438591488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7f883660 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.523+0000 7f0296d9d640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f995d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029659c640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f02880cc1a0 tx=0x7f0288009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a09f8a08151612 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880cb980 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f0288102590 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f028c01a910 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f02880cbd20 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 10 client.19227 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563e7f64e650 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c01a910 con 0x563e7f9876b0 2024-07-24T22:04:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0288102910 con 0x563e7f9876b0 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 10 client.19227 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 20 client.19227 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7f9a7f30 0x563e7f9aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 4 client.19227 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.524+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e7f9008e0 con 0x563e7f9a7f30 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.525+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7f9a7f30 0x563e7f9aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.525+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7f9a7f30 0x563e7f9aa390 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2537033c688f4cd0 server_cookie=5fd32c5142b9ecf3 in_seq=0 out_seq=0 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.525+0000 7f027dffb640 10 client.19227 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f027dffb640 1 -- 172.21.15.60:0/3438591488 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e7f9008e0 con 0x563e7f9a7f30 2024-07-24T22:04:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f027dffb640 10 client.19227 handle_command_reply: tid=0 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 client.19227 shutdown 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 msgr2=0x563e7f9952d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f02880cc1a0 tx=0x7f0288009650 comp rx=0 tx=0).stop 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 shutdown_connections 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7f9a7f30 0x563e7f9aa390 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f995d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f027dffb640 0 client.19227 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3438591488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f9876b0 0x563e7f9952d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 >> 172.21.15.60:0/3438591488 conn(0x563e7f9854e0 msgr2=0x563e7f98b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 shutdown_connections 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 1 -- 172.21.15.60:0/3438591488 wait complete. 2024-07-24T22:04:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.526+0000 7f029eeb8740 20 client.19227 trim_cache size 0 max 16384 2024-07-24T22:04:53.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 Processor -- start 2024-07-24T22:04:53.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 -- start start 2024-07-24T22:04:53.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f986d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 0x563e7f9afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02981ae860 con 0x563e7f998370 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02981ac070 con 0x563e7f997010 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.547+0000 7f029eeb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a9ee0 con 0x563e7f987fd0 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44434/0 (socket says 172.21.15.60:44434) 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0295d9b640 1 -- 172.21.15.60:0/3122777966 learned_addr learned my addr 172.21.15.60:0/3122777966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f986d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0296d9d640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 0x563e7f9afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 -- 172.21.15.60:0/3122777966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 msgr2=0x563e7f9afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 0x563e7f9afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 -- 172.21.15.60:0/3122777966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 msgr2=0x563e7f992e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 -- 172.21.15.60:0/3122777966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7f9941c0 con 0x563e7f987fd0 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f0296d9d640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 0x563e7f9afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f029659c640 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f986d50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02880c9480 tx=0x7f02880095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c418363b15a6ca server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.548+0000 7f027ffff640 1 -- 172.21.15.60:0/3122777966 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880cb980 con 0x563e7f987fd0 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f027ffff640 1 -- 172.21.15.60:0/3122777966 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0288036040 con 0x563e7f987fd0 2024-07-24T22:04:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f027ffff640 1 -- 172.21.15.60:0/3122777966 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880cbfe0 con 0x563e7f987fd0 2024-07-24T22:04:53.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 -- 172.21.15.60:0/3122777966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 msgr2=0x563e7f986d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f986d50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02880c9480 tx=0x7f02880095c0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 -- 172.21.15.60:0/3122777966 shutdown_connections 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f997010 0x563e7f9afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f998370 0x563e7f992e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3122777966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f987fd0 0x563e7f986d50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 -- 172.21.15.60:0/3122777966 >> 172.21.15.60:0/3122777966 conn(0x563e7f9854e0 msgr2=0x563e7f996900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 -- 172.21.15.60:0/3122777966 shutdown_connections 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.549+0000 7f029eeb8740 1 -- 172.21.15.60:0/3122777966 wait complete. 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 -- start start 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 Processor -- start 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f02981aa530 con 0x563e7f997010 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02981ab9a0 con 0x563e7f987fd0 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02981aac00 con 0x563e7f998370 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029eeb8740 10 client.0 _ll_register_callbacks cb 0x7f0288005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f0296d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f0296d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:47040/0 (socket says 172.21.15.60:47040) 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f029659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:52436/0 (socket says 172.21.15.60:52436) 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f0296d9d640 1 -- 172.21.15.60:0/3980883343 learned_addr learned my addr 172.21.15.60:0/3980883343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.550+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44448/0 (socket says 172.21.15.60:44448) 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 -- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 msgr2=0x563e7f99d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 -- 172.21.15.60:0/3980883343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 msgr2=0x563e7f9a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 -- 172.21.15.60:0/3980883343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7f99dd00 con 0x563e7f997010 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0296d9d640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f029659c640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f0295d9b640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f028c00bbd0 tx=0x7f028c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412cb64fb838aaee server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f027dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c02c060 con 0x563e7f997010 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f028c005240 con 0x563e7f997010 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0288036040 con 0x563e7f997010 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f028c015030 con 0x563e7f997010 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f029eeb8740 10 client.19233 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.551+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563e7fa13970 con 0x563e7f997010 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0288036040 con 0x563e7f997010 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f028c004b30 con 0x563e7f997010 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 10 client.19233 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 10 client.19233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 20 client.19233 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7fa13e10 0x563e7fa16230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 4 client.19233 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e7f8fc380 con 0x563e7fa13e10 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.552+0000 7f029659c640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7fa13e10 0x563e7fa16230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029659c640 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7fa13e10 0x563e7fa16230 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916c6f82866a63ef server_cookie=a3e10f5109e795b4 in_seq=0 out_seq=0 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f027dffb640 10 client.19233 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f027dffb640 1 -- 172.21.15.60:0/3980883343 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e7f8fc380 con 0x563e7fa13e10 2024-07-24T22:04:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f027dffb640 10 client.19233 handle_command_reply: tid=0 2024-07-24T22:04:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 client.19233 shutdown 2024-07-24T22:04:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 msgr2=0x563e7f9a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f028c00bbd0 tx=0x7f028c004270 comp rx=0 tx=0).stop 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 shutdown_connections 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x563e7fa13e10 0x563e7fa16230 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563e7f998370 0x563e7f99d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f027dffb640 0 client.19233 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563e7f997010 0x563e7f9a0be0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 --2- 172.21.15.60:0/3980883343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e7f987fd0 0x563e7f9a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 >> 172.21.15.60:0/3980883343 conn(0x563e7f9854e0 msgr2=0x563e7f995240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 shutdown_connections 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.553+0000 7f029eeb8740 1 -- 172.21.15.60:0/3980883343 wait complete. 2024-07-24T22:04:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029eeb8740 20 client.19233 trim_cache size 0 max 16384 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "22962021-c0ae-4d68-940f-e9f650868ef2": { 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "22962021-c0ae-4d68-940f-e9f650868ef2", 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:04:53.561 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 msgr2=0x7f02800b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 0x7f02800b6200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f029819aa20 tx=0x7f0290004730 comp rx=0 tx=0).stop 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 msgr2=0x7f0298199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0288006010 tx=0x7f02880319f0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 shutdown_connections 2024-07-24T22:04:53.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f02800b3d40 0x7f02800b6200 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02981079e0 0x7f029819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0298107060 0x7f0298199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 --2- 172.21.15.60:0/231877219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0298101650 0x7f0298199a40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.554+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 >> 172.21.15.60:0/231877219 conn(0x7f02980fd0c0 msgr2=0x7f02980fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.555+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 shutdown_connections 2024-07-24T22:04:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:53.555+0000 7f029cb07640 1 -- 172.21.15.60:0/231877219 wait complete. 2024-07-24T22:04:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:53 smithi086 ceph-mon[37128]: pgmap v950: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-07-24T22:04:53.808 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:53.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:53 smithi060 ceph-mon[33000]: pgmap v950: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-07-24T22:04:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:53 smithi123 ceph-mon[37143]: pgmap v950: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-07-24T22:04:54.344 INFO:tasks.cephfs.filesystem:scrub status for tag:22962021-c0ae-4d68-940f-e9f650868ef2 - {'path': '/', 'tag': '22962021-c0ae-4d68-940f-e9f650868ef2', 'options': 'recursive,force'} 2024-07-24T22:04:55.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:55 smithi060 ceph-mon[33000]: pgmap v951: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-07-24T22:04:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:55 smithi123 ceph-mon[37143]: pgmap v951: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 -- 172.21.15.60:0/983855964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4102790 msgr2=0x7fc0f4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 --2- 172.21.15.60:0/983855964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4102790 0x7fc0f4102b70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009ca0 tx=0x7fc0ec02a8c0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 -- 172.21.15.60:0/983855964 shutdown_connections 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 --2- 172.21.15.60:0/983855964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4108b20 0x7fc0f410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 --2- 172.21.15.60:0/983855964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 0x7fc0f41085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 --2- 172.21.15.60:0/983855964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4102790 0x7fc0f4102b70 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 -- 172.21.15.60:0/983855964 >> 172.21.15.60:0/983855964 conn(0x7fc0f40fe200 msgr2=0x7fc0f4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 -- 172.21.15.60:0/983855964 shutdown_connections 2024-07-24T22:04:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.968+0000 7fc0fae07640 1 -- 172.21.15.60:0/983855964 wait complete. 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 Processor -- start 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 -- start start 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4102790 0x7fc0f4074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 0x7fc0f40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4112720 con 0x7fc0f4108b20 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41125a0 con 0x7fc0f41081a0 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0fae07640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41128a0 con 0x7fc0f4102790 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58358/0 (socket says 172.21.15.60:58358) 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 -- 172.21.15.60:0/3047277963 learned_addr learned my addr 172.21.15.60:0/3047277963 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f8ba3640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4102790 0x7fc0f4074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 -- 172.21.15.60:0/3047277963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4102790 msgr2=0x7fc0f4074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0ebfff640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 0x7fc0f40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4102790 0x7fc0f4074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 -- 172.21.15.60:0/3047277963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 msgr2=0x7fc0f40753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 0x7fc0f40753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.969+0000 7fc0f93a4640 1 -- 172.21.15.60:0/3047277963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f41a6fd0 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0f93a4640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c970 tx=0x7fc0e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007bb0 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e4007d50 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41a72c0 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4018770 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.970+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a77b0 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.971+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4102b70 con 0x7fc0f4108b20 2024-07-24T22:04:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.972+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fc0e400f990 con 0x7fc0f4108b20 2024-07-24T22:04:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.973+0000 7fc0e9ffb640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 0x7fc0dc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.973+0000 7fc0f8ba3640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 0x7fc0dc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.973+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fc0e40e41f0 con 0x7fc0f4108b20 2024-07-24T22:04:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.974+0000 7fc0f8ba3640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 0x7fc0dc0b6200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec006010 tx=0x7fc0ec000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:55.976+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fc0e4014030 con 0x7fc0f4108b20 2024-07-24T22:04:56.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:55 smithi086 ceph-mon[37128]: pgmap v951: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-07-24T22:04:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.218+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0f41a7c10 con 0x7fc0f4108b20 2024-07-24T22:04:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.219+0000 7fc0e9ffb640 1 -- 172.21.15.60:0/3047277963 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7fc0e40ab560 con 0x7fc0f4108b20 2024-07-24T22:04:56.221 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:04:56.221 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:04:56.222 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 msgr2=0x7fc0dc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 0x7fc0dc0b6200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec006010 tx=0x7fc0ec000f40 comp rx=0 tx=0).stop 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 msgr2=0x7fc0f4075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c970 tx=0x7fc0e400ce30 comp rx=0 tx=0).stop 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 shutdown_connections 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fc0dc0b3d40 0x7fc0dc0b6200 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f4108b20 0x7fc0f4075910 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:56.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f41081a0 0x7fc0f40753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:56.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 --2- 172.21.15.60:0/3047277963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f4102790 0x7fc0f4074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:56.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 >> 172.21.15.60:0/3047277963 conn(0x7fc0f40fe200 msgr2=0x7fc0f40fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:56.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 shutdown_connections 2024-07-24T22:04:56.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:56.221+0000 7fc0fae07640 1 -- 172.21.15.60:0/3047277963 wait complete. 2024-07-24T22:04:56.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:56 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3047277963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:56.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:56 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3047277963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:57.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:56 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3047277963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:04:57.167 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24610 quiesce db --cancel --all' 2024-07-24T22:04:57.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- timeout 105 ceph tell mds.24610 quiesce db --cancel --all 2024-07-24T22:04:57.442 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 -- 172.21.15.60:0/4069542067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99181088f0 msgr2=0x7f991810f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 --2- 172.21.15.60:0/4069542067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99181088f0 0x7f991810f0f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9910009900 tx=0x7f991002a410 comp rx=0 tx=0).stop 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 -- 172.21.15.60:0/4069542067 shutdown_connections 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 --2- 172.21.15.60:0/4069542067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99181088f0 0x7f991810f0f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 --2- 172.21.15.60:0/4069542067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918107f70 0x7f99181083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 --2- 172.21.15.60:0/4069542067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918102570 0x7f9918102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 -- 172.21.15.60:0/4069542067 >> 172.21.15.60:0/4069542067 conn(0x7f99180fdf90 msgr2=0x7f99181003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 -- 172.21.15.60:0/4069542067 shutdown_connections 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.502+0000 7f991e3a0640 1 -- 172.21.15.60:0/4069542067 wait complete. 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 Processor -- start 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 -- start start 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 0x7f9918074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99181088f0 0x7f9918075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9918112660 con 0x7f9918107f70 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99181124e0 con 0x7f9918102570 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991e3a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99181127e0 con 0x7f99181088f0 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58366/0 (socket says 172.21.15.60:58366) 2024-07-24T22:04:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 -- 172.21.15.60:0/880721610 learned_addr learned my addr 172.21.15.60:0/880721610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991db9f640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99181088f0 0x7f9918075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991d39e640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 0x7f9918074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 -- 172.21.15.60:0/880721610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99181088f0 msgr2=0x7f9918075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99181088f0 0x7f9918075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 -- 172.21.15.60:0/880721610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 msgr2=0x7f9918074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 0x7f9918074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 -- 172.21.15.60:0/880721610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99181a6ff0 con 0x7f9918107f70 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991d39e640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 0x7f9918074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.503+0000 7f991cb9d640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f991400b4a0 tx=0x7f991400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.504+0000 7f990a7fc640 1 -- 172.21.15.60:0/880721610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914014070 con 0x7f9918107f70 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.504+0000 7f990a7fc640 1 -- 172.21.15.60:0/880721610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914010ac0 con 0x7f9918107f70 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.504+0000 7f990a7fc640 1 -- 172.21.15.60:0/880721610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914017c30 con 0x7f9918107f70 2024-07-24T22:04:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.504+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99181a72e0 con 0x7f9918107f70 2024-07-24T22:04:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.504+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a7830 con 0x7f9918107f70 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.506+0000 7f9922430740 1 Processor -- start 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.506+0000 7f9922430740 1 -- start start 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.506+0000 7f990a7fc640 1 -- 172.21.15.60:0/880721610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9914017070 con 0x7f9918107f70 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5511153c0 0x55c551120e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c551121350 0x55c551123740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c55108a280 con 0x55c551114aa0 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c551079780 con 0x55c5511153c0 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f9922430740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c55107d7c0 con 0x55c551121350 2024-07-24T22:04:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58378/0 (socket says 172.21.15.60:58378) 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 -- 172.21.15.60:0/668335385 learned_addr learned my addr 172.21.15.60:0/668335385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991cb9d640 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c551121350 0x55c551123740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991d39e640 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5511153c0 0x55c551120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 -- 172.21.15.60:0/668335385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c551121350 msgr2=0x55c551123740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c551121350 0x55c551123740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 -- 172.21.15.60:0/668335385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5511153c0 msgr2=0x55c551120e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5511153c0 0x55c551120e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 -- 172.21.15.60:0/668335385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c550ddd790 con 0x55c551114aa0 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f991db9f640 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f991002aa00 tx=0x7f9910004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5706119442c1724a server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.507+0000 7f990a7fc640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 0x7f99040b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f991d39e640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 0x7f99040b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f99037fe640 1 -- 172.21.15.60:0/668335385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910040070 con 0x55c551114aa0 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f99037fe640 1 -- 172.21.15.60:0/668335385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99100047a0 con 0x55c551114aa0 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f990a7fc640 1 -- 172.21.15.60:0/880721610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f99140e3650 con 0x7f9918107f70 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f99037fe640 1 -- 172.21.15.60:0/668335385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910035040 con 0x55c551114aa0 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f991d39e640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 0x7f99040b6200 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55c551117c20 tx=0x7f990c00e040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 -- 172.21.15.60:0/668335385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 msgr2=0x55c551114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f991002aa00 tx=0x7f9910004390 comp rx=0 tx=0).stop 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 -- 172.21.15.60:0/668335385 shutdown_connections 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c551121350 0x55c551123740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5511153c0 0x55c551120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 --2- 172.21.15.60:0/668335385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551114e80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.508+0000 7f9922430740 1 -- 172.21.15.60:0/668335385 >> 172.21.15.60:0/668335385 conn(0x55c5511128d0 msgr2=0x55c551113a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- 172.21.15.60:0/668335385 shutdown_connections 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- 172.21.15.60:0/668335385 wait complete. 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- start start 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 Processor -- start 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c55111dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c55111e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c550fb3ab0 con 0x55c551114aa0 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5510019e0 con 0x55c551121350 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c551123fd0 con 0x55c5511153c0 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f9922430740 10 client.0 _ll_register_callbacks cb 0x55c550d89a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.509+0000 7f991d39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58392/0 (socket says 172.21.15.60:58392) 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 -- 172.21.15.60:0/823849129 learned_addr learned my addr 172.21.15.60:0/823849129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991db9f640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c55111e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991cb9d640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c55111dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 -- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 msgr2=0x55c55111dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c55111dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 -- 172.21.15.60:0/823849129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 msgr2=0x55c55111e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c55111e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 -- 172.21.15.60:0/823849129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c550e56620 con 0x55c551114aa0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991cb9d640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c55111dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f991d39e640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f990c007e50 tx=0x7f990c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608376f0abdf1e43 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c01e040 con 0x55c551114aa0 2024-07-24T22:04:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f990c022420 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c551028ce0 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f990c022620 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9922430740 10 client.19251 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.510+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c55112f810 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c551028ce0 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f990c02b7a0 con 0x55c551114aa0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 10 client.19251 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 10 client.19251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 20 client.19251 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c551130d70 0x55c5511331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 4 client.19251 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c55108c1d0 con 0x55c551130d70 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f991cb9d640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c551130d70 0x55c5511331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9900ff9640 10 client.19251 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.511+0000 7f991cb9d640 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c551130d70 0x55c5511331e0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea416887af695017 server_cookie=74d3c366a235af35 in_seq=0 out_seq=0 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9900ff9640 1 -- 172.21.15.60:0/823849129 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c55108c1d0 con 0x55c551130d70 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9900ff9640 10 client.19251 handle_command_reply: tid=0 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 client.19251 shutdown 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 msgr2=0x55c551120ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f990c007e50 tx=0x7f990c004a40 comp rx=0 tx=0).stop 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 shutdown_connections 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c551130d70 0x55c5511331e0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c55111e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9900ff9640 0 client.19251 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c55111dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 --2- 172.21.15.60:0/823849129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c551120ac0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.512+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 >> 172.21.15.60:0/823849129 conn(0x55c5511128d0 msgr2=0x55c551118470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.513+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 shutdown_connections 2024-07-24T22:04:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.513+0000 7f9922430740 1 -- 172.21.15.60:0/823849129 wait complete. 2024-07-24T22:04:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.513+0000 7f9922430740 20 client.19251 trim_cache size 0 max 16384 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 Processor -- start 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 -- start start 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551111920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c551138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b2c20 con 0x55c551114aa0 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b0430 con 0x55c551121350 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f9922430740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ae2a0 con 0x55c5511153c0 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58404/0 (socket says 172.21.15.60:58404) 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 -- 172.21.15.60:0/3822769757 learned_addr learned my addr 172.21.15.60:0/3822769757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 -- 172.21.15.60:0/3822769757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 msgr2=0x55c551111920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 -- 172.21.15.60:0/3822769757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 msgr2=0x55c551138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c551138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.528+0000 7f991d39e640 1 -- 172.21.15.60:0/3822769757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5511157a0 con 0x55c551114aa0 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f991d39e640 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f80 tx=0x7f990c0225b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3c8fd1511def1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9902ffd640 1 -- 172.21.15.60:0/3822769757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c039020 con 0x55c551114aa0 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9902ffd640 1 -- 172.21.15.60:0/3822769757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c027070 con 0x55c551114aa0 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9902ffd640 1 -- 172.21.15.60:0/3822769757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c01e040 con 0x55c551114aa0 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 -- 172.21.15.60:0/3822769757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 msgr2=0x55c55112d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f80 tx=0x7f990c0225b0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 -- 172.21.15.60:0/3822769757 shutdown_connections 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551121350 0x55c551138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 --2- 172.21.15.60:0/3822769757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551114aa0 0x55c55112d120 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.529+0000 7f9922430740 1 -- 172.21.15.60:0/3822769757 >> 172.21.15.60:0/3822769757 conn(0x55c5511128d0 msgr2=0x55c551111210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- 172.21.15.60:0/3822769757 shutdown_connections 2024-07-24T22:04:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- 172.21.15.60:0/3822769757 wait complete. 2024-07-24T22:04:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- start start 2024-07-24T22:04:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 Processor -- start 2024-07-24T22:04:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551114aa0 0x55c551127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551128560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ae8f0 con 0x55c551121350 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99181afd60 con 0x55c551114aa0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99181aefc0 con 0x55c5511153c0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f9922430740 10 client.0 _ll_register_callbacks cb 0x7f990c01b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f991db9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f991db9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58412/0 (socket says 172.21.15.60:58412) 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.530+0000 7f991db9f640 1 -- 172.21.15.60:0/3968871939 learned_addr learned my addr 172.21.15.60:0/3968871939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991d39e640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551114aa0 0x55c551127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 -- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 msgr2=0x55c551128560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991cb9d640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551128560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551128560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 -- 172.21.15.60:0/3968871939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551114aa0 msgr2=0x55c551127fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551114aa0 0x55c551127fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 -- 172.21.15.60:0/3968871939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5511941d0 con 0x55c551121350 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991cb9d640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551128560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f991db9f640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78da7cddbd0f7662 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9900ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910035040 con 0x55c551121350 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f9910003da0 con 0x55c551121350 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f990c027070 con 0x55c551121350 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9910040040 con 0x55c551121350 2024-07-24T22:04:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9922430740 10 client.19257 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c55119b590 con 0x55c551121350 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.531+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990c027070 con 0x55c551121350 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f991003c440 con 0x55c551121350 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 10 client.19257 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 10 client.19257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 20 client.19257 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c55119b870 0x55c55119dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 4 client.19257 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c55110dfd0 con 0x55c55119b870 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f991d39e640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c55119b870 0x55c55119dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f991d39e640 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c55119b870 0x55c55119dc60 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1776fb3eab32fda server_cookie=f720a70ca4efc063 in_seq=0 out_seq=0 2024-07-24T22:04:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.532+0000 7f9900ff9640 10 client.19257 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9900ff9640 1 -- 172.21.15.60:0/3968871939 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c55110dfd0 con 0x55c55119b870 2024-07-24T22:04:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9900ff9640 10 client.19257 handle_command_reply: tid=0 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 client.19257 shutdown 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 msgr2=0x55c551193a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910003820 comp rx=0 tx=0).stop 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 shutdown_connections 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c55119b870 0x55c55119dc60 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c551121350 0x55c551193a90 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c5511153c0 0x55c551128560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9900ff9640 0 client.19257 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 --2- 172.21.15.60:0/3968871939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c551114aa0 0x55c551127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 >> 172.21.15.60:0/3968871939 conn(0x55c5511128d0 msgr2=0x55c55111fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 shutdown_connections 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 1 -- 172.21.15.60:0/3968871939 wait complete. 2024-07-24T22:04:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.542+0000 7f9922430740 20 client.19257 trim_cache size 0 max 16384 2024-07-24T22:04:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-07-24T22:04:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-07-24T22:04:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-07-24T22:04:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 msgr2=0x7f99040b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 0x7f99040b6200 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55c551117c20 tx=0x7f990c00e040 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 msgr2=0x7f99180753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f991400b4a0 tx=0x7f991400b960 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 shutdown_connections 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f99040b3d40 0x7f99040b6200 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99181088f0 0x7f9918075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9918107f70 0x7f99180753d0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 --2- 172.21.15.60:0/880721610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9918102570 0x7f9918074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 >> 172.21.15.60:0/880721610 conn(0x7f99180fdf90 msgr2=0x7f9918100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 shutdown_connections 2024-07-24T22:04:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-07-24T22:04:57.543+0000 7f991e3a0640 1 -- 172.21.15.60:0/880721610 wait complete. 2024-07-24T22:04:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:57 smithi060 ceph-mon[33000]: pgmap v952: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-07-24T22:04:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:57 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:04:57.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:57 smithi060 ceph-mon[33000]: scrub summary: idle 2024-07-24T22:04:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:57 smithi123 ceph-mon[37143]: pgmap v952: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-07-24T22:04:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:57 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:04:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:57 smithi123 ceph-mon[37143]: scrub summary: idle 2024-07-24T22:04:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:57 smithi086 ceph-mon[37128]: pgmap v952: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-07-24T22:04:58.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:57 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:04:58.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:57 smithi086 ceph-mon[37128]: scrub summary: idle 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 -- 172.21.15.60:0/4150274871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e0103f70 msgr2=0x7f60e010c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 --2- 172.21.15.60:0/4150274871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e0103f70 0x7f60e010c550 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402a410 comp rx=0 tx=0).stop 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 -- 172.21.15.60:0/4150274871 shutdown_connections 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 --2- 172.21.15.60:0/4150274871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e010cb20 0x7f60e010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 --2- 172.21.15.60:0/4150274871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e0103f70 0x7f60e010c550 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 --2- 172.21.15.60:0/4150274871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103570 0x7f60e0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 -- 172.21.15.60:0/4150274871 >> 172.21.15.60:0/4150274871 conn(0x7f60e00fd100 msgr2=0x7f60e00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 -- 172.21.15.60:0/4150274871 shutdown_connections 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.114+0000 7f60e5d15640 1 -- 172.21.15.60:0/4150274871 wait complete. 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.115+0000 7f60e5d15640 1 Processor -- start 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.115+0000 7f60e5d15640 1 -- start start 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e0103570 0x7f60e0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 0x7f60e0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0112750 con 0x7f60e010cb20 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01125d0 con 0x7f60e0103570 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01128d0 con 0x7f60e0103f70 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58430/0 (socket says 172.21.15.60:58430) 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 -- 172.21.15.60:0/1143123639 learned_addr learned my addr 172.21.15.60:0/1143123639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60df7fe640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e0103570 0x7f60e0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60deffd640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 0x7f60e0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 -- 172.21.15.60:0/1143123639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 msgr2=0x7f60e0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 0x7f60e0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 -- 172.21.15.60:0/1143123639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e0103570 msgr2=0x7f60e0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e0103570 0x7f60e0198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 -- 172.21.15.60:0/1143123639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e019ec90 con 0x7f60e010cb20 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60deffd640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 0x7f60e0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dffff640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f60d800cce0 tx=0x7f60d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dcff9640 1 -- 172.21.15.60:0/1143123639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d800f450 con 0x7f60e010cb20 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e019ef80 con 0x7f60e010cb20 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dcff9640 1 -- 172.21.15.60:0/1143123639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d800faa0 con 0x7f60e010cb20 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.116+0000 7f60dcff9640 1 -- 172.21.15.60:0/1143123639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8018b70 con 0x7f60e010cb20 2024-07-24T22:04:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.117+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01a5640 con 0x7f60e010cb20 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 Processor -- start 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60dcff9640 1 -- 172.21.15.60:0/1143123639 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f60d8018d10 con 0x7f60e010cb20 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 -- start start 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d0f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d100e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56357d0689d0 con 0x56357d0f4180 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56357d057ed0 con 0x56357d0fea50 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60e80c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56357d05bf10 con 0x56357d0f2c20 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.119+0000 7f60deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58434/0 (socket says 172.21.15.60:58434) 2024-07-24T22:04:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 -- 172.21.15.60:0/1346414500 learned_addr learned my addr 172.21.15.60:0/1346414500 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60df7fe640 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d0f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 -- 172.21.15.60:0/1346414500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 msgr2=0x56357d0f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d0f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 -- 172.21.15.60:0/1346414500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 msgr2=0x56357d100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 -- 172.21.15.60:0/1346414500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357d038390 con 0x56357d0f4180 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60df7fe640 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d0f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60deffd640 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a930 tx=0x7f60d4031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad16adaea6be37 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60dcff9640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 0x7f60c80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60c5ffb640 1 -- 172.21.15.60:0/1346414500 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4035050 con 0x56357d0f4180 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60df7fe640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 0x7f60c80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60c5ffb640 1 -- 172.21.15.60:0/1346414500 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40316a0 con 0x56357d0f4180 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60c5ffb640 1 -- 172.21.15.60:0/1346414500 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40047a0 con 0x56357d0f4180 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.120+0000 7f60dcff9640 1 -- 172.21.15.60:0/1143123639 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f60d80e4490 con 0x7f60e010cb20 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- 172.21.15.60:0/1346414500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 msgr2=0x56357d0f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60df7fe640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 0x7f60c80b6200 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x56357d0f5860 tx=0x7f60d000a250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a930 tx=0x7f60d4031de0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- 172.21.15.60:0/1346414500 shutdown_connections 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d0f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 --2- 172.21.15.60:0/1346414500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f26e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- 172.21.15.60:0/1346414500 >> 172.21.15.60:0/1346414500 conn(0x56357d0bce10 msgr2=0x56357d0f0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- 172.21.15.60:0/1346414500 shutdown_connections 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- 172.21.15.60:0/1346414500 wait complete. 2024-07-24T22:04:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.121+0000 7f60e80c6740 1 -- start start 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 Processor -- start 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 0x56357d0fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d0fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56357cebb810 con 0x56357d0f2c20 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56357cffa640 con 0x56357d0fea50 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56357d101950 con 0x56357d0f4180 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60e80c6740 10 client.0 _ll_register_callbacks cb 0x56357cd64b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58446/0 (socket says 172.21.15.60:58446) 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 -- 172.21.15.60:0/649695319 learned_addr learned my addr 172.21.15.60:0/649695319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60deffd640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 0x56357d0fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 -- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 msgr2=0x56357d0fb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60dffff640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d0fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 0x56357d0fb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 -- 172.21.15.60:0/649695319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 msgr2=0x56357d0fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d0fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60df7fe640 1 -- 172.21.15.60:0/649695319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357cdb88b0 con 0x56357d0f2c20 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.122+0000 7f60deffd640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 0x56357d0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60df7fe640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f60d0012b30 tx=0x7f60d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aa54112c430280 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000fda0 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f60d001f420 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d40316a0 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f60d001f620 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60e80c6740 10 client.19269 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56357cff50d0 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40316a0 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f60d002e070 con 0x56357d0f2c20 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60e80c6740 10 client.19269 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.123+0000 7f60e80c6740 10 client.19269 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60e80c6740 20 client.19269 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d10e8a0 0x56357d110d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60e80c6740 4 client.19269 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357d06a920 con 0x56357d10e8a0 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60deffd640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d10e8a0 0x56357d110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60deffd640 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d10e8a0 0x56357d110d40 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f005ced85a9bcc9 server_cookie=7e36e66f93cbb616 in_seq=0 out_seq=0 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.124+0000 7f60beffd640 10 client.19269 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60beffd640 1 -- 172.21.15.60:0/649695319 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56357d06a920 con 0x56357d10e8a0 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60beffd640 10 client.19269 handle_command_reply: tid=0 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 client.19269 shutdown 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 msgr2=0x56357d0fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f60d0012b30 tx=0x7f60d00046c0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 shutdown_connections 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d10e8a0 0x56357d110d40 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0fea50 0x56357d0fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f4180 0x56357d0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 --2- 172.21.15.60:0/649695319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f2c20 0x56357d0fe200 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60beffd640 0 client.19269 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 >> 172.21.15.60:0/649695319 conn(0x56357d0bce10 msgr2=0x56357d0f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 shutdown_connections 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 1 -- 172.21.15.60:0/649695319 wait complete. 2024-07-24T22:04:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.125+0000 7f60e80c6740 20 client.19269 trim_cache size 0 max 16384 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 Processor -- start 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 -- start start 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f2c20 0x56357d0f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0fea50 0x56357d0efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01b09d0 con 0x56357d0f4180 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ae1e0 con 0x56357d0f2c20 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60e80c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ac050 con 0x56357d0fea50 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58456/0 (socket says 172.21.15.60:58456) 2024-07-24T22:04:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 -- 172.21.15.60:0/613206817 learned_addr learned my addr 172.21.15.60:0/613206817 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60df7fe640 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f2c20 0x56357d0f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60dffff640 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0fea50 0x56357d0efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 -- 172.21.15.60:0/613206817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0fea50 msgr2=0x56357d0efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0fea50 0x56357d0efe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 -- 172.21.15.60:0/613206817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f2c20 msgr2=0x56357d0f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f2c20 0x56357d0f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.144+0000 7f60deffd640 1 -- 172.21.15.60:0/613206817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357d0ebdc0 con 0x56357d0f4180 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60deffd640 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f855dd753928f555 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60c57fa640 1 -- 172.21.15.60:0/613206817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d404e070 con 0x56357d0f4180 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60c57fa640 1 -- 172.21.15.60:0/613206817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40035f0 con 0x56357d0f4180 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60c57fa640 1 -- 172.21.15.60:0/613206817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4035040 con 0x56357d0f4180 2024-07-24T22:04:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 -- 172.21.15.60:0/613206817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 msgr2=0x56357d0f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d4000bc0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 -- 172.21.15.60:0/613206817 shutdown_connections 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0fea50 0x56357d0efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0f4180 0x56357d0f5fa0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 --2- 172.21.15.60:0/613206817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f2c20 0x56357d0f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.145+0000 7f60e80c6740 1 -- 172.21.15.60:0/613206817 >> 172.21.15.60:0/613206817 conn(0x56357d0bce10 msgr2=0x56357d0f5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- 172.21.15.60:0/613206817 shutdown_connections 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- 172.21.15.60:0/613206817 wait complete. 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- start start 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 Processor -- start 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f4180 0x56357d107370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ac6a0 con 0x56357d0fea50 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01adb10 con 0x56357d0f4180 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01acd70 con 0x56357d0f2c20 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60e80c6740 10 client.0 _ll_register_callbacks cb 0x7f60d000a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.146+0000 7f60dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58462/0 (socket says 172.21.15.60:58462) 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 -- 172.21.15.60:0/2619537577 learned_addr learned my addr 172.21.15.60:0/2619537577 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60deffd640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f4180 0x56357d107370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60df7fe640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 -- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 msgr2=0x56357d106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d106e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 -- 172.21.15.60:0/2619537577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f4180 msgr2=0x56357d107370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f4180 0x56357d107370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 -- 172.21.15.60:0/2619537577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357d102340 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60dffff640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f60d80ab000 tx=0x7f60d80aa8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b433a64205049b1c server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60beffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d80af450 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f60d80b3c90 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d40035f0 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f60d80ab070 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60e80c6740 10 client.19275 fetch_fsmap learned FSMap version 46 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.147+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56357d17a0e0 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40035f0 con 0x56357d0fea50 2024-07-24T22:04:59.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f60d80aa460 con 0x56357d0fea50 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 10 client.19275 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 10 client.19275 resolve_mds: validated gid 24610 aka daemon mds.i 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 20 client.19275 populate_metadata read hostname 'smithi060' 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d17a890 0x56357d17ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 4 client.19275 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x56357d0663c0 con 0x56357d17a890 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60df7fe640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d17a890 0x56357d17ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60beffd640 10 client.19275 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.148+0000 7f60df7fe640 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d17a890 0x56357d17ccb0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ccba1fd26c9299 server_cookie=aa960ecd18f5ea6d in_seq=0 out_seq=0 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60beffd640 1 -- 172.21.15.60:0/2619537577 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56357d0663c0 con 0x56357d17a890 2024-07-24T22:04:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60beffd640 10 client.19275 handle_command_reply: tid=0 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 client.19275 shutdown 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 msgr2=0x56357d101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f60d80ab000 tx=0x7f60d80aa8a0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 shutdown_connections 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x56357d17a890 0x56357d17ccb0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56357d0fea50 0x56357d101c00 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56357d0f4180 0x56357d107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 --2- 172.21.15.60:0/2619537577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357d0f2c20 0x56357d106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60beffd640 0 client.19275 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 >> 172.21.15.60:0/2619537577 conn(0x56357d0bce10 msgr2=0x56357d0f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 shutdown_connections 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 1 -- 172.21.15.60:0/2619537577 wait complete. 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.149+0000 7f60e80c6740 20 client.19275 trim_cache size 0 max 16384 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 46, 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout: "leader": 24610, 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout: "set_version": 56, 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout: "sets": {} 2024-07-24T22:04:59.165 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 msgr2=0x7f60c80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 0x7f60c80b6200 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x56357d0f5860 tx=0x7f60d000a250 comp rx=0 tx=0).stop 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 msgr2=0x7f60e019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f60d800cce0 tx=0x7f60d800a620 comp rx=0 tx=0).stop 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 shutdown_connections 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f60c80b3d40 0x7f60c80b6200 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60e010cb20 0x7f60e019e4c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e0103f70 0x7f60e0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 --2- 172.21.15.60:0/1143123639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e0103570 0x7f60e0198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:04:59.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 >> 172.21.15.60:0/1143123639 conn(0x7f60e00fd100 msgr2=0x7f60e010ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:04:59.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 shutdown_connections 2024-07-24T22:04:59.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:04:59.150+0000 7f60e5d15640 1 -- 172.21.15.60:0/1143123639 wait complete. 2024-07-24T22:04:59.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:04:59 smithi060 ceph-mon[33000]: pgmap v953: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-07-24T22:04:59.940 INFO:tasks.quiescer:done joining Quiescers 2024-07-24T22:04:59.940 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-07-24T22:04:59.951 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T22:04:59.960 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-07-24T22:04:59.985 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-07-24T22:04:59.985 INFO:teuthology.task.exec:Executing custom commands... 2024-07-24T22:04:59.985 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T22:04:59.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-07-24T22:04:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:04:59 smithi123 ceph-mon[37143]: pgmap v953: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-07-24T22:05:00.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-07-24T22:05:00.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:04:59 smithi086 ceph-mon[37128]: pgmap v953: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-07-24T22:05:00.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-24T22:05:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 -- 172.21.15.60:0/3802777687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac1015e0 msgr2=0x7f7fac1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 --2- 172.21.15.60:0/3802777687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac1015e0 0x7f7fac1019c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009900 tx=0x7f7fa402a410 comp rx=0 tx=0).stop 2024-07-24T22:05:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 -- 172.21.15.60:0/3802777687 shutdown_connections 2024-07-24T22:05:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 --2- 172.21.15.60:0/3802777687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 0x7f7fac10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 --2- 172.21.15.60:0/3802777687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac106f90 0x7f7fac1073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 --2- 172.21.15.60:0/3802777687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac1015e0 0x7f7fac1019c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 -- 172.21.15.60:0/3802777687 >> 172.21.15.60:0/3802777687 conn(0x7f7fac0782e0 msgr2=0x7f7fac0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 -- 172.21.15.60:0/3802777687 shutdown_connections 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.250+0000 7f7fb428d640 1 -- 172.21.15.60:0/3802777687 wait complete. 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 Processor -- start 2024-07-24T22:05:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 -- start start 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac1015e0 0x7f7fac197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 0x7f7fac19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac111620 con 0x7f7fac106f90 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac1114a0 con 0x7f7fac1015e0 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb428d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac1117a0 con 0x7f7fac107910 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58482/0 (socket says 172.21.15.60:58482) 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 -- 172.21.15.60:0/188202006 learned_addr learned my addr 172.21.15.60:0/188202006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb2029640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac1015e0 0x7f7fac197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 -- 172.21.15.60:0/188202006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 msgr2=0x7f7fac19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb282a640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 0x7f7fac19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 0x7f7fac19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 -- 172.21.15.60:0/188202006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac1015e0 msgr2=0x7f7fac197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac1015e0 0x7f7fac197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 -- 172.21.15.60:0/188202006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac19dbe0 con 0x7f7fac106f90 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.251+0000 7f7fb1828640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000ba50 tx=0x7f7fa000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.252+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0019070 con 0x7f7fac106f90 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.252+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa000cbe0 con 0x7f7fac106f90 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.252+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac19ded0 con 0x7f7fac106f90 2024-07-24T22:05:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.252+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00075d0 con 0x7f7fac106f90 2024-07-24T22:05:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.252+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac1a4770 con 0x7f7fac106f90 2024-07-24T22:05:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.253+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac1019c0 con 0x7f7fac106f90 2024-07-24T22:05:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.254+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7fa001c070 con 0x7f7fac106f90 2024-07-24T22:05:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.255+0000 7f7fab7fe640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 0x7f7f940b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.255+0000 7f7fb2029640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 0x7f7f940b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.255+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f7fa00e3a70 con 0x7f7fac106f90 2024-07-24T22:05:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.255+0000 7f7fb2029640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 0x7f7f940b6280 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a930 tx=0x7f7fa4005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:00.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.258+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f7fa00aae00 con 0x7f7fac106f90 2024-07-24T22:05:00.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.478+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7fac062f80 con 0x7f7f940b3dc0 2024-07-24T22:05:00.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.487+0000 7f7fab7fe640 1 -- 172.21.15.60:0/188202006 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7fac062f80 con 0x7f7f940b3dc0 2024-07-24T22:05:00.489 INFO:teuthology.orchestra.run.smithi060.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-24T00:00:00", "created": "2024-07-24T21:46:30", "first": "2024-07-24T21:47:00", "last": "2024-07-24T22:04:00", "last_pruned": "2024-07-24T22:04:00", "created_count": 18, "pruned_count": 12, "active": true} 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 msgr2=0x7f7f940b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 0x7f7f940b6280 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a930 tx=0x7f7fa4005fd0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 msgr2=0x7f7fac198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000ba50 tx=0x7f7fa000bf10 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 shutdown_connections 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f7f940b3dc0 0x7f7f940b6280 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fac107910 0x7f7fac19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fac106f90 0x7f7fac198080 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 --2- 172.21.15.60:0/188202006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fac1015e0 0x7f7fac197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 >> 172.21.15.60:0/188202006 conn(0x7f7fac0782e0 msgr2=0x7f7fac103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 shutdown_connections 2024-07-24T22:05:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.490+0000 7f7fb428d640 1 -- 172.21.15.60:0/188202006 wait complete. 2024-07-24T22:05:00.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.614+0000 7f8df2c64640 1 -- 172.21.15.60:0/977540133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 msgr2=0x7f8dec101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.614+0000 7f8df2c64640 1 --2- 172.21.15.60:0/977540133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec101a20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002a410 comp rx=0 tx=0).stop 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 -- 172.21.15.60:0/977540133 shutdown_connections 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 --2- 172.21.15.60:0/977540133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec1079c0 0x7f8dec10e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 --2- 172.21.15.60:0/977540133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec107040 0x7f8dec107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 --2- 172.21.15.60:0/977540133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec101a20 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 -- 172.21.15.60:0/977540133 >> 172.21.15.60:0/977540133 conn(0x7f8dec0fd080 msgr2=0x7f8dec0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 -- 172.21.15.60:0/977540133 shutdown_connections 2024-07-24T22:05:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 -- 172.21.15.60:0/977540133 wait complete. 2024-07-24T22:05:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 Processor -- start 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 -- start start 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.615+0000 7f8df2c64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 0x7f8dec0fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec1079c0 0x7f8dec0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1115f0 con 0x7f8dec101640 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec111470 con 0x7f8dec107040 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec111770 con 0x7f8dec1079c0 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58494/0 (socket says 172.21.15.60:58494) 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 -- 172.21.15.60:0/3431651837 learned_addr learned my addr 172.21.15.60:0/3431651837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df1201640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec1079c0 0x7f8dec0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8debfff640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 0x7f8dec0fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 -- 172.21.15.60:0/3431651837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec1079c0 msgr2=0x7f8dec0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec1079c0 0x7f8dec0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 -- 172.21.15.60:0/3431651837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 msgr2=0x7f8dec0fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 0x7f8dec0fdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 -- 172.21.15.60:0/3431651837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec1a70a0 con 0x7f8dec101640 2024-07-24T22:05:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8debfff640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 0x7f8dec0fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df0a00640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8de0005940 tx=0x7f8de0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de003f030 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de0003aa0 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec1a7330 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.616+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0035070 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.617+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1a7820 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.618+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dec101a20 con 0x7f8dec101640 2024-07-24T22:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.620+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8de0042030 con 0x7f8dec101640 2024-07-24T22:05:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.621+0000 7f8de9ffb640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.621+0000 7f8debfff640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.621+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f8de0104990 con 0x7f8dec101640 2024-07-24T22:05:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.622+0000 7f8debfff640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0ff330 tx=0x7f8dd8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.625+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8de00cbc70 con 0x7f8dec101640 2024-07-24T22:05:00.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.848+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 --> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8dec062f80 con 0x7f8ddc0b3d40 2024-07-24T22:05:00.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.857+0000 7f8de9ffb640 1 -- 172.21.15.60:0/3431651837 <== mgr.14214 v2:172.21.15.60:6800/1194501493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8dec062f80 con 0x7f8ddc0b3d40 2024-07-24T22:05:00.858 INFO:teuthology.orchestra.run.smithi060.stdout:/ 1m 6m3h 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 msgr2=0x7f8ddc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0ff330 tx=0x7f8dd8005f90 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 msgr2=0x7f8dec0fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8de0005940 tx=0x7f8de0004270 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 shutdown_connections 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dec1079c0 0x7f8dec0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec107040 0x7f8dec0fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 --2- 172.21.15.60:0/3431651837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dec101640 0x7f8dec0fd840 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 >> 172.21.15.60:0/3431651837 conn(0x7f8dec0fd080 msgr2=0x7f8dec06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 shutdown_connections 2024-07-24T22:05:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:00.859+0000 7f8df2c64640 1 -- 172.21.15.60:0/3431651837 wait complete. 2024-07-24T22:05:00.872 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-24T22:05:00.882 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-24T22:05:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:01 smithi060 ceph-mon[33000]: from='client.19281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:01 smithi060 ceph-mon[33000]: pgmap v954: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-07-24T22:05:01.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:01 smithi060 ceph-mon[33000]: from='client.19287 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:01 smithi123 ceph-mon[37143]: from='client.19281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:01 smithi123 ceph-mon[37143]: pgmap v954: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-07-24T22:05:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:01 smithi123 ceph-mon[37143]: from='client.19287 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:01 smithi086 ceph-mon[37128]: from='client.19281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:01 smithi086 ceph-mon[37128]: pgmap v954: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-07-24T22:05:02.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:01 smithi086 ceph-mon[37128]: from='client.19287 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-24T22:05:03.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:03 smithi060 ceph-mon[33000]: pgmap v955: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-07-24T22:05:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:03 smithi123 ceph-mon[37143]: pgmap v955: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-07-24T22:05:04.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:03 smithi086 ceph-mon[37128]: pgmap v955: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-07-24T22:05:05.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:05 smithi060 ceph-mon[33000]: pgmap v956: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-07-24T22:05:05.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:05 smithi123 ceph-mon[37143]: pgmap v956: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-07-24T22:05:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:05 smithi086 ceph-mon[37128]: pgmap v956: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-07-24T22:05:07.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:07 smithi060 ceph-mon[33000]: pgmap v957: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 76 KiB/s wr, 79 op/s 2024-07-24T22:05:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:07 smithi123 ceph-mon[37143]: pgmap v957: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 76 KiB/s wr, 79 op/s 2024-07-24T22:05:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:07 smithi086 ceph-mon[37128]: pgmap v957: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 76 KiB/s wr, 79 op/s 2024-07-24T22:05:09.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:09 smithi060 ceph-mon[33000]: pgmap v958: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-07-24T22:05:09.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:09 smithi123 ceph-mon[37143]: pgmap v958: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-07-24T22:05:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:09 smithi086 ceph-mon[37128]: pgmap v958: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-07-24T22:05:11.848 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:11 smithi086 ceph-mon[37128]: pgmap v959: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:11.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:11 smithi123 ceph-mon[37143]: pgmap v959: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:12.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:11 smithi060 ceph-mon[33000]: pgmap v959: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:13 smithi123 ceph-mon[37143]: pgmap v960: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:13 smithi086 ceph-mon[37128]: pgmap v960: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:13 smithi060 ceph-mon[33000]: pgmap v960: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-07-24T22:05:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:15 smithi123 ceph-mon[37143]: pgmap v961: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-24T22:05:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:05:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:05:15.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:05:15.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:05:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:15 smithi086 ceph-mon[37128]: pgmap v961: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-24T22:05:16.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:05:16.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:05:16.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:05:16.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:05:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:15 smithi060 ceph-mon[33000]: pgmap v961: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-24T22:05:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:05:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:05:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:05:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:05:17.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:17 smithi123 ceph-mon[37143]: pgmap v962: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T22:05:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:17 smithi086 ceph-mon[37128]: pgmap v962: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T22:05:18.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:17 smithi060 ceph-mon[33000]: pgmap v962: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-07-24T22:05:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:19 smithi123 ceph-mon[37143]: pgmap v963: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-07-24T22:05:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:19 smithi086 ceph-mon[37128]: pgmap v963: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-07-24T22:05:20.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:19 smithi060 ceph-mon[33000]: pgmap v963: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-07-24T22:05:21.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:21 smithi123 ceph-mon[37143]: pgmap v964: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-07-24T22:05:22.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:21 smithi086 ceph-mon[37128]: pgmap v964: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-07-24T22:05:22.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:21 smithi060 ceph-mon[33000]: pgmap v964: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-07-24T22:05:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:23 smithi123 ceph-mon[37143]: pgmap v965: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T22:05:24.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:23 smithi086 ceph-mon[37128]: pgmap v965: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T22:05:24.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:23 smithi060 ceph-mon[33000]: pgmap v965: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-24T22:05:24.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-24T22:05:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:25.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:25 smithi060 ceph-mon[33000]: pgmap v966: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:25 smithi123 ceph-mon[37143]: pgmap v966: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:26.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:25 smithi086 ceph-mon[37128]: pgmap v966: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/149270685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc1081a0 msgr2=0x7f0adc1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/149270685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc1081a0 0x7f0adc1085e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009900 tx=0x7f0ad002a410 comp rx=0 tx=0).stop 2024-07-24T22:05:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/149270685 shutdown_connections 2024-07-24T22:05:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/149270685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc108b20 0x7f0adc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/149270685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc1081a0 0x7f0adc1085e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/149270685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 0x7f0adc102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/149270685 >> 172.21.15.60:0/149270685 conn(0x7f0adc0fe200 msgr2=0x7f0adc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/149270685 shutdown_connections 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.311+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/149270685 wait complete. 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 Processor -- start 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 -- start start 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 0x7f0adc074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 0x7f0adc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc112730 con 0x7f0adc108b20 2024-07-24T22:05:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1125b0 con 0x7f0adc102790 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0ae1ca3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1128b0 con 0x7f0adc1081a0 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.312+0000 7f0adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37840/0 (socket says 172.21.15.60:37840) 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 -- 172.21.15.60:0/2648385111 learned_addr learned my addr 172.21.15.60:0/2648385111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 0x7f0adc074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adaffd640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 0x7f0adc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 -- 172.21.15.60:0/2648385111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 msgr2=0x7f0adc0753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 0x7f0adc0753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 -- 172.21.15.60:0/2648385111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 msgr2=0x7f0adc074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 0x7f0adc074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 -- 172.21.15.60:0/2648385111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc1a6fe0 con 0x7f0adc108b20 2024-07-24T22:05:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adaffd640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 0x7f0adc0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0adbfff640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400cce0 tx=0x7f0ad400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0ad8ff9640 1 -- 172.21.15.60:0/2648385111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400f450 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0ad8ff9640 1 -- 172.21.15.60:0/2648385111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad400faa0 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc1a72d0 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0ad8ff9640 1 -- 172.21.15.60:0/2648385111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4018b70 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.313+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1a7790 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ad8ff9640 1 -- 172.21.15.60:0/2648385111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f0ad4018d10 con 0x7f0adc108b20 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 Processor -- start 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 -- start start 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c493e64960 con 0x55c493eed680 2024-07-24T22:05:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c493e53e60 con 0x55c493eedfa0 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ae4054740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c493e57ea0 con 0x55c493ef9f30 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37854/0 (socket says 172.21.15.60:37854) 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0adaffd640 1 -- 172.21.15.60:0/2532370802 learned_addr learned my addr 172.21.15.60:0/2532370802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.316+0000 7f0ad8ff9640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 0x7f0ac40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adbfff640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 0x7f0ac40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 -- 172.21.15.60:0/2532370802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 msgr2=0x55c493efc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 -- 172.21.15.60:0/2532370802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 msgr2=0x55c493ef99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 -- 172.21.15.60:0/2532370802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c493ddd2d0 con 0x55c493eed680 2024-07-24T22:05:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0ad8ff9640 1 -- 172.21.15.60:0/2648385111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f0ad40e44f0 con 0x7f0adc108b20 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adaffd640 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009900 tx=0x7f0ad0032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af0f511f7116744 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0adbfff640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 0x7f0ac40b6200 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40ab000 tx=0x7f0ad4018610 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0ac1ffb640 1 -- 172.21.15.60:0/2532370802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0035050 con 0x55c493eed680 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0ac1ffb640 1 -- 172.21.15.60:0/2532370802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad00316a0 con 0x55c493eed680 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.317+0000 7f0ac1ffb640 1 -- 172.21.15.60:0/2532370802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00046c0 con 0x55c493eed680 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- 172.21.15.60:0/2532370802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 msgr2=0x55c493eeda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009900 tx=0x7f0ad0032df0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- 172.21.15.60:0/2532370802 shutdown_connections 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 --2- 172.21.15.60:0/2532370802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493eeda60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- 172.21.15.60:0/2532370802 >> 172.21.15.60:0/2532370802 conn(0x55c493eeb4b0 msgr2=0x55c493eec650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- 172.21.15.60:0/2532370802 shutdown_connections 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- 172.21.15.60:0/2532370802 wait complete. 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.318+0000 7f0ae4054740 1 -- start start 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 Processor -- start 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493ef78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c493cb7700 con 0x55c493eed680 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c493df65a0 con 0x55c493eedfa0 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c493efcb50 con 0x55c493ef9f30 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0ae4054740 10 client.0 _ll_register_callbacks cb 0x55c493e16910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37868/0 (socket says 172.21.15.60:37868) 2024-07-24T22:05:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 -- 172.21.15.60:0/1016334087 learned_addr learned my addr 172.21.15.60:0/1016334087 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 -- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 msgr2=0x55c493ef78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493ef78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 -- 172.21.15.60:0/1016334087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 msgr2=0x55c493ef73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 -- 172.21.15.60:0/1016334087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c493de9630 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005ec0 tx=0x7f0ad0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50721ab524a7b06f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.319+0000 7f0abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0035070 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f0ad0004f90 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad00316a0 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0ad0005560 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 10 client.19299 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c493bb4650 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad00316a0 con 0x55c493eed680 2024-07-24T22:05:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0ad003dbf0 con 0x55c493eed680 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 10 client.19299 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 10 client.19299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 20 client.19299 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f097f0 0x55c493f0bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 4 client.19299 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.320+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c493e668b0 con 0x55c493f097f0 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.321+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f097f0 0x55c493f0bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.321+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f097f0 0x55c493f0bbe0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37dacb601b51a657 server_cookie=a3734f25c0d7c630 in_seq=0 out_seq=0 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.321+0000 7f0abaffd640 10 client.19299 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0abaffd640 1 -- 172.21.15.60:0/1016334087 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c493e668b0 con 0x55c493f097f0 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0abaffd640 10 client.19299 handle_command_reply: tid=0 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 client.19299 shutdown 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 msgr2=0x55c493ef6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005ec0 tx=0x7f0ad0004e20 comp rx=0 tx=0).stop 2024-07-24T22:05:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 shutdown_connections 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f097f0 0x55c493f0bbe0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493ef78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493ef73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1016334087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef6e60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0abaffd640 0 client.19299 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 >> 172.21.15.60:0/1016334087 conn(0x55c493eeb4b0 msgr2=0x55c493ef1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 shutdown_connections 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 1 -- 172.21.15.60:0/1016334087 wait complete. 2024-07-24T22:05:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.322+0000 7f0ae4054740 20 client.19299 trim_cache size 0 max 16384 2024-07-24T22:05:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 Processor -- start 2024-07-24T22:05:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 -- start start 2024-07-24T22:05:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493f11180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493f116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b2bb0 con 0x55c493eed680 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b03c0 con 0x55c493eedfa0 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0ae4054740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1ae230 con 0x55c493ef9f30 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37876/0 (socket says 172.21.15.60:37876) 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 -- 172.21.15.60:0/3765128918 learned_addr learned my addr 172.21.15.60:0/3765128918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 -- 172.21.15.60:0/3765128918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 msgr2=0x55c493f116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493f116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 -- 172.21.15.60:0/3765128918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 msgr2=0x55c493f11180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493f11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 -- 172.21.15.60:0/3765128918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c493efc7a0 con 0x55c493eed680 2024-07-24T22:05:26.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.341+0000 7f0adaffd640 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0001eb0 tx=0x7f0ad003a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cbf9a615b7e129 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ac17fa640 1 -- 172.21.15.60:0/3765128918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0035050 con 0x55c493eed680 2024-07-24T22:05:26.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ac17fa640 1 -- 172.21.15.60:0/3765128918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad004d070 con 0x55c493eed680 2024-07-24T22:05:26.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ac17fa640 1 -- 172.21.15.60:0/3765128918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad003aa00 con 0x55c493eed680 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 -- 172.21.15.60:0/3765128918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 msgr2=0x55c493ef74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0001eb0 tx=0x7f0ad003a580 comp rx=0 tx=0).stop 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 -- 172.21.15.60:0/3765128918 shutdown_connections 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493f116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eedfa0 0x55c493f11180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 --2- 172.21.15.60:0/3765128918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eed680 0x55c493ef74e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 -- 172.21.15.60:0/3765128918 >> 172.21.15.60:0/3765128918 conn(0x55c493eeb4b0 msgr2=0x55c493ef6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 -- 172.21.15.60:0/3765128918 shutdown_connections 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.342+0000 7f0ae4054740 1 -- 172.21.15.60:0/3765128918 wait complete. 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 -- start start 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 Processor -- start 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 0x55c493f02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1ae880 con 0x55c493eedfa0 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1afcf0 con 0x55c493eed680 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1aef50 con 0x55c493ef9f30 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0ae4054740 10 client.0 _ll_register_callbacks cb 0x7f0acc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0adb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0adb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37890/0 (socket says 172.21.15.60:37890) 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.343+0000 7f0adb7fe640 1 -- 172.21.15.60:0/1840709194 learned_addr learned my addr 172.21.15.60:0/1840709194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 0x55c493f02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 -- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 msgr2=0x55c493efe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 -- 172.21.15.60:0/1840709194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 msgr2=0x55c493f02710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 0x55c493f02710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 -- 172.21.15.60:0/1840709194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c493f73150 con 0x55c493eedfa0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 0x55c493f02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0adb7fe640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55c493eec390 tx=0x7f0acc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a057fbfb60cc8e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0abaffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc011640 con 0x55c493eedfa0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f0acc0117e0 con 0x55c493eedfa0 2024-07-24T22:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad004d070 con 0x55c493eedfa0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0acc0119e0 con 0x55c493eedfa0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0ae4054740 10 client.19305 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.344+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c493f744f0 con 0x55c493eedfa0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad004d070 con 0x55c493eedfa0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f0acc026400 con 0x55c493eedfa0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 10 client.19305 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 10 client.19305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 20 client.19305 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f74d00 0x55c493f77120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 4 client.19305 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c493e62350 con 0x55c493f74d00 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f74d00 0x55c493f77120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0adaffd640 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f74d00 0x55c493f77120 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd1f1724b3d7c0f1 server_cookie=a819831ba579b211 in_seq=0 out_seq=0 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.345+0000 7f0abaffd640 10 client.19305 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0abaffd640 1 -- 172.21.15.60:0/1840709194 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c493e62350 con 0x55c493f74d00 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0abaffd640 10 client.19305 handle_command_reply: tid=0 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 client.19305 shutdown 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 msgr2=0x55c493efdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55c493eec390 tx=0x7f0acc00bf10 comp rx=0 tx=0).stop 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 shutdown_connections 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x55c493f74d00 0x55c493f77120 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c493ef9f30 0x55c493efe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c493eedfa0 0x55c493efdc70 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 --2- 172.21.15.60:0/1840709194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c493eed680 0x55c493f02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 >> 172.21.15.60:0/1840709194 conn(0x55c493eeb4b0 msgr2=0x55c493ef75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0abaffd640 0 client.19305 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 shutdown_connections 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 1 -- 172.21.15.60:0/1840709194 wait complete. 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.346+0000 7f0ae4054740 20 client.19305 trim_cache size 0 max 16384 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-07-24T22:05:26.353 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 msgr2=0x7f0ac40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 0x7f0ac40b6200 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40ab000 tx=0x7f0ad4018610 comp rx=0 tx=0).stop 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 msgr2=0x7f0adc075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400cce0 tx=0x7f0ad400a620 comp rx=0 tx=0).stop 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 shutdown_connections 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f0ac40b3d40 0x7f0ac40b6200 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0adc108b20 0x7f0adc075910 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc1081a0 0x7f0adc0753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 --2- 172.21.15.60:0/2648385111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc102790 0x7f0adc074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 >> 172.21.15.60:0/2648385111 conn(0x7f0adc0fe200 msgr2=0x7f0adc1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 shutdown_connections 2024-07-24T22:05:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:26.347+0000 7f0ae1ca3640 1 -- 172.21.15.60:0/2648385111 wait complete. 2024-07-24T22:05:27.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-24T22:05:27.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:05:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:27.811 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:27 smithi060 ceph-mon[33000]: pgmap v967: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:27 smithi123 ceph-mon[37143]: pgmap v967: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:28.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:27 smithi086 ceph-mon[37128]: pgmap v967: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:29.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 -- 172.21.15.60:0/1773994794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc1070a0 msgr2=0x7f61dc1074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 --2- 172.21.15.60:0/1773994794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc1070a0 0x7f61dc1074e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009900 tx=0x7f61d002a410 comp rx=0 tx=0).stop 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 -- 172.21.15.60:0/1773994794 shutdown_connections 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 --2- 172.21.15.60:0/1773994794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc107a20 0x7f61dc10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 --2- 172.21.15.60:0/1773994794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc1070a0 0x7f61dc1074e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 --2- 172.21.15.60:0/1773994794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc101690 0x7f61dc101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 -- 172.21.15.60:0/1773994794 >> 172.21.15.60:0/1773994794 conn(0x7f61dc0fd100 msgr2=0x7f61dc0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 -- 172.21.15.60:0/1773994794 shutdown_connections 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.058+0000 7f61e4238640 1 -- 172.21.15.60:0/1773994794 wait complete. 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 Processor -- start 2024-07-24T22:05:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 -- start start 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc1070a0 0x7f61dc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc107a20 0x7f61dc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1116d0 con 0x7f61dc101690 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc111550 con 0x7f61dc107a20 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e4238640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc111850 con 0x7f61dc1070a0 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e1fd4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e1fd4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37922/0 (socket says 172.21.15.60:37922) 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e1fd4640 1 -- 172.21.15.60:0/2918536934 learned_addr learned my addr 172.21.15.60:0/2918536934 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.059+0000 7f61e17d3640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc1070a0 0x7f61dc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e27d5640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc107a20 0x7f61dc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:29.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 -- 172.21.15.60:0/2918536934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc1070a0 msgr2=0x7f61dc199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc1070a0 0x7f61dc199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 -- 172.21.15.60:0/2918536934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc107a20 msgr2=0x7f61dc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc107a20 0x7f61dc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 -- 172.21.15.60:0/2918536934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc19ed90 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e1fd4640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b4a0 tx=0x7f61cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc014070 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc010ac0 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc19f080 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc017c30 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.060+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a57c0 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.061+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc101a70 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.063+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f61cc017070 con 0x7f61dc101690 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.064+0000 7f61daffd640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 0x7f61c40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:29.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.064+0000 7f61e17d3640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 0x7f61c40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:29.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.064+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f61cc0e3620 con 0x7f61dc101690 2024-07-24T22:05:29.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.065+0000 7f61e17d3640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 0x7f61c40b6200 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61d0031440 tx=0x7f61d0005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:29.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.067+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f61cc0aa9b0 con 0x7f61dc101690 2024-07-24T22:05:29.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.301+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61dc199e10 con 0x7f61dc101690 2024-07-24T22:05:29.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.302+0000 7f61daffd640 1 -- 172.21.15.60:0/2918536934 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f61cc0aa2b0 con 0x7f61dc101690 2024-07-24T22:05:29.304 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:05:29.304 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:05:29.305 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 msgr2=0x7f61c40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 0x7f61c40b6200 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61d0031440 tx=0x7f61d0005b50 comp rx=0 tx=0).stop 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 msgr2=0x7f61dc198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b4a0 tx=0x7f61cc00b960 comp rx=0 tx=0).stop 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 shutdown_connections 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f61c40b3d40 0x7f61c40b6200 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc107a20 0x7f61dc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61dc1070a0 0x7f61dc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 --2- 172.21.15.60:0/2918536934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61dc101690 0x7f61dc198ce0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 >> 172.21.15.60:0/2918536934 conn(0x7f61dc0fd100 msgr2=0x7f61dc06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 shutdown_connections 2024-07-24T22:05:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:29.304+0000 7f61e4238640 1 -- 172.21.15.60:0/2918536934 wait complete. 2024-07-24T22:05:29.957 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:29 smithi060 ceph-mon[33000]: pgmap v968: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:29.957 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:29 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2918536934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:05:29.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-24T22:05:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:29 smithi123 ceph-mon[37143]: pgmap v968: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:29 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2918536934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:05:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:29 smithi086 ceph-mon[37128]: pgmap v968: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:05:30.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:29 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2918536934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:31.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:31 smithi060 ceph-mon[33000]: pgmap v969: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 -- 172.21.15.60:0/356842890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101270 msgr2=0x7f12b0101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 --2- 172.21.15.60:0/356842890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101270 0x7f12b0101650 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009ca0 tx=0x7f12a002a950 comp rx=0 tx=0).stop 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 -- 172.21.15.60:0/356842890 shutdown_connections 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 --2- 172.21.15.60:0/356842890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b010ee20 0x7f12b0111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 --2- 172.21.15.60:0/356842890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b0101b90 0x7f12b010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 --2- 172.21.15.60:0/356842890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101270 0x7f12b0101650 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.930+0000 7f12b5611640 1 -- 172.21.15.60:0/356842890 >> 172.21.15.60:0/356842890 conn(0x7f12b00fd0c0 msgr2=0x7f12b00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.931+0000 7f12b5611640 1 -- 172.21.15.60:0/356842890 shutdown_connections 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.931+0000 7f12b5611640 1 -- 172.21.15.60:0/356842890 wait complete. 2024-07-24T22:05:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.931+0000 7f12b5611640 1 Processor -- start 2024-07-24T22:05:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.931+0000 7f12b5611640 1 -- start start 2024-07-24T22:05:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b0101270 0x7f12b019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b010ee20 0x7f12b01a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0114930 con 0x7f12b0101b90 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01147b0 con 0x7f12b010ee20 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12b5611640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0114ab0 con 0x7f12b0101270 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b0101270 0x7f12b019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37942/0 (socket says 172.21.15.60:37942) 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 -- 172.21.15.60:0/677324041 learned_addr learned my addr 172.21.15.60:0/677324041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12af7fe640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b010ee20 0x7f12b01a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 -- 172.21.15.60:0/677324041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b0101270 msgr2=0x7f12b019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b0101270 0x7f12b019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 -- 172.21.15.60:0/677324041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b010ee20 msgr2=0x7f12b01a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b010ee20 0x7f12b01a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 -- 172.21.15.60:0/677324041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a0f90 con 0x7f12b0101b90 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.932+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f12a400d950 tx=0x7f12a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.933+0000 7f1297fff640 1 -- 172.21.15.60:0/677324041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4017070 con 0x7f12b0101b90 2024-07-24T22:05:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.933+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a1280 con 0x7f12b0101b90 2024-07-24T22:05:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.933+0000 7f1297fff640 1 -- 172.21.15.60:0/677324041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a4009c30 con 0x7f12b0101b90 2024-07-24T22:05:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.933+0000 7f1297fff640 1 -- 172.21.15.60:0/677324041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400f620 con 0x7f12b0101b90 2024-07-24T22:05:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.933+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a7940 con 0x7f12b0101b90 2024-07-24T22:05:31.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f1297fff640 1 -- 172.21.15.60:0/677324041 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f12a4010460 con 0x7f12b0101b90 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f12b79c2740 1 Processor -- start 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f12b79c2740 1 -- start start 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b409ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b409ff0 0x55a47b415a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.935+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12b79c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a47b3809b0 con 0x55a47b415f80 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12b79c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a47b36feb0 con 0x55a47b409ff0 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12b79c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a47b373ef0 con 0x55a47b4096d0 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37952/0 (socket says 172.21.15.60:37952) 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2828095344 learned_addr learned my addr 172.21.15.60:0/2828095344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12af7fe640 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b409ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b409ff0 0x55a47b415a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2828095344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 msgr2=0x55a47b409ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b409ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2828095344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b409ff0 msgr2=0x55a47b415a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b409ff0 0x55a47b415a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2828095344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47b2f9320 con 0x55a47b415f80 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f1297fff640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 0x7f12980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e1300 tx=0x7f12a40ec7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f05954052fd879e server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.936+0000 7f12af7fe640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 0x7f12980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f1297fff640 1 -- 172.21.15.60:0/677324041 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f12a40e3c70 con 0x7f12b0101b90 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12957fa640 1 -- 172.21.15.60:0/2828095344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4020030 con 0x55a47b415f80 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12957fa640 1 -- 172.21.15.60:0/2828095344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a40aa360 con 0x55a47b415f80 2024-07-24T22:05:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12957fa640 1 -- 172.21.15.60:0/2828095344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40a6710 con 0x55a47b415f80 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 -- 172.21.15.60:0/2828095344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 msgr2=0x55a47b418370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12af7fe640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 0x7f12980b6200 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005d80 tx=0x7f12a800b420 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e1300 tx=0x7f12a40ec7b0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 -- 172.21.15.60:0/2828095344 shutdown_connections 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b415f80 0x55a47b418370 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b409ff0 0x55a47b415a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2828095344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b409ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.937+0000 7f12b79c2740 1 -- 172.21.15.60:0/2828095344 >> 172.21.15.60:0/2828095344 conn(0x55a47b407500 msgr2=0x55a47b4086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- 172.21.15.60:0/2828095344 shutdown_connections 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- 172.21.15.60:0/2828095344 wait complete. 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- start start 2024-07-24T22:05:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 Processor -- start 2024-07-24T22:05:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 0x55a47b413470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b4139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55a47b1d3700 con 0x55a47b4096d0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a47b3125f0 con 0x55a47b415f80 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a47b418c00 con 0x55a47b409ff0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12b79c2740 10 client.0 _ll_register_callbacks cb 0x55a47b332960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37962/0 (socket says 172.21.15.60:37962) 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12aeffd640 1 -- 172.21.15.60:0/193665855 learned_addr learned my addr 172.21.15.60:0/193665855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.938+0000 7f12af7fe640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b4139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 0x55a47b413470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 -- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 msgr2=0x55a47b413470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 0x55a47b413470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 -- 172.21.15.60:0/193665855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 msgr2=0x55a47b4139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b4139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 -- 172.21.15.60:0/193665855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47b305680 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 0x55a47b413470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12aeffd640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009ca0 tx=0x7f12a002fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2765f14a8df18abf server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f1292ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0049060 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f12a0003970 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a40aa360 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f12a0034040 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12b79c2740 10 client.19323 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.939+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a47b0d0650 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a40aa360 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f12a0030440 con 0x55a47b4096d0 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 10 client.19323 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 10 client.19323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 20 client.19323 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b425b80 0x55a47b428000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 4 client.19323 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a47b382900 con 0x55a47b425b80 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.940+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b425b80 0x55a47b428000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.941+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b425b80 0x55a47b428000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74eae3e5fe8652a8 server_cookie=44cb4e31ebc93bf8 in_seq=0 out_seq=0 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.941+0000 7f1292ff5640 10 client.19323 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.941+0000 7f1292ff5640 1 -- 172.21.15.60:0/193665855 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a47b382900 con 0x55a47b425b80 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.941+0000 7f1292ff5640 10 client.19323 handle_command_reply: tid=0 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.941+0000 7f12b79c2740 1 client.19323 shutdown 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 msgr2=0x55a47b412f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009ca0 tx=0x7f12a002fbe0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 shutdown_connections 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b425b80 0x55a47b428000 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b4139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b409ff0 0x55a47b413470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 --2- 172.21.15.60:0/193665855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b4096d0 0x55a47b412f30 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 >> 172.21.15.60:0/193665855 conn(0x55a47b407500 msgr2=0x55a47b40d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f1292ff5640 0 client.19323 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 shutdown_connections 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 1 -- 172.21.15.60:0/193665855 wait complete. 2024-07-24T22:05:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.942+0000 7f12b79c2740 20 client.19323 trim_cache size 0 max 16384 2024-07-24T22:05:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 Processor -- start 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 -- start start 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b40d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b42de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b2e70 con 0x55a47b409ff0 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0680 con 0x55a47b415f80 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12b79c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ae4f0 con 0x55a47b4096d0 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37978/0 (socket says 172.21.15.60:37978) 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2024180297 learned_addr learned my addr 172.21.15.60:0/2024180297 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b40d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2024180297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 msgr2=0x55a47b40d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12af7fe640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b42de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b40d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2024180297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 msgr2=0x55a47b42de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b42de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2024180297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47b421d20 con 0x55a47b409ff0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.956+0000 7f12af7fe640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b42de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e1990 tx=0x7f12a40e77c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca2af50c6c0f5b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f1294ff9640 1 -- 172.21.15.60:0/2024180297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4020040 con 0x55a47b409ff0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f1294ff9640 1 -- 172.21.15.60:0/2024180297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a40e7610 con 0x55a47b409ff0 2024-07-24T22:05:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f1294ff9640 1 -- 172.21.15.60:0/2024180297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40a9fc0 con 0x55a47b409ff0 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f12b79c2740 1 -- 172.21.15.60:0/2024180297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 msgr2=0x55a47b42d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.957+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e1990 tx=0x7f12a40e77c0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- 172.21.15.60:0/2024180297 shutdown_connections 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b42de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b42d8d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2024180297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b40d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- 172.21.15.60:0/2024180297 >> 172.21.15.60:0/2024180297 conn(0x55a47b407500 msgr2=0x55a47b414d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- 172.21.15.60:0/2024180297 shutdown_connections 2024-07-24T22:05:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- 172.21.15.60:0/2024180297 wait complete. 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- start start 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 Processor -- start 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b41e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b40f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01aeb40 con 0x55a47b409ff0 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01affb0 con 0x55a47b415f80 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01af210 con 0x55a47b4096d0 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.958+0000 7f12b79c2740 10 client.0 _ll_register_callbacks cb 0x7f12a002f870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37992/0 (socket says 172.21.15.60:37992) 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2542329669 learned_addr learned my addr 172.21.15.60:0/2542329669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b41e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12af7fe640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b40f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 msgr2=0x55a47b41e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b41e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2542329669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 msgr2=0x55a47b40f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b40f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 -- 172.21.15.60:0/2542329669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47b40ff60 con 0x55a47b409ff0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b41e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12ae7fc640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e7dd0 tx=0x7f12a40ecf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c0c25a052eff89 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f1292ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4020070 con 0x55a47b409ff0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f12a40b3040 con 0x55a47b409ff0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.959+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a40e7610 con 0x55a47b409ff0 2024-07-24T22:05:31.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f12a40aa1a0 con 0x55a47b409ff0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 10 client.19329 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a47b4902d0 con 0x55a47b409ff0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a40e7610 con 0x55a47b409ff0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7f12a40e1610 con 0x55a47b409ff0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 10 client.19329 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 10 client.19329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 20 client.19329 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b490610 0x55a47b492a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 4 client.19329 mds_command: new command op to 24598 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 --> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a47b37e3a0 con 0x55a47b490610 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.960+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b490610 0x55a47b492a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f1292ff5640 10 client.19329 ms_handle_connect on v2:172.21.15.123:6836/2950411419 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12aeffd640 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b490610 0x55a47b492a00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ae513af1f4aece2 server_cookie=8ea6c0a290c93457 in_seq=0 out_seq=0 2024-07-24T22:05:31.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f1292ff5640 1 -- 172.21.15.60:0/2542329669 <== mds.2 v2:172.21.15.123:6836/2950411419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a47b37e3a0 con 0x55a47b490610 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f1292ff5640 10 client.19329 handle_command_reply: tid=0 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 client.19329 shutdown 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 msgr2=0x55a47b41e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f12a40e7dd0 tx=0x7f12a40ecf70 comp rx=0 tx=0).stop 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 shutdown_connections 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:6836/2950411419,v1:172.21.15.123:6837/2950411419] conn(0x55a47b490610 0x55a47b492a00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a47b415f80 0x55a47b40f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a47b409ff0 0x55a47b41e5b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 --2- 172.21.15.60:0/2542329669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a47b4096d0 0x55a47b41e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 >> 172.21.15.60:0/2542329669 conn(0x55a47b407500 msgr2=0x55a47b405d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f1292ff5640 0 client.19329 ms_handle_reset on v2:172.21.15.123:6836/2950411419 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 shutdown_connections 2024-07-24T22:05:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.961+0000 7f12b79c2740 1 -- 172.21.15.60:0/2542329669 wait complete. 2024-07-24T22:05:31.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b79c2740 20 client.19329 trim_cache size 0 max 16384 2024-07-24T22:05:31.969 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:05:31.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 msgr2=0x7f12980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 0x7f12980b6200 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005d80 tx=0x7f12a800b420 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 msgr2=0x7f12b019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f12a400d950 tx=0x7f12a400de10 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 shutdown_connections 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f12980b3d40 0x7f12980b6200 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b010ee20 0x7f12b01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12b0101b90 0x7f12b019b420 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 --2- 172.21.15.60:0/677324041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12b0101270 0x7f12b019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 >> 172.21.15.60:0/677324041 conn(0x7f12b00fd0c0 msgr2=0x7f12b00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 shutdown_connections 2024-07-24T22:05:31.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:31.962+0000 7f12b5611640 1 -- 172.21.15.60:0/677324041 wait complete. 2024-07-24T22:05:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:31 smithi123 ceph-mon[37143]: pgmap v969: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:32.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:31 smithi086 ceph-mon[37128]: pgmap v969: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:32.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-24T22:05:32.883 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:33 smithi123 ceph-mon[37143]: pgmap v970: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:34.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:33 smithi086 ceph-mon[37128]: pgmap v970: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:33 smithi060 ceph-mon[33000]: pgmap v970: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:34.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 -- 172.21.15.60:0/3177316603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae181081a0 msgr2=0x7fae181085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 --2- 172.21.15.60:0/3177316603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae181081a0 0x7fae181085e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009900 tx=0x7fae0c02a410 comp rx=0 tx=0).stop 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 -- 172.21.15.60:0/3177316603 shutdown_connections 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 --2- 172.21.15.60:0/3177316603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae18108b20 0x7fae1810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 --2- 172.21.15.60:0/3177316603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae181081a0 0x7fae181085e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 --2- 172.21.15.60:0/3177316603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18102790 0x7fae18102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 -- 172.21.15.60:0/3177316603 >> 172.21.15.60:0/3177316603 conn(0x7fae180fe200 msgr2=0x7fae18100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 -- 172.21.15.60:0/3177316603 shutdown_connections 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.637+0000 7fae1e362640 1 -- 172.21.15.60:0/3177316603 wait complete. 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 Processor -- start 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 -- start start 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae181081a0 0x7fae180753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18108b20 0x7fae18075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae18112730 con 0x7fae18102790 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae181125b0 con 0x7fae18108b20 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1e362640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae181128b0 con 0x7fae181081a0 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38010/0 (socket says 172.21.15.60:38010) 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae17fff640 1 -- 172.21.15.60:0/1080231928 learned_addr learned my addr 172.21.15.60:0/1080231928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.638+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18108b20 0x7fae18075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae177fe640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae181081a0 0x7fae180753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 -- 172.21.15.60:0/1080231928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae181081a0 msgr2=0x7fae180753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae181081a0 0x7fae180753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 -- 172.21.15.60:0/1080231928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18108b20 msgr2=0x7fae18075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18108b20 0x7fae18075910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 -- 172.21.15.60:0/1080231928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181a6fe0 con 0x7fae18102790 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae17fff640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae157fa640 1 -- 172.21.15.60:0/1080231928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08017070 con 0x7fae18102790 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181a72d0 con 0x7fae18102790 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae157fa640 1 -- 172.21.15.60:0/1080231928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae08009c30 con 0x7fae18102790 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae157fa640 1 -- 172.21.15.60:0/1080231928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800f620 con 0x7fae18102790 2024-07-24T22:05:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.639+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a7830 con 0x7fae18102790 2024-07-24T22:05:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae157fa640 1 -- 172.21.15.60:0/1080231928 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fae08010460 con 0x7fae18102790 2024-07-24T22:05:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 Processor -- start 2024-07-24T22:05:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 -- start start 2024-07-24T22:05:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a95da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a960370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a8c89b0 con 0x55ee2a9516d0 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a8b7eb0 con 0x55ee2a951ff0 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae20713740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a8bbef0 con 0x55ee2a95df80 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38016/0 (socket says 172.21.15.60:38016) 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae1c8ff640 1 -- 172.21.15.60:0/2322541041 learned_addr learned my addr 172.21.15.60:0/2322541041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.642+0000 7fae17fff640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a95da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae177fe640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a960370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 -- 172.21.15.60:0/2322541041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 msgr2=0x55ee2a960370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a960370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 -- 172.21.15.60:0/2322541041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 msgr2=0x55ee2a95da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a95da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 -- 172.21.15.60:0/2322541041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2a841320 con 0x55ee2a9516d0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae157fa640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 0x7fae000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae177fe640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a960370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae177fe640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 0x7fae000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fae10007ec0 tx=0x7fae1000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8597feb96584d74 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fae157fa640 1 -- 172.21.15.60:0/1080231928 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fae080e3bc0 con 0x7fae18102790 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fadfe7fc640 1 -- 172.21.15.60:0/2322541041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10010840 con 0x55ee2a9516d0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fadfe7fc640 1 -- 172.21.15.60:0/2322541041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae100183f0 con 0x55ee2a9516d0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.643+0000 7fadfe7fc640 1 -- 172.21.15.60:0/2322541041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10017600 con 0x55ee2a9516d0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- 172.21.15.60:0/2322541041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 msgr2=0x55ee2a951ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae177fe640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 0x7fae000b6200 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fae0c031440 tx=0x7fae0c005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fae10007ec0 tx=0x7fae1000f790 comp rx=0 tx=0).stop 2024-07-24T22:05:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- 172.21.15.60:0/2322541041 shutdown_connections 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a960370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a95da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 --2- 172.21.15.60:0/2322541041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a951ab0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- 172.21.15.60:0/2322541041 >> 172.21.15.60:0/2322541041 conn(0x55ee2a94f500 msgr2=0x55ee2a9506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- 172.21.15.60:0/2322541041 shutdown_connections 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- 172.21.15.60:0/2322541041 wait complete. 2024-07-24T22:05:34.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.644+0000 7fae20713740 1 -- start start 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 Processor -- start 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a9516d0 0x55ee2a95aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a951ff0 0x55ee2a95b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a71b700 con 0x55ee2a95df80 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a85a5f0 con 0x55ee2a9516d0 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee2a960c00 con 0x55ee2a951ff0 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38032/0 (socket says 172.21.15.60:38032) 2024-07-24T22:05:34.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3999218592 learned_addr learned my addr 172.21.15.60:0/3999218592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae20713740 10 client.0 _ll_register_callbacks cb 0x55ee2a87a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae177fe640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a951ff0 0x55ee2a95b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.645+0000 7fae17fff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a9516d0 0x55ee2a95aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3999218592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a951ff0 msgr2=0x55ee2a95b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a951ff0 0x55ee2a95b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a9516d0 msgr2=0x55ee2a95aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a9516d0 0x55ee2a95aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3999218592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2a84d680 con 0x55ee2a95df80 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fae1001a040 tx=0x7fae10007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f79ef927ed9a6f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fadf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10026070 con 0x55ee2a95df80 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fae10003e00 con 0x55ee2a95df80 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae100183f0 con 0x55ee2a95df80 2024-07-24T22:05:34.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae10032050 con 0x55ee2a95df80 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae20713740 10 client.19341 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee2a618650 con 0x55ee2a95df80 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.646+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae100183f0 con 0x55ee2a95df80 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fae100108e0 con 0x55ee2a95df80 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 10 client.19341 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 10 client.19341 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 20 client.19341 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a96d8c0 0x55ee2a96fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 4 client.19341 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2a8ca900 con 0x55ee2a96d8c0 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae17fff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a96d8c0 0x55ee2a96fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fae17fff640 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a96d8c0 0x55ee2a96fcb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f81d1cef9e3cbc5e server_cookie=44e43eeb362dbe31 in_seq=0 out_seq=0 2024-07-24T22:05:34.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.647+0000 7fadf77fe640 10 client.19341 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fadf77fe640 1 -- 172.21.15.60:0/3999218592 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee2a8ca900 con 0x55ee2a96d8c0 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fadf77fe640 10 client.19341 handle_command_reply: tid=0 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 client.19341 shutdown 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 msgr2=0x55ee2a95b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fae1001a040 tx=0x7fae10007ab0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 shutdown_connections 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a96d8c0 0x55ee2a96fcb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a95b920 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a951ff0 0x55ee2a95b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 --2- 172.21.15.60:0/3999218592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a9516d0 0x55ee2a95aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.648+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 >> 172.21.15.60:0/3999218592 conn(0x55ee2a94f500 msgr2=0x55ee2a9550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.649+0000 7fadf77fe640 0 client.19341 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:05:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.649+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 shutdown_connections 2024-07-24T22:05:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.649+0000 7fae20713740 1 -- 172.21.15.60:0/3999218592 wait complete. 2024-07-24T22:05:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.649+0000 7fae20713740 20 client.19341 trim_cache size 0 max 16384 2024-07-24T22:05:34.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.662+0000 7fae20713740 1 Processor -- start 2024-07-24T22:05:34.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.662+0000 7fae20713740 1 -- start start 2024-07-24T22:05:34.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a9516d0 0x55ee2a95cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a9753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae181b2d60 con 0x55ee2a95df80 2024-07-24T22:05:34.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae181b0570 con 0x55ee2a951ff0 2024-07-24T22:05:34.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae20713740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae181ae3e0 con 0x55ee2a9516d0 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38034/0 (socket says 172.21.15.60:38034) 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3059881998 learned_addr learned my addr 172.21.15.60:0/3059881998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae17fff640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a9516d0 0x55ee2a95cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae177fe640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a9753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3059881998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a9516d0 msgr2=0x55ee2a95cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a9516d0 0x55ee2a95cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3059881998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 msgr2=0x55ee2a9753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a9753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 -- 172.21.15.60:0/3059881998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2a9593a0 con 0x55ee2a95df80 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae177fe640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a9753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.663+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fae10007390 tx=0x7fae10007980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d946601fd535a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fadfdffb640 1 -- 172.21.15.60:0/3059881998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10026040 con 0x55ee2a95df80 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fadfdffb640 1 -- 172.21.15.60:0/3059881998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1003b070 con 0x55ee2a95df80 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fadfdffb640 1 -- 172.21.15.60:0/3059881998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10003c00 con 0x55ee2a95df80 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 -- 172.21.15.60:0/3059881998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 msgr2=0x55ee2a975910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fae10007390 tx=0x7fae10007980 comp rx=0 tx=0).stop 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 -- 172.21.15.60:0/3059881998 shutdown_connections 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a95df80 0x55ee2a975910 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a9753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 --2- 172.21.15.60:0/3059881998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a9516d0 0x55ee2a95cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 -- 172.21.15.60:0/3059881998 >> 172.21.15.60:0/3059881998 conn(0x55ee2a94f500 msgr2=0x55ee2a95c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 -- 172.21.15.60:0/3059881998 shutdown_connections 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.664+0000 7fae20713740 1 -- 172.21.15.60:0/3059881998 wait complete. 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 -- start start 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 Processor -- start 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a961ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a962270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae181aea30 con 0x55ee2a9516d0 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae181afea0 con 0x55ee2a951ff0 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae181af100 con 0x55ee2a95df80 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae20713740 10 client.0 _ll_register_callbacks cb 0x7fae10007d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38040/0 (socket says 172.21.15.60:38040) 2024-07-24T22:05:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae177fe640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a961ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.665+0000 7fae17fff640 1 -- 172.21.15.60:0/3100837720 learned_addr learned my addr 172.21.15.60:0/3100837720 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae1c8ff640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a962270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 -- 172.21.15.60:0/3100837720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 msgr2=0x55ee2a962270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a962270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 -- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 msgr2=0x55ee2a961ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a961ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 -- 172.21.15.60:0/3100837720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2a9d73a0 con 0x55ee2a9516d0 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae177fe640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a961ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae17fff640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fae080a7ac0 tx=0x7fae080e60d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d178e929ec474b7f server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fadf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08020040 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fae080cd0b0 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae1003b070 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae080b3190 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae20713740 10 client.19347 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.666+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee2a9d8530 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1003b070 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fae080a61c0 con 0x55ee2a9516d0 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 10 client.19347 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 10 client.19347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 20 client.19347 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a9d8ce0 0x55ee2a9db100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 4 client.19347 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:05:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 --> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee2a8c63a0 con 0x55ee2a9d8ce0 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.667+0000 7fae177fe640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a9d8ce0 0x55ee2a9db100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fadf77fe640 10 client.19347 ms_handle_connect on v2:172.21.15.86:6838/1688764220 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae177fe640 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a9d8ce0 0x55ee2a9db100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0c1d861a9c522c5 server_cookie=f399d6fe23823a5 in_seq=0 out_seq=0 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fadf77fe640 1 -- 172.21.15.60:0/3100837720 <== mds.1 v2:172.21.15.86:6838/1688764220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee2a8c63a0 con 0x55ee2a9d8ce0 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fadf77fe640 10 client.19347 handle_command_reply: tid=0 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 client.19347 shutdown 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 msgr2=0x55ee2a966680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fae080a7ac0 tx=0x7fae080e60d0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 shutdown_connections 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:6838/1688764220,v1:172.21.15.86:6839/1688764220] conn(0x55ee2a9d8ce0 0x55ee2a9db100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fadf77fe640 0 client.19347 ms_handle_reset on v2:172.21.15.86:6838/1688764220 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee2a95df80 0x55ee2a962270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2a951ff0 0x55ee2a961ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 --2- 172.21.15.60:0/3100837720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee2a9516d0 0x55ee2a966680 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 >> 172.21.15.60:0/3100837720 conn(0x55ee2a94f500 msgr2=0x55ee2a95ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 shutdown_connections 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 1 -- 172.21.15.60:0/3100837720 wait complete. 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.668+0000 7fae20713740 20 client.19347 trim_cache size 0 max 16384 2024-07-24T22:05:34.675 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 msgr2=0x7fae000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 0x7fae000b6200 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fae0c031440 tx=0x7fae0c005b50 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 msgr2=0x7fae18074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 shutdown_connections 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fae000b3d40 0x7fae000b6200 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae18108b20 0x7fae18075910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae181081a0 0x7fae180753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 --2- 172.21.15.60:0/1080231928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae18102790 0x7fae18074e90 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 >> 172.21.15.60:0/1080231928 conn(0x7fae180fe200 msgr2=0x7fae181005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:34.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 shutdown_connections 2024-07-24T22:05:34.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:34.669+0000 7fae1e362640 1 -- 172.21.15.60:0/1080231928 wait complete. 2024-07-24T22:05:35.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-24T22:05:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:35.922 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:35 smithi060 ceph-mon[33000]: pgmap v971: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:35 smithi123 ceph-mon[37143]: pgmap v971: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:36.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:35 smithi086 ceph-mon[37128]: pgmap v971: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:37.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- 172.21.15.60:0/1248762038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101230 msgr2=0x7fdeec101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 --2- 172.21.15.60:0/1248762038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101230 0x7fdeec101610 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009900 tx=0x7fdee002a410 comp rx=0 tx=0).stop 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- 172.21.15.60:0/1248762038 shutdown_connections 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 --2- 172.21.15.60:0/1248762038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec10ede0 0x7fdeec1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 --2- 172.21.15.60:0/1248762038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec101b50 0x7fdeec10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 --2- 172.21.15.60:0/1248762038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101230 0x7fdeec101610 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- 172.21.15.60:0/1248762038 >> 172.21.15.60:0/1248762038 conn(0x7fdeec0fd100 msgr2=0x7fdeec0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- 172.21.15.60:0/1248762038 shutdown_connections 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- 172.21.15.60:0/1248762038 wait complete. 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 Processor -- start 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.277+0000 7fdef3064640 1 -- start start 2024-07-24T22:05:37.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec101230 0x7fdeec19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 0x7fdeec1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec114960 con 0x7fdeec101b50 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1147e0 con 0x7fdeec10ede0 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef3064640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec114ae0 con 0x7fdeec101230 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33298/0 (socket says 172.21.15.60:33298) 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 -- 172.21.15.60:0/2672492282 learned_addr learned my addr 172.21.15.60:0/2672492282 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef0e00640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec101230 0x7fdeec19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef1601640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 0x7fdeec1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 -- 172.21.15.60:0/2672492282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec101230 msgr2=0x7fdeec19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec101230 0x7fdeec19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 -- 172.21.15.60:0/2672492282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 msgr2=0x7fdeec1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 0x7fdeec1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 -- 172.21.15.60:0/2672492282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec1a0f40 con 0x7fdeec101b50 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdef1601640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 0x7fdeec1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdeebfff640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fded800c890 tx=0x7fded800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.278+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2672492282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004220 con 0x7fdeec101b50 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.279+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec1a1230 con 0x7fdeec101b50 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.279+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2672492282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded80043c0 con 0x7fdeec101b50 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.279+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2672492282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004a00 con 0x7fdeec101b50 2024-07-24T22:05:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.279+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec1a78f0 con 0x7fdeec101b50 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.281+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2672492282 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fded80079b0 con 0x7fdeec101b50 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.281+0000 7fdef5415740 1 Processor -- start 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.281+0000 7fdef5415740 1 -- start start 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4dafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dbba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5575e4d269b0 con 0x5575e4dbbf80 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575e4d15eb0 con 0x5575e4dafff0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef5415740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5575e4d19ef0 con 0x5575e4daf6d0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33306/0 (socket says 172.21.15.60:33306) 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 -- 172.21.15.60:0/1673091831 learned_addr learned my addr 172.21.15.60:0/1673091831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef0e00640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dbba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef1601640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4dafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdee9ffb640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 0x7fdedc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef1601640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 0x7fdedc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 -- 172.21.15.60:0/1673091831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 msgr2=0x5575e4dafab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4dafab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 -- 172.21.15.60:0/1673091831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 msgr2=0x5575e4dbba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dbba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdeebfff640 1 -- 172.21.15.60:0/1673091831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e4c9f320 con 0x5575e4dbbf80 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdef0e00640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dbba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.282+0000 7fdee9ffb640 1 -- 172.21.15.60:0/2672492282 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fded80e3e10 con 0x7fdeec101b50 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.283+0000 7fdeebfff640 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fded80e1410 tx=0x7fded8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ce370429c5a965 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.283+0000 7fded6ffd640 1 -- 172.21.15.60:0/1673091831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded801d070 con 0x5575e4dbbf80 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.283+0000 7fdef1601640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 0x7fdedc0b6200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdeec102210 tx=0x7fdee4004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.283+0000 7fded6ffd640 1 -- 172.21.15.60:0/1673091831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded8005290 con 0x5575e4dbbf80 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.283+0000 7fded6ffd640 1 -- 172.21.15.60:0/1673091831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80ab090 con 0x5575e4dbbf80 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- 172.21.15.60:0/1673091831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 msgr2=0x5575e4dbe370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fded80e1410 tx=0x7fded8000bc0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- 172.21.15.60:0/1673091831 shutdown_connections 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dbe370 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dbba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 --2- 172.21.15.60:0/1673091831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4dafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- 172.21.15.60:0/1673091831 >> 172.21.15.60:0/1673091831 conn(0x5575e4dad500 msgr2=0x5575e4dae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- 172.21.15.60:0/1673091831 shutdown_connections 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- 172.21.15.60:0/1673091831 wait complete. 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 -- start start 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.284+0000 7fdef5415740 1 Processor -- start 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4db8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dbbf80 0x5575e4db9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5575e4b79700 con 0x5575e4dafff0 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575e4cb85f0 con 0x5575e4dbbf80 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5575e4dbeba0 con 0x5575e4daf6d0 2024-07-24T22:05:37.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef5415740 10 client.0 _ll_register_callbacks cb 0x5575e4cd8960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33320/0 (socket says 172.21.15.60:33320) 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 -- 172.21.15.60:0/882756503 learned_addr learned my addr 172.21.15.60:0/882756503 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef0e00640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4db8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdef1601640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dbbf80 0x5575e4db9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 -- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 msgr2=0x5575e4db8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4db8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 -- 172.21.15.60:0/882756503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dbbf80 msgr2=0x5575e4db9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dbbf80 0x5575e4db9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 -- 172.21.15.60:0/882756503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e4cab680 con 0x5575e4dafff0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdeebfff640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fded8005840 tx=0x7fded8007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0d924da1cca237 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.285+0000 7fdecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-07-24T22:05:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded801d040 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fded80a6300 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575e4d28900 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fded80aa2e0 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 10 client.19359 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5575e4a76650 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575e4d28900 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fded80e1750 con 0x5575e4dafff0 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 10 client.19359 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 10 client.19359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 20 client.19359 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4dcb880 0x5575e4dcdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 4 client.19359 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-24T22:05:37.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.286+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e4d243a0 con 0x5575e4dcb880 2024-07-24T22:05:37.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.287+0000 7fdef0e00640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4dcb880 0x5575e4dcdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.287+0000 7fdef0e00640 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4dcb880 0x5575e4dcdc70 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75602b6fa4085d5f server_cookie=ec42297da65efb41 in_seq=0 out_seq=0 2024-07-24T22:05:37.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.287+0000 7fdecbfff640 10 client.19359 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdecbfff640 1 -- 172.21.15.60:0/882756503 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575e4d243a0 con 0x5575e4dcb880 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdecbfff640 10 client.19359 handle_command_reply: tid=0 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 client.19359 shutdown 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 msgr2=0x5575e4db9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fded8005840 tx=0x7fded8007ed0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 shutdown_connections 2024-07-24T22:05:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4dcb880 0x5575e4dcdc70 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dbbf80 0x5575e4db9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dafff0 0x5575e4db9430 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 --2- 172.21.15.60:0/882756503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4daf6d0 0x5575e4db8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdecbfff640 0 client.19359 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 >> 172.21.15.60:0/882756503 conn(0x5575e4dad500 msgr2=0x5575e4db30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 shutdown_connections 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 1 -- 172.21.15.60:0/882756503 wait complete. 2024-07-24T22:05:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.288+0000 7fdef5415740 20 client.19359 trim_cache size 0 max 16384 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 Processor -- start 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 -- start start 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dd0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dbbf80 0x5575e4dd11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1b2e40 con 0x5575e4daf6d0 2024-07-24T22:05:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1b0650 con 0x5575e4dafff0 2024-07-24T22:05:37.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef5415740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1ae4c0 con 0x5575e4dbbf80 2024-07-24T22:05:37.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33330/0 (socket says 172.21.15.60:33330) 2024-07-24T22:05:37.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.302+0000 7fdef0e00640 1 -- 172.21.15.60:0/2915993264 learned_addr learned my addr 172.21.15.60:0/2915993264 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef1601640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dbbf80 0x5575e4dd11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 -- 172.21.15.60:0/2915993264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dbbf80 msgr2=0x5575e4dd11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dbbf80 0x5575e4dd11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdeebfff640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dd0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 -- 172.21.15.60:0/2915993264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 msgr2=0x5575e4dd0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dd0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 -- 172.21.15.60:0/2915993264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e4dafe20 con 0x5575e4daf6d0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdeebfff640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dd0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fdef0e00640 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdee002a930 tx=0x7fdee0005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8057ab1df99f35 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fded67fc640 1 -- 172.21.15.60:0/2915993264 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0035040 con 0x5575e4daf6d0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fded67fc640 1 -- 172.21.15.60:0/2915993264 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee0005040 con 0x5575e4daf6d0 2024-07-24T22:05:37.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.303+0000 7fded67fc640 1 -- 172.21.15.60:0/2915993264 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004ad0 con 0x5575e4daf6d0 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 -- 172.21.15.60:0/2915993264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 msgr2=0x5575e4dc7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdee002a930 tx=0x7fdee0005e60 comp rx=0 tx=0).stop 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 -- 172.21.15.60:0/2915993264 shutdown_connections 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dbbf80 0x5575e4dd11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4dafff0 0x5575e4dd0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 --2- 172.21.15.60:0/2915993264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4daf6d0 0x5575e4dc7540 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 -- 172.21.15.60:0/2915993264 >> 172.21.15.60:0/2915993264 conn(0x5575e4dad500 msgr2=0x5575e4dc6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 -- 172.21.15.60:0/2915993264 shutdown_connections 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.304+0000 7fdef5415740 1 -- 172.21.15.60:0/2915993264 wait complete. 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 -- start start 2024-07-24T22:05:37.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 Processor -- start 2024-07-24T22:05:37.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 0x5575e4dce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1aeb10 con 0x5575e4dbbf80 2024-07-24T22:05:37.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1aff80 con 0x5575e4daf6d0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec1af1e0 con 0x5575e4dafff0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.305+0000 7fdef5415740 10 client.0 _ll_register_callbacks cb 0x7fdee0005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef1601640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 0x5575e4dce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.60:46686/0 (socket says 172.21.15.60:46686) 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 -- 172.21.15.60:0/396837966 learned_addr learned my addr 172.21.15.60:0/396837966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 0x5575e4dce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:44278/0 (socket says 172.21.15.60:44278) 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef1601640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33344/0 (socket says 172.21.15.60:33344) 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 -- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 msgr2=0x5575e4dce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 0x5575e4dce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 -- 172.21.15.60:0/396837966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 msgr2=0x5575e4dceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-24T22:05:37.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 -- 172.21.15.60:0/396837966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e4dcf1a0 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef0e00640 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdee0031440 tx=0x7fdee00051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4ca7bce5fee773 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0035070 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fdee0032030 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.306+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee0005040 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdee005e050 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 10 client.28634 fetch_fsmap learned FSMap version 46 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5575e4e3e790 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee0005040 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 8370+0+0 (secure 0 0 0) 0x7fdee0031650 con 0x5575e4daf6d0 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 10 client.28634 fetch_fsmap finished waiting for FSMap version 46 2024-07-24T22:05:37.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 10 client.28634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 20 client.28634 populate_metadata read hostname 'smithi060' 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4e3ef00 0x5575e4e41320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 4 client.28634 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.307+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 --> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575e4d21ae0 con 0x5575e4e3ef00 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdeebfff640 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4e3ef00 0x5575e4e41320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdeebfff640 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4e3ef00 0x5575e4e41320 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b82570ce66bd44 server_cookie=f722d8c7c504d173 in_seq=0 out_seq=0 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdecbfff640 10 client.28634 ms_handle_connect on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdecbfff640 1 -- 172.21.15.60:0/396837966 <== mds.0 v2:172.21.15.123:6838/3718566677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575e4d21ae0 con 0x5575e4e3ef00 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdecbfff640 10 client.28634 handle_command_reply: tid=0 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 client.28634 shutdown 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 msgr2=0x5575e4dd1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdee0031440 tx=0x7fdee00051f0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 shutdown_connections 2024-07-24T22:05:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:6838/3718566677,v1:172.21.15.123:6839/3718566677] conn(0x5575e4e3ef00 0x5575e4e41320 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5575e4dbbf80 0x5575e4dceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdecbfff640 0 client.28634 ms_handle_reset on v2:172.21.15.123:6838/3718566677 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575e4dafff0 0x5575e4dce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 --2- 172.21.15.60:0/396837966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575e4daf6d0 0x5575e4dd1130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.308+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 >> 172.21.15.60:0/396837966 conn(0x5575e4dad500 msgr2=0x5575e4db7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 shutdown_connections 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef5415740 1 -- 172.21.15.60:0/396837966 wait complete. 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef5415740 20 client.28634 trim_cache size 0 max 16384 2024-07-24T22:05:37.318 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-07-24T22:05:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 msgr2=0x7fdedc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 0x7fdedc0b6200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdeec102210 tx=0x7fdee4004ab0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 msgr2=0x7fdeec19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fded800c890 tx=0x7fded800cd50 comp rx=0 tx=0).stop 2024-07-24T22:05:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 shutdown_connections 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdedc0b3d40 0x7fdedc0b6200 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeec10ede0 0x7fdeec1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeec101b50 0x7fdeec19b3d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 --2- 172.21.15.60:0/2672492282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdeec101230 0x7fdeec19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.309+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 >> 172.21.15.60:0/2672492282 conn(0x7fdeec0fd100 msgr2=0x7fdeec0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.310+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 shutdown_connections 2024-07-24T22:05:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:37.310+0000 7fdef3064640 1 -- 172.21.15.60:0/2672492282 wait complete. 2024-07-24T22:05:37.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:37 smithi060 ceph-mon[33000]: pgmap v972: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:37 smithi123 ceph-mon[37143]: pgmap v972: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:38.011 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-07-24T22:05:38.012 INFO:tasks.fwd_scrub:done joining 2024-07-24T22:05:38.012 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-07-24T22:05:38.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-24T22:05:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:37 smithi086 ceph-mon[37128]: pgmap v972: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 -- 172.21.15.60:0/3970312587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0106fe0 msgr2=0x7f80a0107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 --2- 172.21.15.60:0/3970312587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0106fe0 0x7f80a0107420 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8088009900 tx=0x7f808802a410 comp rx=0 tx=0).stop 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 -- 172.21.15.60:0/3970312587 shutdown_connections 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 --2- 172.21.15.60:0/3970312587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0107960 0x7f80a010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 --2- 172.21.15.60:0/3970312587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0106fe0 0x7f80a0107420 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 --2- 172.21.15.60:0/3970312587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 0x7f80a01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 -- 172.21.15.60:0/3970312587 >> 172.21.15.60:0/3970312587 conn(0x7f80a00fd020 msgr2=0x7f80a00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 -- 172.21.15.60:0/3970312587 shutdown_connections 2024-07-24T22:05:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.135+0000 7f80a4c31640 1 -- 172.21.15.60:0/3970312587 wait complete. 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 Processor -- start 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 -- start start 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 0x7f80a00fd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 0x7f80a00fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01115b0 con 0x7f80a0107960 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0111430 con 0x7f80a01015e0 2024-07-24T22:05:38.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f80a4c31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0111730 con 0x7f80a0106fe0 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33358/0 (socket says 172.21.15.60:33358) 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 -- 172.21.15.60:0/2632586546 learned_addr learned my addr 172.21.15.60:0/2632586546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809e59c640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 0x7f80a00fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809dd9b640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 0x7f80a00fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 -- 172.21.15.60:0/2632586546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 msgr2=0x7f80a00fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 0x7f80a00fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 -- 172.21.15.60:0/2632586546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 msgr2=0x7f80a00fd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 0x7f80a00fd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809ed9d640 1 -- 172.21.15.60:0/2632586546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a01a7010 con 0x7f80a0107960 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.136+0000 7f809dd9b640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 0x7f80a00fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f809ed9d640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f809800eaa0 tx=0x7f809800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809800cdb0 con 0x7f80a0107960 2024-07-24T22:05:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a01a7300 con 0x7f80a0107960 2024-07-24T22:05:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8098005020 con 0x7f80a0107960 2024-07-24T22:05:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80980116d0 con 0x7f80a0107960 2024-07-24T22:05:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.137+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01a7850 con 0x7f80a0107960 2024-07-24T22:05:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.138+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a01019c0 con 0x7f80a0107960 2024-07-24T22:05:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.140+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f80980054a0 con 0x7f80a0107960 2024-07-24T22:05:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.140+0000 7f80977fe640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 0x7f808c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.141+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f80980e4660 con 0x7f80a0107960 2024-07-24T22:05:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.141+0000 7f809e59c640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 0x7f808c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.141+0000 7f809e59c640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 0x7f808c0b6200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80900058c0 tx=0x7f809000a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.144+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8098016030 con 0x7f80a0107960 2024-07-24T22:05:38.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.340+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80a01a7c70 con 0x7f80a0107960 2024-07-24T22:05:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.341+0000 7f80977fe640 1 -- 172.21.15.60:0/2632586546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+23165 (secure 0 0 0) 0x7f80980ab940 con 0x7f80a0107960 2024-07-24T22:05:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:05:38.344 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":114,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T22:04:48.237225+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T22:05:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 msgr2=0x7f808c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 0x7f808c0b6200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f80900058c0 tx=0x7f809000a4c0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 msgr2=0x7f80a00fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f809800eaa0 tx=0x7f809800ef60 comp rx=0 tx=0).stop 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 shutdown_connections 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f808c0b3d40 0x7f808c0b6200 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80a0107960 0x7f80a00fdfa0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a0106fe0 0x7f80a00fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 --2- 172.21.15.60:0/2632586546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01015e0 0x7f80a00fd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 >> 172.21.15.60:0/2632586546 conn(0x7f80a00fd020 msgr2=0x7f80a006aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 shutdown_connections 2024-07-24T22:05:38.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.343+0000 7f80a4c31640 1 -- 172.21.15.60:0/2632586546 wait complete. 2024-07-24T22:05:38.357 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-24T21:38:46.032062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-24T21:43:07.304504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-24T21:43:10.965950+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-24T22:05:38.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-24T22:05:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 -- 172.21.15.60:0/4114350954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0568103ef0 msgr2=0x7f056810c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 --2- 172.21.15.60:0/4114350954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0568103ef0 0x7f056810c3d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0554009900 tx=0x7f055402a410 comp rx=0 tx=0).stop 2024-07-24T22:05:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 -- 172.21.15.60:0/4114350954 shutdown_connections 2024-07-24T22:05:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 --2- 172.21.15.60:0/4114350954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f056810ca20 0x7f056810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 --2- 172.21.15.60:0/4114350954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0568103ef0 0x7f056810c3d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 --2- 172.21.15.60:0/4114350954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05681035d0 0x7f05681039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 -- 172.21.15.60:0/4114350954 >> 172.21.15.60:0/4114350954 conn(0x7f05680fd020 msgr2=0x7f05680ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 -- 172.21.15.60:0/4114350954 shutdown_connections 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.463+0000 7f056f10c640 1 -- 172.21.15.60:0/4114350954 wait complete. 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 Processor -- start 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 -- start start 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568103ef0 0x7f0568199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056810ca20 0x7f056819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05681126f0 con 0x7f05681035d0 2024-07-24T22:05:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0568112570 con 0x7f056810ca20 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056f10c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0568112870 con 0x7f0568103ef0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056cea8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056cea8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33386/0 (socket says 172.21.15.60:33386) 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.464+0000 7f056cea8640 1 -- 172.21.15.60:0/1903814052 learned_addr learned my addr 172.21.15.60:0/1903814052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f0567fff640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568103ef0 0x7f0568199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056d6a9640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056810ca20 0x7f056819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 -- 172.21.15.60:0/1903814052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568103ef0 msgr2=0x7f0568199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568103ef0 0x7f0568199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 -- 172.21.15.60:0/1903814052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056810ca20 msgr2=0x7f056819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056810ca20 0x7f056819e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 -- 172.21.15.60:0/1903814052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819ec90 con 0x7f05681035d0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056cea8640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c920 tx=0x7f055c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c007c70 con 0x7f05681035d0 2024-07-24T22:05:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819ef80 con 0x7f05681035d0 2024-07-24T22:05:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c007e10 con 0x7f05681035d0 2024-07-24T22:05:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c018820 con 0x7f05681035d0 2024-07-24T22:05:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.465+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681a5820 con 0x7f05681035d0 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.466+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05681039b0 con 0x7f05681035d0 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.468+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f055c00f450 con 0x7f05681035d0 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.468+0000 7f0565ffb640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 0x7f05580b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.468+0000 7f0567fff640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 0x7f05580b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.469+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f055c0e40e0 con 0x7f05681035d0 2024-07-24T22:05:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.469+0000 7f0567fff640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 0x7f05580b6200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0554000c00 tx=0x7f0554005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.472+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f055c014030 con 0x7f05681035d0 2024-07-24T22:05:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.669+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05681a5ac0 con 0x7f05681035d0 2024-07-24T22:05:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.669+0000 7f0565ffb640 1 -- 172.21.15.60:0/1903814052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f055c0ab3c0 con 0x7f05681035d0 2024-07-24T22:05:38.671 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-07-24T22:05:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 msgr2=0x7f05580b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 0x7f05580b6200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0554000c00 tx=0x7f0554005e80 comp rx=0 tx=0).stop 2024-07-24T22:05:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 msgr2=0x7f0568198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c920 tx=0x7f055c00cde0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 shutdown_connections 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f05580b3d40 0x7f05580b6200 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056810ca20 0x7f056819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568103ef0 0x7f0568199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 --2- 172.21.15.60:0/1903814052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05681035d0 0x7f0568198bf0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 >> 172.21.15.60:0/1903814052 conn(0x7f05680fd020 msgr2=0x7f056810ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.671+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 shutdown_connections 2024-07-24T22:05:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.672+0000 7f056f10c640 1 -- 172.21.15.60:0/1903814052 wait complete. 2024-07-24T22:05:38.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- 172.21.15.60:0/1987042292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 msgr2=0x7fdaf41073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 --2- 172.21.15.60:0/1987042292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf41073d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a410 comp rx=0 tx=0).stop 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- 172.21.15.60:0/1987042292 shutdown_connections 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 --2- 172.21.15.60:0/1987042292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4107910 0x7fdaf410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 --2- 172.21.15.60:0/1987042292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf41073d0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 --2- 172.21.15.60:0/1987042292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4101590 0x7fdaf4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- 172.21.15.60:0/1987042292 >> 172.21.15.60:0/1987042292 conn(0x7fdaf40fcfb0 msgr2=0x7fdaf40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- 172.21.15.60:0/1987042292 shutdown_connections 2024-07-24T22:05:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- 172.21.15.60:0/1987042292 wait complete. 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 Processor -- start 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.791+0000 7fdafba38640 1 -- start start 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 0x7fdaf4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4107910 0x7fdaf419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41126f0 con 0x7fdaf4106f90 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112570 con 0x7fdaf4101590 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdafba38640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112870 con 0x7fdaf4107910 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33404/0 (socket says 172.21.15.60:33404) 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 -- 172.21.15.60:0/1467467463 learned_addr learned my addr 172.21.15.60:0/1467467463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf97d4640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 0x7fdaf4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf9fd5640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4107910 0x7fdaf419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 -- 172.21.15.60:0/1467467463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4107910 msgr2=0x7fdaf419e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4107910 0x7fdaf419e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 -- 172.21.15.60:0/1467467463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 msgr2=0x7fdaf4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 0x7fdaf4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 -- 172.21.15.60:0/1467467463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf419ec90 con 0x7fdaf4106f90 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf97d4640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 0x7fdaf4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.792+0000 7fdaf8fd3640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005ab0 tx=0x7fdae4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.793+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae403f070 con 0x7fdaf4106f90 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.793+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae4031d60 con 0x7fdaf4106f90 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.793+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf419ef20 con 0x7fdaf4106f90 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.793+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035030 con 0x7fdaf4106f90 2024-07-24T22:05:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.793+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a5640 con 0x7fdaf4106f90 2024-07-24T22:05:38.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.794+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf4101970 con 0x7fdaf4106f90 2024-07-24T22:05:38.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.795+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fdae4042070 con 0x7fdaf4106f90 2024-07-24T22:05:38.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.797+0000 7fdaea7fc640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 0x7fdae00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:38.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.797+0000 7fdaf97d4640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 0x7fdae00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:38.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.797+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fdae4104e60 con 0x7fdaf4106f90 2024-07-24T22:05:38.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.797+0000 7fdaf97d4640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 0x7fdae00b6280 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00058c0 tx=0x7fdaf0005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:38.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.800+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fdae40cc140 con 0x7fdaf4106f90 2024-07-24T22:05:38.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:38 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2632586546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:05:38.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.991+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdaf41096f0 con 0x7fdaf4106f90 2024-07-24T22:05:38.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.992+0000 7fdaea7fc640 1 -- 172.21.15.60:0/1467467463 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7fdae40d4410 con 0x7fdaf4106f90 2024-07-24T22:05:38.994 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 msgr2=0x7fdae00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 0x7fdae00b6280 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00058c0 tx=0x7fdaf0005850 comp rx=0 tx=0).stop 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 msgr2=0x7fdaf4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005ab0 tx=0x7fdae4003c80 comp rx=0 tx=0).stop 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 shutdown_connections 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fdae00b3dc0 0x7fdae00b6280 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaf4107910 0x7fdaf419e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaf4106f90 0x7fdaf4199130 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 --2- 172.21.15.60:0/1467467463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4101590 0x7fdaf4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:38.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 >> 172.21.15.60:0/1467467463 conn(0x7fdaf40fcfb0 msgr2=0x7fdaf40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:38.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 shutdown_connections 2024-07-24T22:05:38.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:38.994+0000 7fdafba38640 1 -- 172.21.15.60:0/1467467463 wait complete. 2024-07-24T22:05:39.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:38 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2632586546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:05:39.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-24T22:05:39.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:38 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2632586546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:05:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2694230823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 msgr2=0x7f4dcc10c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2694230823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc10c3d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002a410 comp rx=0 tx=0).stop 2024-07-24T22:05:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2694230823 shutdown_connections 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2694230823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc10ca20 0x7f4dcc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2694230823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc10c3d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2694230823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc1035d0 0x7f4dcc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2694230823 >> 172.21.15.60:0/2694230823 conn(0x7f4dcc0fd020 msgr2=0x7f4dcc0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2694230823 shutdown_connections 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.112+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2694230823 wait complete. 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 Processor -- start 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 -- start start 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1035d0 0x7f4dcc198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc10ca20 0x7f4dcc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1126f0 con 0x7f4dcc103ef0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112570 con 0x7f4dcc1035d0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd35d0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112870 con 0x7f4dcc10ca20 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd0b6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd0b6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33426/0 (socket says 172.21.15.60:33426) 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.113+0000 7f4dd0b6b640 1 -- 172.21.15.60:0/2279948497 learned_addr learned my addr 172.21.15.60:0/2279948497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd1b6d640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc10ca20 0x7f4dcc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd136c640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1035d0 0x7f4dcc198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 -- 172.21.15.60:0/2279948497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc10ca20 msgr2=0x7f4dcc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc10ca20 0x7f4dcc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 -- 172.21.15.60:0/2279948497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1035d0 msgr2=0x7f4dcc198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1035d0 0x7f4dcc198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 -- 172.21.15.60:0/2279948497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc19ed90 con 0x7f4dcc103ef0 2024-07-24T22:05:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd0b6b640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005ab0 tx=0x7f4dc0003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc003f070 con 0x7f4dcc103ef0 2024-07-24T22:05:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc0031b50 con 0x7f4dcc103ef0 2024-07-24T22:05:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc19f020 con 0x7f4dcc103ef0 2024-07-24T22:05:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0035030 con 0x7f4dcc103ef0 2024-07-24T22:05:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.114+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a5740 con 0x7f4dcc103ef0 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.115+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc1039b0 con 0x7f4dcc103ef0 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.117+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f4dc0042070 con 0x7f4dcc103ef0 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.117+0000 7f4dbe7fc640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 0x7f4db80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.117+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7f4dc01049e0 con 0x7f4dcc103ef0 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.117+0000 7f4dd136c640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 0x7f4db80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.118+0000 7f4dd136c640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 0x7f4db80b6200 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009b60 tx=0x7f4dc8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.120+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f4dc00cbcc0 con 0x7f4dcc103ef0 2024-07-24T22:05:39.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.314+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4dcc02e000 con 0x7f4dcc103ef0 2024-07-24T22:05:39.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.314+0000 7f4dbe7fc640 1 -- 172.21.15.60:0/2279948497 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7f4dc00cb5c0 con 0x7f4dcc103ef0 2024-07-24T22:05:39.316 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-07-24T22:05:39.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 msgr2=0x7f4db80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 0x7f4db80b6200 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009b60 tx=0x7f4dc8005f90 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 msgr2=0x7f4dcc199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005ab0 tx=0x7f4dc0003ae0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 shutdown_connections 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f4db80b3d40 0x7f4db80b6200 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dcc10ca20 0x7f4dcc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dcc103ef0 0x7f4dcc199230 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 --2- 172.21.15.60:0/2279948497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dcc1035d0 0x7f4dcc198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 >> 172.21.15.60:0/2279948497 conn(0x7f4dcc0fd020 msgr2=0x7f4dcc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 shutdown_connections 2024-07-24T22:05:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.316+0000 7f4dd35d0640 1 -- 172.21.15.60:0/2279948497 wait complete. 2024-07-24T22:05:39.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- 172.21.15.60:0/3676481132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba9810ca40 msgr2=0x7fba9810eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 --2- 172.21.15.60:0/3676481132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba9810ca40 0x7fba9810eec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fba90009ca0 tx=0x7fba9002a8c0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- 172.21.15.60:0/3676481132 shutdown_connections 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 --2- 172.21.15.60:0/3676481132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba9810ca40 0x7fba9810eec0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 --2- 172.21.15.60:0/3676481132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba9810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 --2- 172.21.15.60:0/3676481132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba981035f0 0x7fba981039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- 172.21.15.60:0/3676481132 >> 172.21.15.60:0/3676481132 conn(0x7fba980fd020 msgr2=0x7fba980ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:39.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- 172.21.15.60:0/3676481132 shutdown_connections 2024-07-24T22:05:39.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- 172.21.15.60:0/3676481132 wait complete. 2024-07-24T22:05:39.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 Processor -- start 2024-07-24T22:05:39.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.433+0000 7fba9da79640 1 -- start start 2024-07-24T22:05:39.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba98199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 0x7fba9819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fba981126c0 con 0x7fba981035f0 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba98112540 con 0x7fba98103f10 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba98112840 con 0x7fba9810ca40 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33442/0 (socket says 172.21.15.60:33442) 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 -- 172.21.15.60:0/2146588652 learned_addr learned my addr 172.21.15.60:0/2146588652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba97fff640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 0x7fba9819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba96ffd640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba98199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 -- 172.21.15.60:0/2146588652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 msgr2=0x7fba9819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 0x7fba9819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 -- 172.21.15.60:0/2146588652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 msgr2=0x7fba98199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba98199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 -- 172.21.15.60:0/2146588652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9819ecc0 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba97fff640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 0x7fba9819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba96ffd640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba98199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba977fe640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fba8800e9f0 tx=0x7fba8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8800cd80 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba880054d0 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9819efb0 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88010690 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.434+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba981a5670 con 0x7fba981035f0 2024-07-24T22:05:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.436+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba981039d0 con 0x7fba981035f0 2024-07-24T22:05:39.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.437+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fba88004e50 con 0x7fba981035f0 2024-07-24T22:05:39.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.438+0000 7fba94ff9640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 0x7fba800b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:05:39.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.438+0000 7fba96ffd640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 0x7fba800b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:05:39.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.438+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8565+0+0 (secure 0 0 0) 0x7fba880e47b0 con 0x7fba981035f0 2024-07-24T22:05:39.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.438+0000 7fba96ffd640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 0x7fba800b6200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c005f20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:05:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.441+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fba880555e0 con 0x7fba981035f0 2024-07-24T22:05:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.688+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fba98199d40 con 0x7fba981035f0 2024-07-24T22:05:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.696+0000 7fba94ff9640 1 -- 172.21.15.60:0/2146588652 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+821908 (secure 0 0 0) 0x7fba880aba90 con 0x7fba981035f0 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 25, 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "active_gid": 14214, 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "active_name": "x", 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addrs": { 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6800", 2024-07-24T22:05:39.708 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1194501493 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6801", 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1194501493 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addr": "172.21.15.60:6801/1194501493", 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "active_change": "2024-07-24T21:36:35.954770+0000", 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "active_mgr_features": 4540701547738038271, 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "available": true, 2024-07-24T22:05:39.709 INFO:teuthology.orchestra.run.smithi060.stdout: "standbys": [ 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24139, 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "z", 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.710 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.712 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.714 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.720 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.723 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.725 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-07-24T22:05:39.726 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-07-24T22:05:39.727 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.729 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-24T22:05:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.734 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.735 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.736 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.737 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.739 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.740 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:39.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.743 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-07-24T22:05:39.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-07-24T22:05:39.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.747 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-07-24T22:05:39.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-07-24T22:05:39.749 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:39.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.753 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.754 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.756 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-07-24T22:05:39.760 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:39.761 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-07-24T22:05:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-07-24T22:05:39.763 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.767 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nginx": { 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nginx", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.768 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nginx container image", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.771 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba": { 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-24T22:05:39.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB container image", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.776 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-07-24T22:05:39.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-07-24T22:05:39.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-07-24T22:05:39.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-07-24T22:05:39.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-07-24T22:05:39.781 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "grafana_dashboards_path": { 2024-07-24T22:05:39.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "grafana_dashboards_path", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-07-24T22:05:39.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.787 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.788 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.791 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-07-24T22:05:39.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.795 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.796 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.799 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-24T22:05:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-24T22:05:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-07-24T22:05:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-24T22:05:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-24T22:05:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-24T22:05:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-24T22:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "[]", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "MULTICLUSTER_CONFIG": { 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "{}", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-24T22:05:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-24T22:05:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-24T22:05:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-24T22:05:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-07-24T22:05:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "UNSAFE_TLS_v1_2": { 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-07-24T22:05:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-07-24T22:05:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-07-24T22:05:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-07-24T22:05:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-07-24T22:05:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-07-24T22:05:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-07-24T22:05:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-07-24T22:05:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.942 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.944 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.945 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.948 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.950 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.951 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-07-24T22:05:39.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.957 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.958 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.959 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-07-24T22:05:39.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-07-24T22:05:39.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.962 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-07-24T22:05:39.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-07-24T22:05:39.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.968 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.970 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.971 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.974 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.975 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.980 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.981 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.985 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.986 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.991 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.993 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.995 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:39.998 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:39.999 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-07-24T22:05:40.002 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.003 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.004 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.006 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.008 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.010 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.012 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-07-24T22:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.019 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.020 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.023 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.026 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.027 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.028 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.030 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.034 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-07-24T22:05:40.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-24T22:05:40.037 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.038 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.039 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.040 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.041 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.043 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:40.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.046 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.047 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.048 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.050 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.054 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.055 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-07-24T22:05:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:40.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.059 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.067 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.068 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.069 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.070 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "secondary_zone_period_retry_limit": { 2024-07-24T22:05:40.072 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.073 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.075 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.076 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.081 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.082 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.083 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.087 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.088 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.091 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.093 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.095 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.096 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-07-24T22:05:40.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-07-24T22:05:40.100 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smb", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "internal_store_backend": { 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "internal_store_backend", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.102 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.106 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.108 INFO:teuthology.orchestra.run.smithi060.stdout: "update_orchestration": { 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "update_orchestration", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.109 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.110 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.113 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.114 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.116 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.120 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.122 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.123 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.126 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.129 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.130 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.132 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.134 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.136 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.137 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.138 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.139 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-07-24T22:05:40.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-07-24T22:05:40.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-07-24T22:05:40.142 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-07-24T22:05:40.143 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.144 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.145 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.146 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-07-24T22:05:40.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:40.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.149 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.150 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.151 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.152 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.154 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.155 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.158 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.159 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.166 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.167 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.168 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-07-24T22:05:40.169 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.170 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.171 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.172 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.174 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-24T22:05:40.175 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.176 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.177 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.178 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-07-24T22:05:40.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-07-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.183 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.185 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.186 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.187 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.188 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.193 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24152, 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "y", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.196 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.197 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.198 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.200 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.202 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.203 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-07-24T22:05:40.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-07-24T22:05:40.205 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-07-24T22:05:40.206 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-07-24T22:05:40.207 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.209 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-07-24T22:05:40.210 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:40.211 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-24T22:05:40.212 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.213 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.214 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-07-24T22:05:40.215 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-07-24T22:05:40.216 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.218 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.221 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.222 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.223 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.224 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.226 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-24T22:05:40.227 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.228 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.229 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.230 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-07-24T22:05:40.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-07-24T22:05:40.232 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.233 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.236 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.238 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.241 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.242 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-24T22:05:40.243 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-24T22:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.246 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.247 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.248 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.249 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.251 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nginx": { 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nginx", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.252 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nginx container image", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-07-24T22:05:40.253 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-07-24T22:05:40.254 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-07-24T22:05:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.256 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.257 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba": { 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB container image", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.259 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.260 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.261 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.264 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.265 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.266 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "grafana_dashboards_path": { 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "grafana_dashboards_path", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-24T22:05:40.267 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.268 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.269 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.271 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.272 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.273 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.274 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.275 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-24T22:05:40.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-24T22:05:40.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.279 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-07-24T22:05:40.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-07-24T22:05:40.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:40.282 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.283 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.284 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-24T22:05:40.286 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.287 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.288 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.289 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.290 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.291 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.292 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.293 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.294 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.295 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.296 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.297 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.298 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.299 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.300 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.301 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.302 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.303 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.306 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-24T22:05:40.307 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-24T22:05:40.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-07-24T22:05:40.309 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-24T22:05:40.310 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-07-24T22:05:40.311 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-24T22:05:40.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-07-24T22:05:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-24T22:05:40.314 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-24T22:05:40.315 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.316 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.321 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.322 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.323 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.328 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "[]", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "MULTICLUSTER_CONFIG": { 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "{}", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.339 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.340 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.341 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.342 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.344 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.345 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.346 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.347 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.349 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.350 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "UNSAFE_TLS_v1_2": { 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.351 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.352 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.353 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.355 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.356 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-07-24T22:05:40.357 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.358 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.359 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.360 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.366 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.367 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.368 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.371 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.375 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.376 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.377 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.380 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.382 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.383 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.384 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.385 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-07-24T22:05:40.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.388 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.389 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.391 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.392 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.392 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.397 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.400 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.401 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.402 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-07-24T22:05:40.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-07-24T22:05:40.404 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.405 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.406 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.407 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.409 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.411 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.413 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.414 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.415 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.420 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.421 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.422 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.423 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.424 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.426 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.429 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.430 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.431 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.432 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.433 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-24T22:05:40.434 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.435 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.437 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.438 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.439 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.440 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-07-24T22:05:40.441 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.442 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.443 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.445 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.446 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.447 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-07-24T22:05:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-07-24T22:05:40.449 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-07-24T22:05:40.450 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-07-24T22:05:40.451 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.452 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.453 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.455 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.457 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.458 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.460 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.462 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.465 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.468 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.471 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.472 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.475 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.477 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.479 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.480 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.481 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.484 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.485 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.486 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.488 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.489 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.491 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.492 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.493 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.494 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.495 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.496 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.497 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.498 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.499 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.500 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.501 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.504 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-24T22:05:40.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.507 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.508 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.511 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.512 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.513 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.514 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.515 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.517 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.519 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.520 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.521 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.522 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.523 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.524 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.526 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.528 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.529 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-07-24T22:05:40.530 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:40.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.532 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.534 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.537 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.538 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.540 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.541 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.542 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.543 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.544 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.545 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.546 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.547 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.548 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.549 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.550 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.551 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.552 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.553 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.555 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.556 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "secondary_zone_period_retry_limit": { 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.563 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.564 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.565 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.566 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-07-24T22:05:40.568 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.572 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.575 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.576 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.577 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-07-24T22:05:40.578 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-07-24T22:05:40.579 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.580 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.581 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.582 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.583 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.586 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.587 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smb", 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.589 INFO:teuthology.orchestra.run.smithi060.stdout: "internal_store_backend": { 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "internal_store_backend", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.591 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.592 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.593 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.595 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.596 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "update_orchestration": { 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "update_orchestration", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.597 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-07-24T22:05:40.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-07-24T22:05:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.601 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.602 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.603 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.604 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.605 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.606 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.607 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.608 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.610 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.611 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.613 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.614 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.615 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.616 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.617 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.618 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.619 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.622 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.624 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.625 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.626 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.627 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-07-24T22:05:40.628 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-24T22:05:40.630 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.631 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.632 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-24T22:05:40.633 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.634 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.638 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.646 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.651 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.652 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.653 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.654 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.655 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.656 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.658 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.659 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.661 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.662 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.665 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.666 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-07-24T22:05:40.668 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.671 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.672 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.673 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.675 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.676 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.677 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "modules": [ 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "iostat", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "nfs", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "restful", 2024-07-24T22:05:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_schedule" 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.692 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.693 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.694 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.695 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.698 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.699 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:40.700 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-07-24T22:05:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.702 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-07-24T22:05:40.703 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-07-24T22:05:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.706 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.707 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.710 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.711 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.712 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-24T22:05:40.714 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.718 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.719 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.721 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.722 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.723 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.724 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.727 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.729 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.730 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.731 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.732 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nginx": { 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nginx", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nginx container image", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.744 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba": { 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB container image", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-07-24T22:05:40.746 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-07-24T22:05:40.747 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.748 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.749 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.751 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "grafana_dashboards_path": { 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "grafana_dashboards_path", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-24T22:05:40.754 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.757 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.759 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.761 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.765 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-07-24T22:05:40.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-24T22:05:40.771 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.772 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.774 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.776 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.777 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.780 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-24T22:05:40.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.797 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.798 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.799 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.800 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.802 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.804 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.808 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.809 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.811 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.813 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.814 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.815 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "[]", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "MULTICLUSTER_CONFIG": { 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "{}", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-24T22:05:40.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-24T22:05:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-24T22:05:40.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-24T22:05:40.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-24T22:05:40.825 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.826 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.827 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.828 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.830 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.836 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.838 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "UNSAFE_TLS_v1_2": { 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.839 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.840 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.842 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.843 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.846 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.848 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.851 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.852 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.853 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-07-24T22:05:40.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:40.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:40.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.858 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.859 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:40.862 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.863 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.865 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.866 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.868 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.871 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.872 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.873 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.876 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-24T22:05:40.878 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.882 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-07-24T22:05:40.884 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-07-24T22:05:40.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.886 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.889 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.890 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-07-24T22:05:40.892 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.893 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.894 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.896 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.900 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-07-24T22:05:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-07-24T22:05:40.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-07-24T22:05:40.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-07-24T22:05:40.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.906 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.907 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.908 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.909 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.910 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.912 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.913 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.915 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.916 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.917 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.919 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-07-24T22:05:40.920 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.922 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.924 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-07-24T22:05:40.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.929 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.930 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.931 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.932 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.934 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-07-24T22:05:40.935 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-07-24T22:05:40.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.937 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.939 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.942 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.943 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.944 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.948 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.955 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.957 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.958 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.964 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.965 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.966 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-07-24T22:05:40.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.968 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.972 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.973 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.975 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.976 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.977 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.978 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.981 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.982 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.983 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.984 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.985 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.986 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:40.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:40.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-07-24T22:05:40.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:40.991 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.995 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.998 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:40.999 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.001 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.002 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-07-24T22:05:41.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-07-24T22:05:41.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.006 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.008 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.009 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-07-24T22:05:41.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.013 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.014 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.017 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-07-24T22:05:41.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.019 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.022 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.027 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.028 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.029 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-07-24T22:05:41.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.032 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.033 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.036 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.037 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.039 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.041 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.042 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.043 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "secondary_zone_period_retry_limit": { 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-24T22:05:41.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.046 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.047 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.048 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.050 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.051 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.052 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.056 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.059 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-07-24T22:05:41.062 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.063 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.068 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-07-24T22:05:41.071 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-07-24T22:05:41.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smb", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "internal_store_backend": { 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "internal_store_backend", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.075 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.076 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.079 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.080 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "update_orchestration": { 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "update_orchestration", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.083 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.084 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.086 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.088 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.091 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.095 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.098 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.099 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.100 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.102 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.104 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:41.106 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.114 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.116 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.119 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.121 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.123 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.124 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.125 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.130 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.132 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-07-24T22:05:41.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.140 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.141 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.142 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.144 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.146 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.147 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-07-24T22:05:41.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.158 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-07-24T22:05:41.160 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-07-24T22:05:41.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-07-24T22:05:41.164 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-07-24T22:05:41.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "services": { 2024-07-24T22:05:41.166 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard": "https://172.21.15.60:8443/" 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "always_on_modules": { 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "octopus": [ 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-07-24T22:05:41.167 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "pacific": [ 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.168 INFO:teuthology.orchestra.run.smithi060.stdout: "quincy": [ 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "reef": [ 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-07-24T22:05:41.169 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "squid": [ 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-07-24T22:05:41.170 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "last_failure_osd_epoch": 5, 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "active_clients": [ 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-07-24T22:05:41.171 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4138987889 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "libcephsqlite", 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1577212541 2024-07-24T22:05:41.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 708808603 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.173 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 873087876 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.174 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3293767799 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2330701917 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.175 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:41.176 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-07-24T22:05:41.176 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 msgr2=0x7fba800b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 0x7fba800b6200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c005f20 comp rx=0 tx=0).stop 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 msgr2=0x7fba98198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fba8800e9f0 tx=0x7fba8800edf0 comp rx=0 tx=0).stop 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 shutdown_connections 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fba800b3d40 0x7fba800b6200 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:41.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba9810ca40 0x7fba9819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:41.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba98103f10 0x7fba98199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:41.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 --2- 172.21.15.60:0/2146588652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba981035f0 0x7fba98198c20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:05:41.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.701+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 >> 172.21.15.60:0/2146588652 conn(0x7fba980fd020 msgr2=0x7fba980fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:05:41.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.702+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 shutdown_connections 2024-07-24T22:05:41.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:05:39.702+0000 7fba9da79640 1 -- 172.21.15.60:0/2146588652 wait complete. 2024-07-24T22:05:41.230 DEBUG:tasks.check_counter:Getting stats from a 2024-07-24T22:05:41.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.a perf dump 2024-07-24T22:05:41.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:39 smithi060 ceph-mon[33000]: pgmap v973: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:41.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1903814052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1467467463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2279948497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:40 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/2146588652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-24T22:05:41.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:39 smithi086 ceph-mon[37128]: pgmap v973: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:41.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1903814052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1467467463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2279948497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:40 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/2146588652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:39 smithi123 ceph-mon[37143]: pgmap v973: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1903814052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1467467463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2279948497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-24T22:05:41.233 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:40 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/2146588652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-24T22:05:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:41.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:41 smithi060 ceph-mon[33000]: pgmap v974: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:41.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:41 smithi086 ceph-mon[37128]: pgmap v974: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:41 smithi123 ceph-mon[37143]: pgmap v974: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:43.283 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:43.283 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 368, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 362, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1013536, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 224256, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.059167612, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.025261387, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.040042458, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 361, 2024-07-24T22:05:43.284 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.022306818, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000061791 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1013536, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 224256 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 278, 2024-07-24T22:05:43.285 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 320, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 207936, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.020965499, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.016331626, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001965406, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 276, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.018049150, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000065395 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.286 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 207936 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 4, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 4, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 2992, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 448, 2024-07-24T22:05:43.287 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.002355909, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000046569, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.002289595, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004065235, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001355078 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:43.288 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 448 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:43.289 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:43.290 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:43.291 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148682, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 29, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-07-24T22:05:43.292 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:43.293 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 370, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 961505, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 370, 2024-07-24T22:05:43.294 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 370, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 961505, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:43.295 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:43 smithi123 ceph-mon[37143]: pgmap v975: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:44.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:43 smithi060 ceph-mon[33000]: pgmap v975: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-07-24T22:05:44.032 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-07-24T22:05:44.033 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-07-24T22:05:44.033 DEBUG:tasks.check_counter:Getting stats from d 2024-07-24T22:05:44.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.d perf dump 2024-07-24T22:05:44.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:43 smithi086 ceph-mon[37128]: pgmap v975: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:44.299 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:45 smithi060 ceph-mon[33000]: pgmap v976: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:45 smithi123 ceph-mon[37143]: pgmap v976: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:45 smithi086 ceph-mon[37128]: pgmap v976: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:46.114 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:46.114 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:46.114 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 375, 2024-07-24T22:05:46.114 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 369, 2024-07-24T22:05:46.114 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1015504, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 228256, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.065947949, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.025165205, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.051439297, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 368, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.025740219, 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000069946 2024-07-24T22:05:46.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1015504, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 228256 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 6, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 284, 2024-07-24T22:05:46.116 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3312, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 209088, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.024319364, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.016853915, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.004488286, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 283, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.020089029, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000070985 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:46.117 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 209088 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-07-24T22:05:46.118 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.000324348, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000236567, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:46.119 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:46.121 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:46.122 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:46.123 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148744, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 30, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:46.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-07-24T22:05:46.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 377, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 962479, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 377, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 377, 2024-07-24T22:05:46.126 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 962479, 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:46.127 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-07-24T22:05:46.869 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-07-24T22:05:46.869 DEBUG:tasks.check_counter:Getting stats from g 2024-07-24T22:05:46.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.g perf dump 2024-07-24T22:05:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:47.959 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:47 smithi060 ceph-mon[33000]: pgmap v977: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:47 smithi123 ceph-mon[37143]: pgmap v977: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:47 smithi086 ceph-mon[37128]: pgmap v977: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:48.923 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.001453063, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000258464, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:48.924 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 375, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 367, 2024-07-24T22:05:48.925 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1017344, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 226624, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.061802013, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.025494537, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.044006828, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 367, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.036913754, 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000100582 2024-07-24T22:05:48.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1017344, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 226624 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 279, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 128432, 2024-07-24T22:05:48.927 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.021558202, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.017832096, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001146460, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 277, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.020937971, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000075588 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:48.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 128432 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:48.929 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:48.930 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:48.931 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148967, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 29, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-07-24T22:05:48.932 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:48.933 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 372, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 961868, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 372, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-07-24T22:05:48.934 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 372, 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 961868, 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:49.652 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-07-24T22:05:49.652 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-07-24T22:05:49.652 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-07-24T22:05:49.652 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-07-24T22:05:49.652 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-07-24T22:05:49.653 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-07-24T22:05:49.653 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-07-24T22:05:49.653 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-07-24T22:05:49.653 DEBUG:tasks.check_counter:Getting stats from j 2024-07-24T22:05:49.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.j perf dump 2024-07-24T22:05:49.924 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:05:49.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:49 smithi123 ceph-mon[37143]: pgmap v978: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:50.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:49 smithi086 ceph-mon[37128]: pgmap v978: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:50.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:49 smithi060 ceph-mon[33000]: pgmap v978: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:51.728 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.001259600, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000218208, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:51.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 284, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 320, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 208992, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.021493724, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.016485852, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001788426, 2024-07-24T22:05:51.730 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 282, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.017744107, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000062922 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 208992 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:51.731 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 381, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 373, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1019040, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 229984, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.060316675, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.026299818, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.042405201, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 373, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.024168462, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000064794 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.732 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1019040, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 229984 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:51.734 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:51.735 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148721, 2024-07-24T22:05:51.736 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 30, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:51.737 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-07-24T22:05:51.738 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 379, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 962739, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 379, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 379, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 962739, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-07-24T22:05:51.739 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-07-24T22:05:51.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:51 smithi123 ceph-mon[37143]: pgmap v979: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:52.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:51 smithi086 ceph-mon[37128]: pgmap v979: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:52.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:51 smithi060 ceph-mon[33000]: pgmap v979: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-07-24T22:05:52.600 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-07-24T22:05:52.600 DEBUG:tasks.check_counter:Getting stats from b 2024-07-24T22:05:52.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.b perf dump 2024-07-24T22:05:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T22:05:53.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:53 smithi086 ceph-mon[37128]: pgmap v980: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:53.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:53 smithi123 ceph-mon[37143]: pgmap v980: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:54.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:53 smithi060 ceph-mon[33000]: pgmap v980: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-07-24T22:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 163701, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 164185, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 137251891, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 314451144, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 14, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 8.573085461, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 2.968824526, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 3.942868288, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 2.063396032, 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:55.211 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 164182, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 18.727705528, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000114066 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 137809, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.019440144, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000141 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1160576, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 752000 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 182734, 2024-07-24T22:05:55.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 210982, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 177575218, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 595144546, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 48, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 11.146622244, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 4.650916576, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 5.009798987, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 2.061373224, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 210983, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 37.180053441, 2024-07-24T22:05:55.213 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000176222 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 202231, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.034469648, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000170 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 576464 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:55.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 497953, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 729206, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 176532627, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 861907553, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 11, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 23.729635982, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 11.119289478, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 17.727026569, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 2.514207419, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 729204, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 77.428785853, 2024-07-24T22:05:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000106182 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 548550, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.120158783, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000219 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-07-24T22:05:55.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 168892, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 241.270676569, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001428550 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:05:55.217 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 62964, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 15.955842372, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000253412 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 226637, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 209380, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 209380, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5731.788554318, 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.027375052 2024-07-24T22:05:55.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 17247, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 451, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 3, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 2558, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 213, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 2, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 50478, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 11129, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 37503, 2024-07-24T22:05:55.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 1846, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 34579, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 302775, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 2, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 2, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 5, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 370374, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 295956, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 17175, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 2331, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 40, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 516, 2024-07-24T22:05:55.220 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 2043, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 229637, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 1903, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 244764, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 216487, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 2431, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 54331533, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 216487, 2024-07-24T22:05:55.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 1491, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 1491, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 214996, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 366, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 81444, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 97277, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 2, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:05:55.222 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 112106, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 47518, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 19255, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 86539 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 3344, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 3908, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 2124, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 4162, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 6722, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 139, 2024-07-24T22:05:55.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 36228, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 36089, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 14, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 155769, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-07-24T22:05:55.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 2044, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 223, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 8, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 233642, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 171906, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 159771, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 73871, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 12135, 2024-07-24T22:05:55.225 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 369, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 241, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 221, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 148, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 7, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 20, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 467828032, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 1091962139, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 12049, 2024-07-24T22:05:55.226 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1947.120593112, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.161600182 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 32225, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 271529, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 239304, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1072, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 7800, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 6728, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 50478, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 418867, 2024-07-24T22:05:55.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 368389, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 2, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 83453, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 83451, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 302156, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 223512 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 260387, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 6958, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 226637, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 537, 2024-07-24T22:05:55.228 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 18610, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 40096, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 329.672535817, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.008222080 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2188, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 991.855192786, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.453315901 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 80101, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3421.602748173, 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.042716105 2024-07-24T22:05:55.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000118871, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000118871 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.232 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 255, 2024-07-24T22:05:55.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.973668183, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.015583012 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 14488, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 200.470913772, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.013837031 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1600, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4.816704701, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003010440 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 40102, 2024-07-24T22:05:55.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 572.574526076, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.014277954 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 178, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 47.944337214, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.269350209 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.158396696, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.079198348 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.239 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 30369, 2024-07-24T22:05:55.240 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 158.719412029, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005226362 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-07-24T22:05:55.241 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 17142, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 8571, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1137, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:55.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:55.243 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-07-24T22:05:55.244 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 2644714, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 32471, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:55.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8360, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 185, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 112481125, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 1076501, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-07-24T22:05:55.246 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 279890, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 1149790791, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 31, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 279890, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 279890, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3023.481645708, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.010802392 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 279890, 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 299787 2024-07-24T22:05:55.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 279890, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 219081, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 60809, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 644, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 7241, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 899, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 12960, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 481, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 207, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 36774, 2024-07-24T22:05:55.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 216512, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 14482, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 2, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-07-24T22:05:55.249 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-07-24T22:05:55.250 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 114, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 6196, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2149, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 1240 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-07-24T22:05:55.251 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 147521 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 18, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 10, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 36369, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 36089, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:55.252 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 844384, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 421651963, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 844384, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 844384, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 421651963, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 279890, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1154817089, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 279890, 2024-07-24T22:05:55.254 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1154817089, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:55.255 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 279890, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 279890, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 279890, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 279890, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 36089, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 5075221, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 36089, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 901, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 5075221, 2024-07-24T22:05:55.257 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x563851ebc6e0": { 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 233642, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1087767835, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:55.258 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 233642, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 12049, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1087767835, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:55.259 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-07-24T22:05:55.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:55 smithi123 ceph-mon[37143]: pgmap v981: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:56.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:55 smithi086 ceph-mon[37128]: pgmap v981: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:56.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:55 smithi060 ceph-mon[33000]: pgmap v981: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds.exported=2043 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds.imported=1903 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=3344 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3908 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-07-24T22:05:56.083 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=213 2024-07-24T22:05:56.083 DEBUG:tasks.check_counter:Getting stats from e 2024-07-24T22:05:56.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.e perf dump 2024-07-24T22:05:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T22:05:57.946 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:57 smithi086 ceph-mon[37128]: pgmap v982: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:57.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:57 smithi123 ceph-mon[37143]: pgmap v982: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:58.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:57 smithi060 ceph-mon[33000]: pgmap v982: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1174, 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1176, 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 247791595, 2024-07-24T22:05:58.209 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 373396, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.209435511, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.029185821, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.145412961, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.025977194, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1174, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.042368251, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000036088 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:58.210 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2015, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1989, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 240659639, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 739063, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-07-24T22:05:58.211 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.305282857, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.053287178, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.211508128, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.042238493, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1988, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.072637414, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000036537 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.212 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1133712, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 235664 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2261, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2541, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 137845918, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1292592, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.285258090, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.064586847, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.125768053, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.077232530, 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:05:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2540, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.127702853, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000050276 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 575184 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.214 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7884, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.322689461, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000040929 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.215 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.217 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 102439, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 15271, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 87168, 2024-07-24T22:05:58.218 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 80623, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-07-24T22:05:58.219 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:05:58.220 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-07-24T22:05:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-07-24T22:05:58.222 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-07-24T22:05:58.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 163444, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-07-24T22:05:58.224 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 142, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 15, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 206394735, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 627879101, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 627868924, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 163444 2024-07-24T22:05:58.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 41297, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 129960, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 88663, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1368, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 2810, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 1442, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 102439, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 196702, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 94263, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-07-24T22:05:58.226 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 252308, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215320 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.227 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.229 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.230 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.232 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.233 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.234 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.235 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-07-24T22:05:58.236 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:05:58.238 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 3, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-07-24T22:05:58.239 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2595, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:05:58.240 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:05:58.241 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 1788510, 2024-07-24T22:05:58.242 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 41945, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8360, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 185, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-07-24T22:05:58.243 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 169053584, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 1495682, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 5013, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-07-24T22:05:58.244 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 5013, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5013, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 39.755146727, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007930410 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5013, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5014 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 5013, 2024-07-24T22:05:58.245 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 5013, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 3114, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1897, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-07-24T22:05:58.246 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-07-24T22:05:58.247 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-07-24T22:05:58.248 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 114, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 11, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-07-24T22:05:58.249 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-07-24T22:05:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 5446, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 625781991, 2024-07-24T22:05:58.251 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 5446, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5446, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 625781991, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:58.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5013, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 623773046, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5013, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 623773046, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.253 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:58.254 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5013, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 5013, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5013, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 5013, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:58.255 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:05:58.256 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x56048c86c960": { 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:05:58.257 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:05:58.258 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:05:58.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:05:58.258 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-07-24T22:05:59.017 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-07-24T22:05:59.018 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-07-24T22:05:59.018 DEBUG:tasks.check_counter:Getting stats from h 2024-07-24T22:05:59.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.h perf dump 2024-07-24T22:05:59.283 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T22:05:59.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:05:59 smithi086 ceph-mon[37128]: pgmap v983: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:05:59.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:05:59 smithi123 ceph-mon[37143]: pgmap v983: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:06:00.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:05:59 smithi060 ceph-mon[33000]: pgmap v983: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-24T22:06:01.045 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3369, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 3371, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 434459230, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1071349, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.528877389, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.083768730, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.335241324, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.087525646, 2024-07-24T22:06:01.046 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3369, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.126098022, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000037428 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.047 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1347, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1628, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 331780952, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1091468, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.297931395, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.038274961, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.216033208, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.032649804, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1626, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.079090208, 2024-07-24T22:06:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048640 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 663696 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1511, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1483, 2024-07-24T22:06:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 324407797, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 577640, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.320777180, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.044742767, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.255158859, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.024946495, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1483, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.065472842, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000044148 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:01.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1136432, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 235472 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:06:01.051 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8876, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.357399480, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000040265 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.052 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-07-24T22:06:01.053 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 105398, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 25235, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 80163, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 69347, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 2, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-07-24T22:06:01.054 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-07-24T22:06:01.055 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-07-24T22:06:01.056 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-07-24T22:06:01.057 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-07-24T22:06:01.058 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-07-24T22:06:01.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 233642, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 148, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 21, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 467828032, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 1091962139, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 1091961565, 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-07-24T22:06:01.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 233642 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 36827, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 264238, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 227411, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1052, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 6685, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 5633, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 105398, 2024-07-24T22:06:01.061 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 403365, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 297967, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 273252, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215320 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-07-24T22:06:01.062 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.063 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-07-24T22:06:01.064 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.065 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.066 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.068 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.070 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-07-24T22:06:01.071 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-07-24T22:06:01.072 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.073 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-07-24T22:06:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2597, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:01.076 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:01.077 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 1619858, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 37367, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8360, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 185, 2024-07-24T22:06:01.078 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 160412700, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 1413518, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-07-24T22:06:01.079 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 5790, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 5790, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5790, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 29.349373374, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005068976 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5790, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5791 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 5790, 2024-07-24T22:06:01.080 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 5790, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 3656, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 2132, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-07-24T22:06:01.081 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:06:01.082 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-07-24T22:06:01.083 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-07-24T22:06:01.084 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 114, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 11, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-07-24T22:06:01.085 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-07-24T22:06:01.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:06:01.087 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 6223, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1090062897, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 6223, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 6223, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1090062897, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.088 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:01.089 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5790, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1087921227, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5790, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1087921227, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-07-24T22:06:01.090 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 5790, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 5790, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 5790, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 5790, 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:01.091 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:01.092 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x556b28d52960": { 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:06:01.093 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:01.094 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:01.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:01.095 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-07-24T22:06:01.864 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:01 smithi086 ceph-mon[37128]: pgmap v984: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-07-24T22:06:01.892 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-07-24T22:06:01.893 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-07-24T22:06:01.893 DEBUG:tasks.check_counter:Getting stats from k 2024-07-24T22:06:01.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.k perf dump 2024-07-24T22:06:01.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:01 smithi123 ceph-mon[37143]: pgmap v984: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:06:02.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:01 smithi060 ceph-mon[33000]: pgmap v984: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-24T22:06:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.b/config 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 297, 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 299, 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 21045471, 2024-07-24T22:06:03.769 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 94452, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.036517299, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.006002051, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.017082001, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.009961264, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 297, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.016412507, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000055260 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:03.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 535, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 820, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3314310, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 470780, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-07-24T22:06:03.771 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.062789355, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.026622422, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.015638844, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.014474825, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 818, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.056379139, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000068923 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 301216 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 483, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 475, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 30066346, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 261799, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-07-24T22:06:03.773 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.078724785, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.025298651, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.059817420, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.002030942, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 475, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.030957789, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000065174 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.774 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1038288, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 234768 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:06:03.775 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1447, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.082331964, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000056898 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.776 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-07-24T22:06:03.777 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 11765, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 8209, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 3556, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 37, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-07-24T22:06:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-07-24T22:06:03.779 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 4, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 234, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 7, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-07-24T22:06:03.780 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-07-24T22:06:03.781 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-07-24T22:06:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-07-24T22:06:03.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 8813, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 15, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 711951967, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 711987785, 2024-07-24T22:06:03.784 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 711987701, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 8813 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3866, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 8565, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 4699, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 215, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 893, 2024-07-24T22:06:03.785 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 678, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 11765, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 14083, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 2318, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 75624, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215320 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-07-24T22:06:03.786 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-07-24T22:06:03.791 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-07-24T22:06:03.792 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-07-24T22:06:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.795 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-07-24T22:06:03.798 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2600, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:03.799 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:03.800 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 376367, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 4019, 2024-07-24T22:06:03.801 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8360, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 185, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:03.802 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 17443205, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 133400, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 915, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 26, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 4, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 915, 2024-07-24T22:06:03.803 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 915, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.549968070, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001693954 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 915, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 916 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 915, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 915, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-07-24T22:06:03.804 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 578, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 334, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-07-24T22:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 2, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:06:03.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-07-24T22:06:03.807 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 114, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 7, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 10, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 10, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-07-24T22:06:03.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-07-24T22:06:03.809 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1311, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 54284819, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1311, 2024-07-24T22:06:03.810 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1311, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 54284819, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-07-24T22:06:03.811 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 915, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 53127414, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 915, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 53127414, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 915, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 915, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 915, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 915, 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:03.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:03.814 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5610cb78c960": { 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-07-24T22:06:03.815 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-07-24T22:06:03.816 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-07-24T22:06:03.817 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:03 smithi086 ceph-mon[37128]: pgmap v985: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-07-24T22:06:03.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:03 smithi123 ceph-mon[37143]: pgmap v985: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-07-24T22:06:04.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:03 smithi060 ceph-mon[33000]: pgmap v985: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=7 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-07-24T22:06:04.532 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-07-24T22:06:04.532 DEBUG:tasks.check_counter:Getting stats from c 2024-07-24T22:06:04.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.c perf dump 2024-07-24T22:06:05.110 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T22:06:05.856 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:05 smithi123 ceph-mon[37143]: pgmap v986: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-24T22:06:06.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:05 smithi086 ceph-mon[37128]: pgmap v986: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-24T22:06:06.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:05 smithi060 ceph-mon[33000]: pgmap v986: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 123667, 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 130999, 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 74238267, 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 270463685, 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 18, 2024-07-24T22:06:06.828 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 9.455484390, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.768111806, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.175892932, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.614661993, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 130999, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 23.178011314, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000176932 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 98687, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.016100858, 2024-07-24T22:06:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000163 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 142730, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 142221, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 88440349, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 421503143, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 39, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 10.554348356, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 4.449286702, 2024-07-24T22:06:06.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 4.033326661, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.448211998, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 142222, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 28.873823254, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000203019 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 138304, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.021866091, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000158 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1154016, 2024-07-24T22:06:06.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 305056 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 375813, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 523799, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 124573065, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 340602707, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 17, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 8, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 17.103781106, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 7.776096720, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 14.619515711, 2024-07-24T22:06:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 1.684001424, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 523794, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 63.882165193, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000121960 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 408129, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.093780049, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000229 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 9232, 2024-07-24T22:06:06.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 604880 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 105576, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 177.536886434, 2024-07-24T22:06:06.834 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001681602 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 42739, 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.927984382, 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000255691 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-07-24T22:06:06.835 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 170957, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 157176, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 157176, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3950.665997150, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.025135300 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 13758, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 670, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 2028, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 134, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 3, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 63446, 2024-07-24T22:06:06.836 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 14871, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 41022, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 7553, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 42199, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 111103, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 2, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 5, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 291058, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 229882, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 13192, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 1830, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 33, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-07-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 1264, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 737, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 93407, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 1523, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 78674, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 142840, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 2431, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 54331533, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 6, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 142840, 2024-07-24T22:06:06.838 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 1138, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 1138, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 141702, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 668, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 78250, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 69593, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 1, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 88356, 2024-07-24T22:06:06.839 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 30966, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 23486, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 56445 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 2115, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 7574, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 4349, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 5813, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 4677, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 182, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 24777, 2024-07-24T22:06:06.840 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 24596, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 14, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 86450, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 742, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-07-24T22:06:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 138, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 4, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 163444, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 83944, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 73741, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 89703, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 10203, 2024-07-24T22:06:06.842 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 243, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 115, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 101, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 142, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 8, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 14, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 206394735, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 627879101, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 12087, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4055.529828017, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.335528239 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.843 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 39292, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 137005, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 97713, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 1395, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 6125, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 4730, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 63446, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 207483, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 144037, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 2, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 59847, 2024-07-24T22:06:06.844 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 59845, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 307888, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223512 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 185878, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 11980, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 170957, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 347, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 25673, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 23160, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 337.915587914, 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.014590483 2024-07-24T22:06:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 145, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 27.198486455, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.187575768 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 78856, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2686.940016933, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.034074008 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-07-24T22:06:06.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000118419, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000118419 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 124, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 27.093370301, 2024-07-24T22:06:06.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.218494921 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9546, 2024-07-24T22:06:06.850 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 87.071767529, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.009121283 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1113, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 8.084755153, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007263930 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 23176, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 652.943520463, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.028173262 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-07-24T22:06:06.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 181, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 30.431380836, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.168129175 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.852 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.842537814, 2024-07-24T22:06:06.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 1.842537814 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-07-24T22:06:06.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-07-24T22:06:06.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20873, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 91.144455333, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004366619 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.856 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 7060, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 3530, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1149, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-07-24T22:06:06.857 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:06.858 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:06.859 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 3201835, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 38062, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:06.860 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8360, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 185, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 138490413, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 1351740, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:06.861 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 199084, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 677089770, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 199084, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 199084, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 5659.896317811, 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.028429689 2024-07-24T22:06:06.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 199084, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 228260 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 199084, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 146416, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 52668, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 801, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 11994, 2024-07-24T22:06:06.863 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 731, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 12818, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 408, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 1, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 132, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 24815, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-07-24T22:06:06.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 142849, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 23988, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-07-24T22:06:06.865 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 1, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-07-24T22:06:06.866 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 115, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 32, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-07-24T22:06:06.867 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 4928, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 3515, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1279 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 86289 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.868 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 10, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 24596, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 24596, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 642206, 2024-07-24T22:06:06.869 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 234571038, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 642206, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 642206, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 234571038, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.870 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 199084, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 680383289, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 199084, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 680383289, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 199084, 2024-07-24T22:06:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 199084, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 199084, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 199084, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 24596, 2024-07-24T22:06:06.873 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 3377468, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 24596, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 731, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3377468, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5561afca06e0": { 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 163444, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 623684797, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 163444, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 12087, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 623684797, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:06.875 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:06.876 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds.exported=737 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds.imported=1523 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=2115 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=7574 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-07-24T22:06:07.708 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-07-24T22:06:07.709 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-07-24T22:06:07.709 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=134 2024-07-24T22:06:07.709 DEBUG:tasks.check_counter:Getting stats from f 2024-07-24T22:06:07.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.f perf dump 2024-07-24T22:06:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:07 smithi123 ceph-mon[37143]: pgmap v987: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-24T22:06:07.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:07 smithi123 ceph-mon[37143]: osdmap e115: 12 total, 12 up, 12 in 2024-07-24T22:06:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:07 smithi086 ceph-mon[37128]: pgmap v987: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-24T22:06:08.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:07 smithi086 ceph-mon[37128]: osdmap e115: 12 total, 12 up, 12 in 2024-07-24T22:06:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:07 smithi060 ceph-mon[33000]: pgmap v987: 97 pgs: 97 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-24T22:06:08.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:07 smithi060 ceph-mon[33000]: osdmap e115: 12 total, 12 up, 12 in 2024-07-24T22:06:08.220 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T22:06:09.931 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:09 smithi123 ceph-mon[37143]: pgmap v989: 97 pgs: 12 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:10.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:09 smithi086 ceph-mon[37128]: pgmap v989: 97 pgs: 12 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:10.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:09 smithi060 ceph-mon[33000]: pgmap v989: 97 pgs: 12 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 68 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001777650, 2024-07-24T22:06:10.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000605990, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:06:10.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 152, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 151, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 197184, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 92208, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.019698045, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.007310702, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.014778791, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 150, 2024-07-24T22:06:10.560 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.012877435, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000085849 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 197184, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 92208 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 5, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 116, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3152, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 99840, 2024-07-24T22:06:10.561 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.010202333, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.004510667, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.004474835, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 115, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.017778709, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000154597 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:10.562 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 99840 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:10.564 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:10.565 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 149024, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-07-24T22:06:10.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 153, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 175717, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 153, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 153, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 175717, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:10.568 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:10.569 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:10.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:10.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:10.569 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-07-24T22:06:11.495 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-07-24T22:06:11.495 DEBUG:tasks.check_counter:Getting stats from i 2024-07-24T22:06:11.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.i perf dump 2024-07-24T22:06:11.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:11 smithi123 ceph-mon[37143]: pgmap v990: 97 pgs: 16 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:11.982 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T22:06:12.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:11 smithi086 ceph-mon[37128]: pgmap v990: 97 pgs: 16 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:12.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:11 smithi060 ceph-mon[33000]: pgmap v990: 97 pgs: 16 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:13.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:13 smithi123 ceph-mon[37143]: pgmap v991: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:14.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:13 smithi086 ceph-mon[37128]: pgmap v991: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:14.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:13 smithi060 ceph-mon[33000]: pgmap v991: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 156804, 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 148384, 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 86428139, 2024-07-24T22:06:14.225 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 491832958, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 19, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 12.804071950, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 5.093478774, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.584491917, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 4.138766088, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 148382, 2024-07-24T22:06:14.226 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 44.961268717, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000303010 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 101324, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.015686996, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000154 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 621472, 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 206400 2024-07-24T22:06:14.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 480014, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 635803, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 183716205, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 535559878, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 334, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 8, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 24.497917657, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 9.993632591, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 15.819755791, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 4.599130612, 2024-07-24T22:06:14.228 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 635802, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 243.682560953, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000383268 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 451552, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.098125607, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000217 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1226448, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 335472 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.229 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 202512, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 174813, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 308552601, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 295757966, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 43, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 11.962614502, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 4.180703142, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 6.512939755, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.995658225, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 174812, 2024-07-24T22:06:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 44.596751242, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000255112 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 209385, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.030232262, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000144 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 592816 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-07-24T22:06:14.231 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 147382, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 236.575417473, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001605185 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-07-24T22:06:14.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 48503, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.708084500, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000220771 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 201319, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 185966, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 185966, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4289.783520499, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.023067568 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 15295, 2024-07-24T22:06:14.233 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 1090, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 1, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11259, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 148, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 2, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 41317, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 28793, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 12345, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 179, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 182, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 252214, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 5, 2024-07-24T22:06:14.234 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 338437, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 275660, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 14984, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 1942, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 7, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 468, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 100, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 1844, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 186578, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 1198, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 186184, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-07-24T22:06:14.235 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 183475, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 7, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 183475, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 1417, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 20, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 1397, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 182058, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 245, 2024-07-24T22:06:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 75579, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 73721, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 6, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 94093, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 37750, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 24603, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 58853 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 2863, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 7315, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 4002, 2024-07-24T22:06:14.237 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 6022, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 5042, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 170, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 28334, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 28164, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-07-24T22:06:14.238 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 14, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 198049, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 20, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 1852, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 158, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 12, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 178999, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 178983, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 178983, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 16, 2024-07-24T22:06:14.239 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 11, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 333, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 332, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 332, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 1, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 1, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 1, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 711951967, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 711987957, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 11419, 2024-07-24T22:06:14.240 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3590.854960237, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.314463171 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 17343, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 223945, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 206602, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 1320, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 7975, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 6655, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 41317, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 336916, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 295599, 2024-07-24T22:06:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 63568, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 63567, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 341528, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231704 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 217266, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 4288, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 201319, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 332, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 11269, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25552, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 283.516901025, 2024-07-24T22:06:14.242 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.011095683 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3488, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 225.705824018, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.064709238 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 95727, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3132.345148140, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.032721647 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.244 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.245 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.007520109, 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000376005 2024-07-24T22:06:14.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 160, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 28.193662601, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.176210391 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 15.438636142, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.771931807 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.247 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10029, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 110.250828587, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.010993202 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1337, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.416894784, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007791245 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25151, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 361.649306993, 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.014379122 2024-07-24T22:06:14.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 174, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 64.941018561, 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.373224244 2024-07-24T22:06:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 13, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7.669383662, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.589952589 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-07-24T22:06:14.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.039541816, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005648830 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.251 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.018301035, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002614433 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002337609, 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000779203 2024-07-24T22:06:14.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 24278, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 49.588215417, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002042516 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.253 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 803, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 401, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1172, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 120, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 120, 2024-07-24T22:06:14.254 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:14.255 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 2355652, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 17589, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:14.257 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8480, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 188, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 72356835, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 709176, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.258 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 293484, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 856708296, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 293484, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 293484, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 52167.907989648, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.177753840 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-07-24T22:06:14.259 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 293484, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 420606 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 293484, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 189067, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 104417, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 9867, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 50613, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 679, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 12084, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 640, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-07-24T22:06:14.260 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 148, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 29022, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 183513, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 101227, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-07-24T22:06:14.261 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 6, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-07-24T22:06:14.262 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 115, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 32, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 21779, 2024-07-24T22:06:14.263 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 5984, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 5044 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 170056 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-07-24T22:06:14.264 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 10, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 28164, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 28164, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 839326, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 508857661, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:14.265 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 839326, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 839326, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 508857661, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-07-24T22:06:14.266 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 293484, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 860415160, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 293484, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 860415160, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 293484, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 293484, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 293484, 2024-07-24T22:06:14.268 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 293484, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 28164, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 3851964, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:14.269 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 28164, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 659, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3851964, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55de913466e0": { 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 178999, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 707793653, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 178999, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 11419, 2024-07-24T22:06:14.271 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 707793653, 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:14.272 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-07-24T22:06:15.150 INFO:tasks.check_counter:Daemon mds.i mds.exported=1844 2024-07-24T22:06:15.150 INFO:tasks.check_counter:Daemon mds.i mds.imported=1198 2024-07-24T22:06:15.150 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2863 2024-07-24T22:06:15.150 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=7315 2024-07-24T22:06:15.150 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-07-24T22:06:15.151 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=20 2024-07-24T22:06:15.151 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=13 2024-07-24T22:06:15.151 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=148 2024-07-24T22:06:15.151 DEBUG:tasks.check_counter:Getting stats from l 2024-07-24T22:06:15.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph daemon mds.l perf dump 2024-07-24T22:06:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.c/config 2024-07-24T22:06:15.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:15 smithi123 ceph-mon[37143]: pgmap v992: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-24T22:06:15.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:15 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:06:16.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:15 smithi086 ceph-mon[37128]: pgmap v992: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-24T22:06:16.048 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:15 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:06:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:15 smithi060 ceph-mon[33000]: pgmap v992: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-24T22:06:16.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:15 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-24T22:06:17.954 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:17 smithi123 ceph-mon[37143]: pgmap v993: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-24T22:06:17.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:17.955 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:17 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 79, 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 77, 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 177760, 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 45440, 2024-07-24T22:06:17.955 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013678429, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.003570130, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.011258935, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 77, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.010869412, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000141161 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:17.956 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 177760, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 45440 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 57, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 89360, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.005402416, 2024-07-24T22:06:17.957 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.002303204, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001550601, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 55, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.005744099, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000104438 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:17.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 89360 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.000271808, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000172869, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-24T22:06:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.960 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-07-24T22:06:17.961 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-07-24T22:06:17.962 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148823, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 27, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-07-24T22:06:17.963 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-07-24T22:06:17.964 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 77, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 164396, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 77, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-07-24T22:06:17.965 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 77, 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 164396, 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-07-24T22:06:17.966 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-07-24T22:06:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:17 smithi086 ceph-mon[37128]: pgmap v993: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-24T22:06:18.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:18.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:17 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:18.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:17 smithi060 ceph-mon[33000]: pgmap v993: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-24T22:06:18.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:18.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:17 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:18.944 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:18.944 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:18 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:19.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:19.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:18 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:19.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:18 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:19.283 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-07-24T22:06:19.284 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-07-24T22:06:19.284 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-24T22:06:19.298 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-24T22:06:19.298 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T22:06:19.298 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-24T22:06:19.337 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-07-24T22:06:19.337 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-24T22:06:19.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd blocklist ls 2024-07-24T22:06:19.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:19 smithi060 ceph-mon[33000]: pgmap v994: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 6 op/s 2024-07-24T22:06:19.933 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:19.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:19 smithi123 ceph-mon[37143]: pgmap v994: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 6 op/s 2024-07-24T22:06:20.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:19 smithi086 ceph-mon[37128]: pgmap v994: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 6 op/s 2024-07-24T22:06:21.308 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: pgmap v995: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:21.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:06:21.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:06:21.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:21 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: pgmap v995: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:21.493 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:06:21.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:06:21.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:21 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: pgmap v995: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:21.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-24T22:06:21.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-24T22:06:21.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:21 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' 2024-07-24T22:06:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 -- 172.21.15.60:0/1637132097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da406e0f0 msgr2=0x7f5da406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 --2- 172.21.15.60:0/1637132097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da406e0f0 0x7f5da406e4d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5d9401a910 tx=0x7f5d9403b470 comp rx=0 tx=0).stop 2024-07-24T22:06:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 -- 172.21.15.60:0/1637132097 shutdown_connections 2024-07-24T22:06:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 --2- 172.21.15.60:0/1637132097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da411b9d0 0x7f5da411ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 --2- 172.21.15.60:0/1637132097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406ea10 0x7f5da411b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 --2- 172.21.15.60:0/1637132097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da406e0f0 0x7f5da406e4d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 -- 172.21.15.60:0/1637132097 >> 172.21.15.60:0/1637132097 conn(0x7f5da406c320 msgr2=0x7f5da406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 -- 172.21.15.60:0/1637132097 shutdown_connections 2024-07-24T22:06:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.031+0000 7f5dab781640 1 -- 172.21.15.60:0/1637132097 wait complete. 2024-07-24T22:06:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.032+0000 7f5dab781640 1 Processor -- start 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.032+0000 7f5dab781640 1 -- start start 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406e0f0 0x7f5da411aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 0x7f5da411b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 0x7f5da41173b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5da9d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 0x7f5da41173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5da9d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 0x7f5da41173b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37532/0 (socket says 172.21.15.60:37532) 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5da9d1e640 1 -- 172.21.15.60:0/205908170 learned_addr learned my addr 172.21.15.60:0/205908170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f5b0 con 0x7f5da411b9d0 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f430 con 0x7f5da406e0f0 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410f730 con 0x7f5da406ea10 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5da8d1c640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 0x7f5da411b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.033+0000 7f5da951d640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406e0f0 0x7f5da411aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.034+0000 7f5da8d1c640 1 -- 172.21.15.60:0/205908170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406e0f0 msgr2=0x7f5da411aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.034+0000 7f5da8d1c640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406e0f0 0x7f5da411aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.034+0000 7f5da8d1c640 1 -- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 msgr2=0x7f5da41173b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.034+0000 7f5da8d1c640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 0x7f5da41173b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.034+0000 7f5da8d1c640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da4117c70 con 0x7f5da406ea10 2024-07-24T22:06:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.039+0000 7f5da8d1c640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 0x7f5da411b430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009fd0 tx=0x7f5da000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.040+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0015020 con 0x7f5da406ea10 2024-07-24T22:06:22.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.040+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da4117f00 con 0x7f5da406ea10 2024-07-24T22:06:22.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.040+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da4075100 con 0x7f5da406ea10 2024-07-24T22:06:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.041+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da0011ce0 con 0x7f5da406ea10 2024-07-24T22:06:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.041+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0010e40 con 0x7f5da406ea10 2024-07-24T22:06:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.042+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d880052c0 con 0x7f5da406ea10 2024-07-24T22:06:22.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.050+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f5da0018020 con 0x7f5da406ea10 2024-07-24T22:06:22.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.051+0000 7f5d9a7fc640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 0x7f5d900b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.052+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8609+0+0 (secure 0 0 0) 0x7f5da00e3ba0 con 0x7f5da406ea10 2024-07-24T22:06:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.052+0000 7f5da951d640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 0x7f5d900b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.052+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f5da001f020 con 0x7f5da406ea10 2024-07-24T22:06:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.053+0000 7f5da951d640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 0x7f5d900b6370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005730 tx=0x7f5d940056c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:22.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.455+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d880055b0 con 0x7f5da406ea10 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/895848880 2024-07-25T21:36:35.954660+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/458230597 2024-07-25T21:36:35.954660+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/764504102 2024-07-25T21:36:35.954660+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2052402589 2024-07-25T21:34:59.599034+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/287976863 2024-07-25T21:34:59.599034+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/2613800927 2024-07-25T21:34:59.599034+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3417400078 2024-07-25T21:34:59.599034+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/317343788 2024-07-25T21:36:35.954660+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/751678544 2024-07-25T21:35:24.132100+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/458230597 2024-07-25T21:36:35.954660+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3626499665 2024-07-25T21:35:24.132100+0000 2024-07-24T22:06:22.458 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/485987782 2024-07-25T21:35:24.132100+0000 2024-07-24T22:06:22.459 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/2613800927 2024-07-25T21:34:59.599034+0000 2024-07-24T22:06:22.459 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/485987782 2024-07-25T21:35:24.132100+0000 2024-07-24T22:06:22.459 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3095475402 2024-07-25T21:35:24.132100+0000 2024-07-24T22:06:22.459 INFO:teuthology.orchestra.run.smithi060.stderr:listed 15 entries 2024-07-24T22:06:22.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.455+0000 7f5d9a7fc640 1 -- 172.21.15.60:0/205908170 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v115) ==== 81+0+879 (secure 0 0 0) 0x7f5da00aaf00 con 0x7f5da406ea10 2024-07-24T22:06:22.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.465+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 msgr2=0x7f5d900b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:22.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.465+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 0x7f5d900b6370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005730 tx=0x7f5d940056c0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.465+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 msgr2=0x7f5da411b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:22.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.465+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 0x7f5da411b430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009fd0 tx=0x7f5da000b870 comp rx=0 tx=0).stop 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.469+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 shutdown_connections 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f5d900b3eb0 0x7f5d900b6370 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da411b9d0 0x7f5da41173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da406ea10 0x7f5da411b430 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 --2- 172.21.15.60:0/205908170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da406e0f0 0x7f5da411aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 >> 172.21.15.60:0/205908170 conn(0x7f5da406c320 msgr2=0x7f5da411be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:22.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 shutdown_connections 2024-07-24T22:06:22.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:22.470+0000 7f5dab781640 1 -- 172.21.15.60:0/205908170 wait complete. 2024-07-24T22:06:22.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:22 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/205908170' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-24T22:06:22.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:22 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/205908170' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-24T22:06:22.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:22 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/205908170' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-24T22:06:23.295 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-24T22:06:23.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T22:06:23.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-24T22:06:23.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T22:06:23.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T22:06:23.388 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-24T22:06:23.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T22:06:23.430 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-07-24T22:06:23.519 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-24T22:06:23.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T22:06:23.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-24T22:06:23.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-24T22:06:23.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-07-24T22:06:23.712 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-24T22:06:23.723 INFO:tasks.cephadm:Teardown begin 2024-07-24T22:06:23.723 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-24T22:06:23.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:06:23.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:23 smithi060 ceph-mon[33000]: pgmap v996: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:06:23.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:23 smithi086 ceph-mon[37128]: pgmap v996: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:06:23.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:23 smithi123 ceph-mon[37143]: pgmap v996: 97 pgs: 18 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 41 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-24T22:06:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:25.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:25 smithi060 ceph-mon[33000]: pgmap v997: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T22:06:25.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:25 smithi086 ceph-mon[37128]: pgmap v997: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T22:06:25.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:25 smithi123 ceph-mon[37143]: pgmap v997: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-24T22:06:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.670+0000 7f75608e0640 1 -- 172.21.15.60:0/1468005054 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558032d80 con 0x7f7554094bb0 2024-07-24T22:06:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 -- 172.21.15.60:0/1468005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554094bb0 msgr2=0x7f7554094f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 --2- 172.21.15.60:0/1468005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554094bb0 0x7f7554094f90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7558009860 tx=0x7f755802c570 comp rx=0 tx=0).stop 2024-07-24T22:06:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 -- 172.21.15.60:0/1468005054 shutdown_connections 2024-07-24T22:06:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 --2- 172.21.15.60:0/1468005054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f75540a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 --2- 172.21.15.60:0/1468005054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554096fe0 0x7f7554097440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 --2- 172.21.15.60:0/1468005054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554094bb0 0x7f7554094f90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.671+0000 7f75628e4640 1 -- 172.21.15.60:0/1468005054 >> 172.21.15.60:0/1468005054 conn(0x7f755400a200 msgr2=0x7f755400ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.673+0000 7f75628e4640 1 -- 172.21.15.60:0/1468005054 shutdown_connections 2024-07-24T22:06:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.673+0000 7f75628e4640 1 -- 172.21.15.60:0/1468005054 wait complete. 2024-07-24T22:06:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.674+0000 7f75628e4640 1 Processor -- start 2024-07-24T22:06:26.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 -- start start 2024-07-24T22:06:26.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 0x7f755413ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:26.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f755413b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:26.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554140620 0x7f7554140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:26.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f75540a3740 con 0x7f7554094bb0 2024-07-24T22:06:26.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75540a35c0 con 0x7f7554140620 2024-07-24T22:06:26.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75628e4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75540a38c0 con 0x7f7554097980 2024-07-24T22:06:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.675+0000 7f75610e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f755413b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 0x7f755413ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75610e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f755413b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:41924/0 (socket says 172.21.15.60:41924) 2024-07-24T22:06:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75610e1640 1 -- 172.21.15.60:0/645096596 learned_addr learned my addr 172.21.15.60:0/645096596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75620e3640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554140620 0x7f7554140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 -- 172.21.15.60:0/645096596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 msgr2=0x7f755413b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f755413b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 -- 172.21.15.60:0/645096596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554140620 msgr2=0x7f7554140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554140620 0x7f7554140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.676+0000 7f75618e2640 1 -- 172.21.15.60:0/645096596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541411b0 con 0x7f7554094bb0 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.677+0000 7f75618e2640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 0x7f755413ad40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.679+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558041020 con 0x7f7554094bb0 2024-07-24T22:06:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.679+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558004460 con 0x7f7554094bb0 2024-07-24T22:06:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.679+0000 7f75628e4640 1 -- 172.21.15.60:0/645096596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7554141440 con 0x7f7554094bb0 2024-07-24T22:06:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.679+0000 7f75628e4640 1 -- 172.21.15.60:0/645096596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755414bf40 con 0x7f7554094bb0 2024-07-24T22:06:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.679+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580046c0 con 0x7f7554094bb0 2024-07-24T22:06:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.681+0000 7f75628e4640 1 -- 172.21.15.60:0/645096596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75400052c0 con 0x7f7554094bb0 2024-07-24T22:06:26.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.684+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f7558044020 con 0x7f7554094bb0 2024-07-24T22:06:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.685+0000 7f7552ffd640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 0x7f75480b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.685+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8609+0+0 (secure 0 0 0) 0x7f7558105010 con 0x7f7554094bb0 2024-07-24T22:06:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.687+0000 7f75610e1640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 0x7f75480b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:26.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.691+0000 7f75610e1640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 0x7f75480b63d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f755409a520 tx=0x7f754c009190 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:26.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:26.692+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f755804b030 con 0x7f7554094bb0 2024-07-24T22:06:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.141+0000 7f75628e4640 1 -- 172.21.15.60:0/645096596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75400055b0 con 0x7f7554094bb0 2024-07-24T22:06:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.143+0000 7f7552ffd640 1 -- 172.21.15.60:0/645096596 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f75580cc2e0 con 0x7f7554094bb0 2024-07-24T22:06:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:06:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:06:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:06:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.147+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 msgr2=0x7f75480b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.147+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 0x7f75480b63d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f755409a520 tx=0x7f754c009190 comp rx=0 tx=0).stop 2024-07-24T22:06:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.147+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 msgr2=0x7f755413ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.147+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 0x7f755413ad40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558032d70 comp rx=0 tx=0).stop 2024-07-24T22:06:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 shutdown_connections 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f75480b3f10 0x7f75480b63d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554140620 0x7f7554140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7554097980 0x7f755413b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 --2- 172.21.15.60:0/645096596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554094bb0 0x7f755413ad40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.148+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 >> 172.21.15.60:0/645096596 conn(0x7f755400a200 msgr2=0x7f75540980d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.149+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 shutdown_connections 2024-07-24T22:06:27.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:27.149+0000 7f7550ff9640 1 -- 172.21.15.60:0/645096596 wait complete. 2024-07-24T22:06:27.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:27 smithi086 ceph-mon[37128]: pgmap v998: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T22:06:27.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:27 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/645096596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:27 smithi060 ceph-mon[33000]: pgmap v998: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T22:06:27.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:27 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/645096596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:27.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:27 smithi123 ceph-mon[37143]: pgmap v998: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-24T22:06:27.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:27 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/645096596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:29.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-24T22:06:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:29.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:29 smithi086 ceph-mon[37128]: pgmap v999: 97 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:29.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:29 smithi060 ceph-mon[33000]: pgmap v999: 97 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:29.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:29 smithi123 ceph-mon[37143]: pgmap v999: 97 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 58 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:31.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:31 smithi086 ceph-mon[37128]: pgmap v1000: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:31.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:31 smithi060 ceph-mon[33000]: pgmap v1000: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:31.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:31 smithi123 ceph-mon[37143]: pgmap v1000: 97 pgs: 11 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 59 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-24T22:06:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 -- 172.21.15.60:0/2556534969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 msgr2=0x7f18881015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 --2- 172.21.15.60:0/2556534969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f18881015a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1884009900 tx=0x7f188402a410 comp rx=0 tx=0).stop 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 -- 172.21.15.60:0/2556534969 shutdown_connections 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 --2- 172.21.15.60:0/2556534969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 0x7f188810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 --2- 172.21.15.60:0/2556534969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f188810c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 --2- 172.21.15.60:0/2556534969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f18881015a0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 -- 172.21.15.60:0/2556534969 >> 172.21.15.60:0/2556534969 conn(0x7f1888076890 msgr2=0x7f1888076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 -- 172.21.15.60:0/2556534969 shutdown_connections 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.576+0000 7f188f2f6640 1 -- 172.21.15.60:0/2556534969 wait complete. 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 Processor -- start 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 -- start start 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f1888198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 0x7f188819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1888112740 con 0x7f18881011c0 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18881125c0 con 0x7f188810ca80 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188f2f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18881128c0 con 0x7f1888101bc0 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188d092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f1888198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188c891640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188c891640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:41946/0 (socket says 172.21.15.60:41946) 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188c891640 1 -- 172.21.15.60:0/3623445550 learned_addr learned my addr 172.21.15.60:0/3623445550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.577+0000 7f188d893640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 0x7f188819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 -- 172.21.15.60:0/3623445550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 msgr2=0x7f188819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 0x7f188819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 -- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 msgr2=0x7f1888198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f1888198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 -- 172.21.15.60:0/3623445550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188819ede0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188d092640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f1888198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188c891640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f187800e9f0 tx=0x7f187800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800ccc0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18780054d0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188819f0d0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878010430 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.578+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18881a5790 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.579+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18881015a0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.581+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f18780046c0 con 0x7f1888101bc0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.582+0000 7f187e7fc640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 0x7f18740b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.582+0000 7f188d092640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 0x7f18740b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"btime":"2024-07-24T22:01:50:734497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14709,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1887502837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1887502837},{"type":"v1","addr":"172.21.15.60:6835","nonce":1887502837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2896838776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2896838776},{"type":"v1","addr":"172.21.15.60:6837","nonce":2896838776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14760,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3968790109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3968790109},{"type":"v1","addr":"172.21.15.60:6839","nonce":3968790109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/2105555291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":2105555291},{"type":"v1","addr":"172.21.15.60:6841","nonce":2105555291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2832733292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2832733292},{"type":"v1","addr":"172.21.15.123:6835","nonce":2832733292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":18642,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3168301857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3168301857},{"type":"v1","addr":"172.21.15.123:6833","nonce":3168301857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-24T21:43:14.079489+0000","modified":"2024-07-24T22:01:50.734476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24610,"mds_1":24605,"mds_2":24598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3131020456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3131020456},{"type":"v1","addr":"172.21.15.86:6833","nonce":3131020456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/1999114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1999114136},{"type":"v1","addr":"172.21.15.86:6835","nonce":1999114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/549557535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":549557535},{"type":"v1","addr":"172.21.15.86:6837","nonce":549557535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/2950411419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2950411419},{"type":"v1","addr":"172.21.15.123:6837","nonce":2950411419}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6839/1688764220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1688764220},{"type":"v1","addr":"172.21.15.86:6839","nonce":1688764220}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.123:6839/3718566677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3718566677},{"type":"v1","addr":"172.21.15.123:6839","nonce":3718566677}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24610,"qdb_cluster":[24598,24605,24610]},"id":1}]} 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.582+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8609+0+0 (secure 0 0 0) 0x7f18780e4550 con 0x7f1888101bc0 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.583+0000 7f188d092640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 0x7f18740b6280 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1884031440 tx=0x7f1884005b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.587+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f18780555e0 con 0x7f1888101bc0 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.978+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1888199e60 con 0x7f1888101bc0 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.981+0000 7f187e7fc640 1 -- 172.21.15.60:0/3623445550 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11923 (secure 0 0 0) 0x7f1888199e60 con 0x7f1888101bc0 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 msgr2=0x7f18740b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:32.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 0x7f18740b6280 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1884031440 tx=0x7f1884005b50 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 msgr2=0x7f1888199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f187800e9f0 tx=0x7f187800eeb0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 shutdown_connections 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f18740b3dc0 0x7f18740b6280 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188810ca80 0x7f188819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1888101bc0 0x7f1888199270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 --2- 172.21.15.60:0/3623445550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18881011c0 0x7f1888198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 >> 172.21.15.60:0/3623445550 conn(0x7f1888076890 msgr2=0x7f18880fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 shutdown_connections 2024-07-24T22:06:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:31.985+0000 7f188f2f6640 1 -- 172.21.15.60:0/3623445550 wait complete. 2024-07-24T22:06:32.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:32 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3623445550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:32.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:32 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3623445550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:33.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-24T22:06:33.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:32 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3623445550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-24T22:06:33.351 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:33.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:33 smithi060 ceph-mon[33000]: pgmap v1001: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:33.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:33 smithi123 ceph-mon[37143]: pgmap v1001: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:34.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:33 smithi086 ceph-mon[37128]: pgmap v1001: 97 pgs: 7 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 72 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1782359570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 msgr2=0x7f3d98101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1782359570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d98101a70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009900 tx=0x7f3d8802a410 comp rx=0 tx=0).stop 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1782359570 shutdown_connections 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1782359570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 0x7f3d9810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1782359570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d981070a0 0x7f3d981074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1782359570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d98101a70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1782359570 >> 172.21.15.60:0/1782359570 conn(0x7f3d980fd100 msgr2=0x7f3d980ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1782359570 shutdown_connections 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.244+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1782359570 wait complete. 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 Processor -- start 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 -- start start 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d981070a0 0x7f3d980fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 0x7f3d980fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98111650 con 0x7f3d98101690 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d981114d0 con 0x7f3d98107a20 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9ea1c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3d981117d0 con 0x7f3d981070a0 2024-07-24T22:06:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51586/0 (socket says 172.21.15.60:51586) 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d97fff640 1 -- 172.21.15.60:0/1864045065 learned_addr learned my addr 172.21.15.60:0/1864045065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.245+0000 7f3d9cfb9640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 0x7f3d980fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 -- 172.21.15.60:0/1864045065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d981070a0 msgr2=0x7f3d980fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d981070a0 0x7f3d980fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 -- 172.21.15.60:0/1864045065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 msgr2=0x7f3d980fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 0x7f3d980fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 -- 172.21.15.60:0/1864045065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d981a7040 con 0x7f3d98101690 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d97fff640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005940 tx=0x7f3d880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8803f030 con 0x7f3d98101690 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d88003cb0 con 0x7f3d98101690 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88035070 con 0x7f3d98101690 2024-07-24T22:06:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d981a72d0 con 0x7f3d98101690 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.246+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d981a77b0 con 0x7f3d98101690 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.247+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d98101a70 con 0x7f3d98101690 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.248+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f3d88042030 con 0x7f3d98101690 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.249+0000 7f3d95ffb640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 0x7f3d840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.249+0000 7f3d8ffff640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 0x7f3d840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.249+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8609+0+0 (secure 0 0 0) 0x7f3d88104c90 con 0x7f3d98101690 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.250+0000 7f3d8ffff640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 0x7f3d840b6200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005fd0 tx=0x7f3d80005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.252+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f3d880cbf40 con 0x7f3d98101690 2024-07-24T22:06:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.447+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d9810a280 con 0x7f3d98101690 2024-07-24T22:06:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.448+0000 7f3d95ffb640 1 -- 172.21.15.60:0/1864045065 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23233 (secure 0 0 0) 0x7f3d880d4210 con 0x7f3d98101690 2024-07-24T22:06:35.450 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-07-24T22:06:35.451 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":115,"fsid":"646042b4-4a04-11ef-bcad-c7b262605968","created":"2024-07-24T21:34:36.682798+0000","modified":"2024-07-24T22:06:06.657352+0000","last_up_change":"2024-07-24T21:42:13.292912+0000","last_in_change":"2024-07-24T21:41:54.651817+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-24T21:38:46.032062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-24T21:43:07.304504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-24T21:43:10.965950+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":34,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5cc77b0-13c2-4164-822c-c458ae4cac91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6803","nonce":2629637807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6805","nonce":2629637807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6809","nonce":2629637807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2629637807},{"type":"v1","addr":"172.21.15.60:6807","nonce":2629637807}]},"public_addr":"172.21.15.60:6803/2629637807","cluster_addr":"172.21.15.60:6805/2629637807","heartbeat_back_addr":"172.21.15.60:6809/2629637807","heartbeat_front_addr":"172.21.15.60:6807/2629637807","state":["exists","up"]},{"osd":1,"uuid":"0a9d0f0b-703f-4b07-b1c3-6c12a00261d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6801","nonce":2205288191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6803","nonce":2205288191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6807","nonce":2205288191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2205288191},{"type":"v1","addr":"172.21.15.86:6805","nonce":2205288191}]},"public_addr":"172.21.15.86:6801/2205288191","cluster_addr":"172.21.15.86:6803/2205288191","heartbeat_back_addr":"172.21.15.86:6807/2205288191","heartbeat_front_addr":"172.21.15.86:6805/2205288191","state":["exists","up"]},{"osd":2,"uuid":"c9f91f69-71de-4bb0-81b0-a3a5da7c0733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6801","nonce":3586156659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6803","nonce":3586156659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6807","nonce":3586156659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3586156659},{"type":"v1","addr":"172.21.15.123:6805","nonce":3586156659}]},"public_addr":"172.21.15.123:6801/3586156659","cluster_addr":"172.21.15.123:6803/3586156659","heartbeat_back_addr":"172.21.15.123:6807/3586156659","heartbeat_front_addr":"172.21.15.123:6805/3586156659","state":["exists","up"]},{"osd":3,"uuid":"824da9ca-8ade-4664-8ec4-084f409bcacc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6811","nonce":1240167260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6813","nonce":1240167260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6817","nonce":1240167260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1240167260},{"type":"v1","addr":"172.21.15.60:6815","nonce":1240167260}]},"public_addr":"172.21.15.60:6811/1240167260","cluster_addr":"172.21.15.60:6813/1240167260","heartbeat_back_addr":"172.21.15.60:6817/1240167260","heartbeat_front_addr":"172.21.15.60:6815/1240167260","state":["exists","up"]},{"osd":4,"uuid":"bfc77006-c095-442e-8894-7d5901632cc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6809","nonce":1082160365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6811","nonce":1082160365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6815","nonce":1082160365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1082160365},{"type":"v1","addr":"172.21.15.86:6813","nonce":1082160365}]},"public_addr":"172.21.15.86:6809/1082160365","cluster_addr":"172.21.15.86:6811/1082160365","heartbeat_back_addr":"172.21.15.86:6815/1082160365","heartbeat_front_addr":"172.21.15.86:6813/1082160365","state":["exists","up"]},{"osd":5,"uuid":"a841523e-44a1-4112-81ab-b61d962ed02e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6809","nonce":44992122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6811","nonce":44992122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6815","nonce":44992122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":44992122},{"type":"v1","addr":"172.21.15.123:6813","nonce":44992122}]},"public_addr":"172.21.15.123:6809/44992122","cluster_addr":"172.21.15.123:6811/44992122","heartbeat_back_addr":"172.21.15.123:6815/44992122","heartbeat_front_addr":"172.21.15.123:6813/44992122","state":["exists","up"]},{"osd":6,"uuid":"cdea9ac1-bcee-44d9-be88-5b587cb1f7bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6819","nonce":2857305108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6821","nonce":2857305108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6825","nonce":2857305108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2857305108},{"type":"v1","addr":"172.21.15.60:6823","nonce":2857305108}]},"public_addr":"172.21.15.60:6819/2857305108","cluster_addr":"172.21.15.60:6821/2857305108","heartbeat_back_addr":"172.21.15.60:6825/2857305108","heartbeat_front_addr":"172.21.15.60:6823/2857305108","state":["exists","up"]},{"osd":7,"uuid":"bc920fda-05a3-4fde-92f9-efdfb7776621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6817","nonce":3700904835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6819","nonce":3700904835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6823","nonce":3700904835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3700904835},{"type":"v1","addr":"172.21.15.86:6821","nonce":3700904835}]},"public_addr":"172.21.15.86:6817/3700904835","cluster_addr":"172.21.15.86:6819/3700904835","heartbeat_back_addr":"172.21.15.86:6823/3700904835","heartbeat_front_addr":"172.21.15.86:6821/3700904835","state":["exists","up"]},{"osd":8,"uuid":"16f5ae1a-e0de-4d8d-82da-211c1a11328a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6817","nonce":2616635066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6819","nonce":2616635066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6823","nonce":2616635066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2616635066},{"type":"v1","addr":"172.21.15.123:6821","nonce":2616635066}]},"public_addr":"172.21.15.123:6817/2616635066","cluster_addr":"172.21.15.123:6819/2616635066","heartbeat_back_addr":"172.21.15.123:6823/2616635066","heartbeat_front_addr":"172.21.15.123:6821/2616635066","state":["exists","up"]},{"osd":9,"uuid":"a27b6df3-b9da-4c55-b541-2dbce29fcf6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6827","nonce":451392111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6829","nonce":451392111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6833","nonce":451392111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":451392111},{"type":"v1","addr":"172.21.15.60:6831","nonce":451392111}]},"public_addr":"172.21.15.60:6827/451392111","cluster_addr":"172.21.15.60:6829/451392111","heartbeat_back_addr":"172.21.15.60:6833/451392111","heartbeat_front_addr":"172.21.15.60:6831/451392111","state":["exists","up"]},{"osd":10,"uuid":"ce9b27b5-9ff8-4004-b2b7-fa97986a66cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6825","nonce":2411147999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6827","nonce":2411147999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6831","nonce":2411147999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2411147999},{"type":"v1","addr":"172.21.15.86:6829","nonce":2411147999}]},"public_addr":"172.21.15.86:6825/2411147999","cluster_addr":"172.21.15.86:6827/2411147999","heartbeat_back_addr":"172.21.15.86:6831/2411147999","heartbeat_front_addr":"172.21.15.86:6829/2411147999","state":["exists","up"]},{"osd":11,"uuid":"384e3184-0630-4bce-bb27-ca026eb0597d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6825","nonce":1138852346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6827","nonce":1138852346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6831","nonce":1138852346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":1138852346},{"type":"v1","addr":"172.21.15.123:6829","nonce":1138852346}]},"public_addr":"172.21.15.123:6825/1138852346","cluster_addr":"172.21.15.123:6827/1138852346","heartbeat_back_addr":"172.21.15.123:6831/1138852346","heartbeat_front_addr":"172.21.15.123:6829/1138852346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:37:58.971847+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:21.610358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:38:43.541137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:06.602714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:29.297148+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:39:51.716460+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:15.756331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:40:38.908721+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:01.447871+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:24.974051+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:41:48.427164+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-24T21:42:11.576922+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/895848880":"2024-07-25T21:36:35.954660+0000","172.21.15.60:6801/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/764504102":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/2052402589":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/287976863":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/3417400078":"2024-07-25T21:34:59.599034+0000","172.21.15.60:0/317343788":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/751678544":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/458230597":"2024-07-25T21:36:35.954660+0000","172.21.15.60:0/3626499665":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:6800/2613800927":"2024-07-25T21:34:59.599034+0000","172.21.15.60:6801/485987782":"2024-07-25T21:35:24.132100+0000","172.21.15.60:0/3095475402":"2024-07-25T21:35:24.132100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":34,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":34,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-24T22:06:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 msgr2=0x7f3d840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 0x7f3d840b6200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005fd0 tx=0x7f3d80005850 comp rx=0 tx=0).stop 2024-07-24T22:06:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 msgr2=0x7f3d980fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005940 tx=0x7f3d880042f0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 shutdown_connections 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f3d840b3d40 0x7f3d840b6200 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d98107a20 0x7f3d980fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d981070a0 0x7f3d980fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 --2- 172.21.15.60:0/1864045065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d98101690 0x7f3d980fd600 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.450+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 >> 172.21.15.60:0/1864045065 conn(0x7f3d980fd100 msgr2=0x7f3d9806b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.451+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 shutdown_connections 2024-07-24T22:06:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:35.451+0000 7f3d9ea1c640 1 -- 172.21.15.60:0/1864045065 wait complete. 2024-07-24T22:06:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:35 smithi060 ceph-mon[33000]: pgmap v1002: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:35.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:35 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1864045065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:06:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:35 smithi123 ceph-mon[37143]: pgmap v1002: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:35.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:35 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1864045065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:06:36.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:35 smithi086 ceph-mon[37128]: pgmap v1002: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-24T22:06:36.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:35 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1864045065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-24T22:06:36.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs fail cephfs 2024-07-24T22:06:36.486 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:37.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:37 smithi060 ceph-mon[33000]: pgmap v1003: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-24T22:06:37.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:37 smithi123 ceph-mon[37143]: pgmap v1003: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-24T22:06:38.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:37 smithi086 ceph-mon[37128]: pgmap v1003: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-24T22:06:38.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 -- 172.21.15.60:0/2584696331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf4107060 msgr2=0x7f9bf41074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/2584696331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf4107060 0x7f9bf41074a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009900 tx=0x7f9be002a410 comp rx=0 tx=0).stop 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 -- 172.21.15.60:0/2584696331 shutdown_connections 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/2584696331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf41079e0 0x7f9bf410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/2584696331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf4107060 0x7f9bf41074a0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/2584696331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4101650 0x7f9bf4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 -- 172.21.15.60:0/2584696331 >> 172.21.15.60:0/2584696331 conn(0x7f9bf40fd0c0 msgr2=0x7f9bf40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 -- 172.21.15.60:0/2584696331 shutdown_connections 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.270+0000 7f9bfab4a640 1 -- 172.21.15.60:0/2584696331 wait complete. 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 Processor -- start 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 -- start start 2024-07-24T22:06:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf4101650 0x7f9bf4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:38.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4107060 0x7f9bf41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf41115e0 con 0x7f9bf41079e0 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4111460 con 0x7f9bf4107060 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bfab4a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4111760 con 0x7f9bf4101650 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf90e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf90e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46730/0 (socket says 172.21.15.60:46730) 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf90e7640 1 -- 172.21.15.60:0/3591318509 learned_addr learned my addr 172.21.15.60:0/3591318509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf3fff640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4107060 0x7f9bf41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf88e6640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf4101650 0x7f9bf4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf90e7640 1 -- 172.21.15.60:0/3591318509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf4101650 msgr2=0x7f9bf4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.271+0000 7f9bf90e7640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf4101650 0x7f9bf4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf90e7640 1 -- 172.21.15.60:0/3591318509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4107060 msgr2=0x7f9bf41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf90e7640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4107060 0x7f9bf41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf90e7640 1 -- 172.21.15.60:0/3591318509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf419ed30 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf90e7640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eaa0 tx=0x7f9bec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00cdb0 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec005020 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf419f020 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0116d0 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.272+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf41a56e0 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.273+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4101a30 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.274+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f9bec0051d0 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.275+0000 7f9bf1ffb640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 0x7f9be40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.275+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8609+0+0 (secure 0 0 0) 0x7f9bec0e4410 con 0x7f9bf41079e0 2024-07-24T22:06:38.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.275+0000 7f9bf88e6640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 0x7f9be40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.276+0000 7f9bf88e6640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 0x7f9be40b6280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9be800bbe0 tx=0x7f9be800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.278+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f9bec016030 con 0x7f9bf41079e0 2024-07-24T22:06:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.516+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9bf41a5a40 con 0x7f9bf41079e0 2024-07-24T22:06:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.518+0000 7f9bf1ffb640 1 -- 172.21.15.60:0/3591318509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f9bec0ab0f0 con 0x7f9bf41079e0 2024-07-24T22:06:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-24T22:06:38.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 msgr2=0x7f9be40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:38.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 0x7f9be40b6280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9be800bbe0 tx=0x7f9be800b390 comp rx=0 tx=0).stop 2024-07-24T22:06:38.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 msgr2=0x7f9bf419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:38.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eaa0 tx=0x7f9bec00ef60 comp rx=0 tx=0).stop 2024-07-24T22:06:38.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 shutdown_connections 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f9be40b3dc0 0x7f9be40b6280 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9be800bbe0 tx=0x7f9be800b390 comp rx=0 tx=0).stop 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf41079e0 0x7f9bf419e560 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf4107060 0x7f9bf41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 --2- 172.21.15.60:0/3591318509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf4101650 0x7f9bf4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.521+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 >> 172.21.15.60:0/3591318509 conn(0x7f9bf40fd0c0 msgr2=0x7f9bf4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.522+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 shutdown_connections 2024-07-24T22:06:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:38.522+0000 7f9bfab4a640 1 -- 172.21.15.60:0/3591318509 wait complete. 2024-07-24T22:06:38.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a[32996]: 2024-07-24T22:06:38.516+0000 7f1385951640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-24T22:06:38.993 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:38.993 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:38.993 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-mds[69475]: -- 172.21.15.123:0/3378677946 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e907e3860 con 0x564e907c1000 2024-07-24T22:06:38.993 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: starting mds.c at 2024-07-24T22:06:38.994 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:38.994 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:38.994 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-mds[69774]: -- 172.21.15.123:0/693303828 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56335c855860 con 0x56335c833000 2024-07-24T22:06:38.994 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:06:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: starting mds.i at 2024-07-24T22:06:39.045 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-mds[69199]: -- 172.21.15.86:0/1913182243 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9b5755860 con 0x55f9b5733400 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: starting mds.b at 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-mds[68591]: -- 172.21.15.86:0/4262492535 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f793815860 con 0x55f7937f2800 2024-07-24T22:06:39.046 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: starting mds.e at 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: starting mds.h at 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: ignoring --setuser ceph since I am not root 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: ignoring --setgroup ceph since I am not root 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-mds[68276]: -- 172.21.15.86:0/1069201034 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec435e3860 con 0x55ec435c1400 2024-07-24T22:06:39.047 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:06:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: starting mds.k at 2024-07-24T22:06:39.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-07-24T22:06:39.514 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: pgmap v1004: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:39.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-24T22:06:39.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-24T22:06:39.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-24T22:06:39.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-24T22:06:39.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: osdmap e116: 12 total, 12 up, 12 in 2024-07-24T22:06:39.546 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:39 smithi086 ceph-mon[37128]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: pgmap v1004: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: osdmap e116: 12 total, 12 up, 12 in 2024-07-24T22:06:39.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:39 smithi060 ceph-mon[33000]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-07-24T22:06:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: pgmap v1004: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-07-24T22:06:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-24T22:06:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-24T22:06:39.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-24T22:06:39.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/3591318509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-24T22:06:39.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: osdmap e116: 12 total, 12 up, 12 in 2024-07-24T22:06:39.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:39 smithi123 ceph-mon[37143]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-07-24T22:06:40.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6836/4156971068,v1:172.21.15.123:6839/4156971068] up:boot 2024-07-24T22:06:40.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.123:6837/1769052480,v1:172.21.15.123:6838/1769052480] up:boot 2024-07-24T22:06:40.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6832/2739377467,v1:172.21.15.86:6834/2739377467] up:boot 2024-07-24T22:06:40.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6833/2090278322,v1:172.21.15.86:6835/2090278322] up:boot 2024-07-24T22:06:40.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6836/1301803352,v1:172.21.15.86:6837/1301803352] up:boot 2024-07-24T22:06:40.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: mds.? [v2:172.21.15.86:6838/1506687285,v1:172.21.15.86:6839/1506687285] up:boot 2024-07-24T22:06:40.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-07-24T22:06:40.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T22:06:40.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T22:06:40.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T22:06:40.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T22:06:40.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T22:06:40.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:40 smithi060 ceph-mon[33000]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6836/4156971068,v1:172.21.15.123:6839/4156971068] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.123:6837/1769052480,v1:172.21.15.123:6838/1769052480] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6832/2739377467,v1:172.21.15.86:6834/2739377467] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6833/2090278322,v1:172.21.15.86:6835/2090278322] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6836/1301803352,v1:172.21.15.86:6837/1301803352] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: mds.? [v2:172.21.15.86:6838/1506687285,v1:172.21.15.86:6839/1506687285] up:boot 2024-07-24T22:06:40.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T22:06:40.797 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:40 smithi086 ceph-mon[37128]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T22:06:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6836/4156971068,v1:172.21.15.123:6839/4156971068] up:boot 2024-07-24T22:06:40.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.123:6837/1769052480,v1:172.21.15.123:6838/1769052480] up:boot 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6832/2739377467,v1:172.21.15.86:6834/2739377467] up:boot 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6833/2090278322,v1:172.21.15.86:6835/2090278322] up:boot 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6836/1301803352,v1:172.21.15.86:6837/1301803352] up:boot 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: mds.? [v2:172.21.15.86:6838/1506687285,v1:172.21.15.86:6839/1506687285] up:boot 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-07-24T22:06:40.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-24T22:06:40.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-24T22:06:40.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-24T22:06:40.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-24T22:06:40.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-24T22:06:40.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:40 smithi123 ceph-mon[37143]: from='mgr.14214 172.21.15.60:0/218535122' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- 172.21.15.60:0/2860950718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff824107060 msgr2=0x7ff8241074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/2860950718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff824107060 0x7ff8241074a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff818009900 tx=0x7ff81802a410 comp rx=0 tx=0).stop 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- 172.21.15.60:0/2860950718 shutdown_connections 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/2860950718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8241079e0 0x7ff82410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/2860950718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff824107060 0x7ff8241074a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/2860950718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 0x7ff824101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- 172.21.15.60:0/2860950718 >> 172.21.15.60:0/2860950718 conn(0x7ff8240fd0c0 msgr2=0x7ff8240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:41.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- 172.21.15.60:0/2860950718 shutdown_connections 2024-07-24T22:06:41.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- 172.21.15.60:0/2860950718 wait complete. 2024-07-24T22:06:41.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 Processor -- start 2024-07-24T22:06:41.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.228+0000 7ff82a1b1640 1 -- start start 2024-07-24T22:06:41.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 0x7ff824198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:41.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff824107060 0x7ff8241991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241127c0 con 0x7ff8241079e0 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff824112640 con 0x7ff824107060 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff82a1b1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff824112940 con 0x7ff824101650 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff822ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff824107060 0x7ff8241991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46752/0 (socket says 172.21.15.60:46752) 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 -- 172.21.15.60:0/1175298493 learned_addr learned my addr 172.21.15.60:0/1175298493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff8237fe640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 0x7ff824198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 -- 172.21.15.60:0/1175298493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 msgr2=0x7ff824198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 0x7ff824198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 -- 172.21.15.60:0/1175298493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff824107060 msgr2=0x7ff8241991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff824107060 0x7ff8241991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 -- 172.21.15.60:0/1175298493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82419ed20 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.229+0000 7ff823fff640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009fd0 tx=0x7ff81c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.230+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c017070 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.230+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81c003e20 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.230+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82419f010 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.230+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c010460 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.230+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a56d0 con 0x7ff8241079e0 2024-07-24T22:06:41.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.231+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff824102900 con 0x7ff8241079e0 2024-07-24T22:06:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.232+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7ff81c003970 con 0x7ff8241079e0 2024-07-24T22:06:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.233+0000 7ff820ff9640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 0x7ff80c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.233+0000 7ff8237fe640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 0x7ff80c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.233+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8823+0+0 (secure 0 0 0) 0x7ff81c0e41b0 con 0x7ff8241079e0 2024-07-24T22:06:41.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.233+0000 7ff8237fe640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 0x7ff80c0b6280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff81400bbe0 tx=0x7ff81400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:41.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.236+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7ff81c0aadc0 con 0x7ff8241079e0 2024-07-24T22:06:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.472+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff8241097c0 con 0x7ff8241079e0 2024-07-24T22:06:41.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.526+0000 7ff820ff9640 1 -- 172.21.15.60:0/1175298493 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7ff81c0af310 con 0x7ff8241079e0 2024-07-24T22:06:41.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 msgr2=0x7ff80c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:41.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 0x7ff80c0b6280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff81400bbe0 tx=0x7ff81400b390 comp rx=0 tx=0).stop 2024-07-24T22:06:41.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 msgr2=0x7ff82419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009fd0 tx=0x7ff81c00b260 comp rx=0 tx=0).stop 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 shutdown_connections 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7ff80c0b3dc0 0x7ff80c0b6280 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8241079e0 0x7ff82419e550 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff824107060 0x7ff8241991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 --2- 172.21.15.60:0/1175298493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff824101650 0x7ff824198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.528+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 >> 172.21.15.60:0/1175298493 conn(0x7ff8240fd0c0 msgr2=0x7ff8240ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.529+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 shutdown_connections 2024-07-24T22:06:41.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:41.529+0000 7ff82a1b1640 1 -- 172.21.15.60:0/1175298493 wait complete. 2024-07-24T22:06:41.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:41 smithi086 ceph-mon[37128]: pgmap v1006: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-24T22:06:41.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:41 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T22:06:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:41 smithi060 ceph-mon[33000]: pgmap v1006: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-24T22:06:41.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:41 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T22:06:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:41 smithi123 ceph-mon[37143]: pgmap v1006: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-24T22:06:41.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:41 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-24T22:06:42.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-24T22:06:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:42 smithi086 ceph-mon[37128]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-24T22:06:42.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:42 smithi086 ceph-mon[37128]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-24T22:06:42.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:42 smithi086 ceph-mon[37128]: Cluster is now healthy 2024-07-24T22:06:42.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:42 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-24T22:06:42.796 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:42 smithi086 ceph-mon[37128]: fsmap 12 up:standby 2024-07-24T22:06:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:42 smithi060 ceph-mon[33000]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-24T22:06:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:42 smithi060 ceph-mon[33000]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-24T22:06:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:42 smithi060 ceph-mon[33000]: Cluster is now healthy 2024-07-24T22:06:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:42 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-24T22:06:42.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:42 smithi060 ceph-mon[33000]: fsmap 12 up:standby 2024-07-24T22:06:42.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:42 smithi123 ceph-mon[37143]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-24T22:06:42.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:42 smithi123 ceph-mon[37143]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-24T22:06:42.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:42 smithi123 ceph-mon[37143]: Cluster is now healthy 2024-07-24T22:06:42.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:42 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1175298493' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-24T22:06:42.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:42 smithi123 ceph-mon[37143]: fsmap 12 up:standby 2024-07-24T22:06:43.724 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:43 smithi060 ceph-mon[33000]: pgmap v1007: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T22:06:43.724 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:43 smithi060 ceph-mon[33000]: osdmap e117: 12 total, 12 up, 12 in 2024-07-24T22:06:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:43 smithi086 ceph-mon[37128]: pgmap v1007: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T22:06:43.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:43 smithi086 ceph-mon[37128]: osdmap e117: 12 total, 12 up, 12 in 2024-07-24T22:06:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:43 smithi123 ceph-mon[37143]: pgmap v1007: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-24T22:06:43.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:43 smithi123 ceph-mon[37143]: osdmap e117: 12 total, 12 up, 12 in 2024-07-24T22:06:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- 172.21.15.60:0/3457940680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81079e0 msgr2=0x7f8ad810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 --2- 172.21.15.60:0/3457940680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81079e0 0x7f8ad810e1e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009ca0 tx=0x7f8ad002a8c0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- 172.21.15.60:0/3457940680 shutdown_connections 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 --2- 172.21.15.60:0/3457940680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81079e0 0x7f8ad810e1e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 --2- 172.21.15.60:0/3457940680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad8107060 0x7f8ad81074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 --2- 172.21.15.60:0/3457940680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 0x7f8ad8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- 172.21.15.60:0/3457940680 >> 172.21.15.60:0/3457940680 conn(0x7f8ad80fd0c0 msgr2=0x7f8ad80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- 172.21.15.60:0/3457940680 shutdown_connections 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- 172.21.15.60:0/3457940680 wait complete. 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 Processor -- start 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.222+0000 7f8added5640 1 -- start start 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 0x7f8ad80fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 0x7f8ad80fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8111610 con 0x7f8ad8107060 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8111490 con 0x7f8ad8101650 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8added5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8111790 con 0x7f8ad81079e0 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46770/0 (socket says 172.21.15.60:46770) 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 -- 172.21.15.60:0/1235766091 learned_addr learned my addr 172.21.15.60:0/1235766091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8ad7fff640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 0x7f8ad80fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8ad77fe640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 0x7f8ad80fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 -- 172.21.15.60:0/1235766091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 msgr2=0x7f8ad80fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 0x7f8ad80fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 -- 172.21.15.60:0/1235766091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 msgr2=0x7f8ad80fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 0x7f8ad80fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 -- 172.21.15.60:0/1235766091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad81a7090 con 0x7f8ad8107060 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8ad7fff640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 0x7f8ad80fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.223+0000 7f8acffff640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c890 tx=0x7f8ac000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.224+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0004220 con 0x7f8ad8107060 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.224+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad81a7380 con 0x7f8ad8107060 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.224+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac00043c0 con 0x7f8ad8107060 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.224+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0004a00 con 0x7f8ad8107060 2024-07-24T22:06:44.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.224+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81a7910 con 0x7f8ad8107060 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.225+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad8101a30 con 0x7f8ad8107060 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.226+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7f8ac00079b0 con 0x7f8ad8107060 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.227+0000 7f8ad57fa640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 0x7f8ac40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.227+0000 7f8ad77fe640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 0x7f8ac40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.227+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8823+0+0 (secure 0 0 0) 0x7f8ac00e4300 con 0x7f8ad8107060 2024-07-24T22:06:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.227+0000 7f8ad77fe640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 0x7f8ac40b6200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005fd0 tx=0x7f8ac8005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:44.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.230+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7f8ac0014030 con 0x7f8ad8107060 2024-07-24T22:06:44.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.424+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ad80fe8c0 con 0x7f8ad8107060 2024-07-24T22:06:44.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.543+0000 7f8ad57fa640 1 -- 172.21.15.60:0/1235766091 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7f8ad80fe8c0 con 0x7f8ad8107060 2024-07-24T22:06:44.545 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.546+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 msgr2=0x7f8ac40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.546+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 0x7f8ac40b6200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005fd0 tx=0x7f8ac8005850 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.546+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 msgr2=0x7f8ad80fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.546+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c890 tx=0x7f8ac000cd50 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.546+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 shutdown_connections 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.547+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7f8ac40b3d40 0x7f8ac40b6200 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.547+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ad81079e0 0x7f8ad80fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.547+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad8107060 0x7f8ad80fdb00 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.547+0000 7f8added5640 1 --2- 172.21.15.60:0/1235766091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ad8101650 0x7f8ad80fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:44.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.547+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 >> 172.21.15.60:0/1235766091 conn(0x7f8ad80fd0c0 msgr2=0x7f8ad806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:44.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.548+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 shutdown_connections 2024-07-24T22:06:44.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:44.549+0000 7f8added5640 1 -- 172.21.15.60:0/1235766091 wait complete. 2024-07-24T22:06:44.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:44 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:44 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:44.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:44 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:45.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2 shell --fsid 646042b4-4a04-11ef-bcad-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-24T22:06:45.586 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/mon.a/config 2024-07-24T22:06:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:45 smithi060 ceph-mon[33000]: pgmap v1009: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-07-24T22:06:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:45 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:45.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:45 smithi060 ceph-mon[33000]: osdmap e118: 12 total, 12 up, 12 in 2024-07-24T22:06:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:45 smithi123 ceph-mon[37143]: pgmap v1009: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-07-24T22:06:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:45 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:45.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:45 smithi123 ceph-mon[37143]: osdmap e118: 12 total, 12 up, 12 in 2024-07-24T22:06:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:45 smithi086 ceph-mon[37128]: pgmap v1009: 97 pgs: 97 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-07-24T22:06:46.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:45 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1235766091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:46.046 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:45 smithi086 ceph-mon[37128]: osdmap e118: 12 total, 12 up, 12 in 2024-07-24T22:06:47.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 -- 172.21.15.60:0/754930211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b01079e0 msgr2=0x7fe2b010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 --2- 172.21.15.60:0/754930211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b01079e0 0x7fe2b010e1e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009900 tx=0x7fe2a802a410 comp rx=0 tx=0).stop 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 -- 172.21.15.60:0/754930211 shutdown_connections 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 --2- 172.21.15.60:0/754930211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b01079e0 0x7fe2b010e1e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 --2- 172.21.15.60:0/754930211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b0107060 0x7fe2b01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 --2- 172.21.15.60:0/754930211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0101650 0x7fe2b0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 -- 172.21.15.60:0/754930211 >> 172.21.15.60:0/754930211 conn(0x7fe2b00fd0c0 msgr2=0x7fe2b00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 -- 172.21.15.60:0/754930211 shutdown_connections 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 -- 172.21.15.60:0/754930211 wait complete. 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.273+0000 7fe2b7012640 1 Processor -- start 2024-07-24T22:06:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 -- start start 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 0x7fe2b010ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b01079e0 0x7fe2b010f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01127c0 con 0x7fe2b0101650 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0112640 con 0x7fe2b01079e0 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b7012640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0112940 con 0x7fe2b0107060 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 0x7fe2b010ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2affff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2affff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.60:36480/0 (socket says 172.21.15.60:36480) 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2affff640 1 -- 172.21.15.60:0/1158337244 learned_addr learned my addr 172.21.15.60:0/1158337244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-07-24T22:06:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b55af640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b01079e0 0x7fe2b010f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 -- 172.21.15.60:0/1158337244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 msgr2=0x7fe2b010f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 -- 172.21.15.60:0/1158337244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b01079e0 msgr2=0x7fe2b010f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b01079e0 0x7fe2b010f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2b4dae640 1 -- 172.21.15.60:0/1158337244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01a6ff0 con 0x7fe2b0101650 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.274+0000 7fe2affff640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2b4dae640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 0x7fe2b010ecf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009fd0 tx=0x7fe2a000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0004b20 con 0x7fe2b0101650 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a0004cc0 con 0x7fe2b0101650 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b01a72e0 con 0x7fe2b0101650 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0010720 con 0x7fe2b0101650 2024-07-24T22:06:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.275+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a7820 con 0x7fe2b0101650 2024-07-24T22:06:47.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.276+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0101a30 con 0x7fe2b0101650 2024-07-24T22:06:47.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.277+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 151190+0+0 (secure 0 0 0) 0x7fe2a0005020 con 0x7fe2b0101650 2024-07-24T22:06:47.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.278+0000 7fe2adffb640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 0x7fe29c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-24T22:06:47.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.279+0000 7fe2affff640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 0x7fe29c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-24T22:06:47.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.279+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8225+0+0 (secure 0 0 0) 0x7fe2a00e4630 con 0x7fe2b0101650 2024-07-24T22:06:47.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.279+0000 7fe2affff640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 0x7fe29c0b6280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe2b01107e0 tx=0x7fe298005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-24T22:06:47.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.283+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+193992 (secure 0 0 0) 0x7fe2a00aba60 con 0x7fe2b0101650 2024-07-24T22:06:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.476+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2b002e000 con 0x7fe2b0101650 2024-07-24T22:06:47.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.547+0000 7fe2adffb640 1 -- 172.21.15.60:0/1158337244 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7fe2a00ab360 con 0x7fe2b0101650 2024-07-24T22:06:47.549 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.551+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 msgr2=0x7fe29c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.551+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 0x7fe29c0b6280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe2b01107e0 tx=0x7fe298005f90 comp rx=0 tx=0).stop 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.551+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 msgr2=0x7fe2b010ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.551+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 0x7fe2b010ecf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009fd0 tx=0x7fe2a000ef90 comp rx=0 tx=0).stop 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 shutdown_connections 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:6800/1194501493,v1:172.21.15.60:6801/1194501493] conn(0x7fe29c0b3dc0 0x7fe29c0b6280 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2b01079e0 0x7fe2b010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2b0107060 0x7fe2b010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 --2- 172.21.15.60:0/1158337244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b0101650 0x7fe2b010ecf0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-24T22:06:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 >> 172.21.15.60:0/1158337244 conn(0x7fe2b00fd0c0 msgr2=0x7fe2b00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-24T22:06:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.552+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 shutdown_connections 2024-07-24T22:06:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-07-24T22:06:47.553+0000 7fe2b7012640 1 -- 172.21.15.60:0/1158337244 wait complete. 2024-07-24T22:06:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:47 smithi060 ceph-mon[33000]: pgmap v1011: 65 pgs: 65 active+clean; 754 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T22:06:47.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:47 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:47 smithi123 ceph-mon[37143]: pgmap v1011: 65 pgs: 65 active+clean; 754 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T22:06:47.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:47 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:48.045 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:47 smithi086 ceph-mon[37128]: pgmap v1011: 65 pgs: 65 active+clean; 754 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-07-24T22:06:48.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:47 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-24T22:06:48.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:06:48.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:06:48.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:06:48.544 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-24T22:06:48.544 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-24T22:06:48.569 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-24T22:06:48.584 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-24T22:06:48.600 INFO:tasks.cephadm:Stopping all daemons... 2024-07-24T22:06:48.601 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-07-24T22:06:48.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a 2024-07-24T22:06:48.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:48 smithi060 ceph-mon[33000]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:48.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:48 smithi060 ceph-mon[33000]: osdmap e119: 12 total, 12 up, 12 in 2024-07-24T22:06:48.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:48 smithi086 ceph-mon[37128]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:48.795 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:48 smithi086 ceph-mon[37128]: osdmap e119: 12 total, 12 up, 12 in 2024-07-24T22:06:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:48 smithi123 ceph-mon[37143]: from='client.? 172.21.15.60:0/1158337244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-24T22:06:48.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:48 smithi123 ceph-mon[37143]: osdmap e119: 12 total, 12 up, 12 in 2024-07-24T22:06:49.066 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:48 smithi060 systemd[1]: Stopping Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a[32996]: 2024-07-24T22:06:49.063+0000 7f138b15c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:06:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a[32996]: 2024-07-24T22:06:49.063+0000 7f138b15c640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-07-24T22:06:49.322 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 podman[142905]: 2024-07-24 22:06:49.107156825 +0000 UTC m=+0.177742146 container died fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:06:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 podman[142905]: 2024-07-24 22:06:49.620171489 +0000 UTC m=+0.690756809 container cleanup fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-24T22:06:49.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 bash[142905]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a 2024-07-24T22:06:50.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service' 2024-07-24T22:06:50.137 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:49 smithi060 podman[142917]: 2024-07-24 22:06:49.880414164 +0000 UTC m=+0.764676347 container remove fece5d008e5011ee169aff979188207863b6cb31651259a408a85eaab683103d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_CLEAN=True) 2024-07-24T22:06:50.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:50 smithi060 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-24T22:06:50.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:50 smithi060 systemd[1]: Stopped Ceph mon.a for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:06:50.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Jul 24 22:06:50 smithi060 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.a.service: Consumed 29.059s CPU time. 2024-07-24T22:06:50.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:50.162 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-07-24T22:06:50.162 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-07-24T22:06:50.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.b 2024-07-24T22:06:50.545 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:50 smithi086 systemd[1]: Stopping Ceph mon.b for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:50.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b[37124]: 2024-07-24T22:06:50.553+0000 7effaa410640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:06:50.820 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b[37124]: 2024-07-24T22:06:50.553+0000 7effaa410640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-07-24T22:06:51.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:50 smithi086 podman[76304]: 2024-07-24 22:06:50.81852343 +0000 UTC m=+0.381971439 container died 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1) 2024-07-24T22:06:51.676 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 podman[76304]: 2024-07-24 22:06:51.463525905 +0000 UTC m=+1.026973919 container cleanup 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:06:51.676 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 bash[76304]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b 2024-07-24T22:06:51.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.b.service' 2024-07-24T22:06:51.934 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 podman[76318]: 2024-07-24 22:06:51.673337781 +0000 UTC m=+0.852545891 container remove 4327c010b8aff6e4509636b57b11f9094fd2d77e2593b3082ae26cf647330325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=main-86385b1, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-24T22:06:51.935 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.b.service: Deactivated successfully. 2024-07-24T22:06:51.935 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 systemd[1]: Stopped Ceph mon.b for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:06:51.935 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jul 24 22:06:51 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.b.service: Consumed 17.509s CPU time. 2024-07-24T22:06:51.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:51.957 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-07-24T22:06:51.957 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-07-24T22:06:51.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.c 2024-07-24T22:06:52.243 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 systemd[1]: Stopping Ceph mon.c for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:52.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-c[37139]: 2024-07-24T22:06:52.353+0000 7fecf3c68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:06:52.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-c[37139]: 2024-07-24T22:06:52.353+0000 7fecf3c68640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-07-24T22:06:52.699 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 podman[76633]: 2024-07-24 22:06:52.425925173 +0000 UTC m=+0.235413842 container died cd1607c2f83262b066fd06774c70a7b96ca4dcfd757ec9cda94aba90b5cd23ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:06:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 podman[76633]: 2024-07-24 22:06:52.795930025 +0000 UTC m=+0.605418693 container cleanup cd1607c2f83262b066fd06774c70a7b96ca4dcfd757ec9cda94aba90b5cd23ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-24T22:06:52.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Jul 24 22:06:52 smithi123 bash[76633]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mon-c 2024-07-24T22:06:53.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mon.c.service' 2024-07-24T22:06:53.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:53.270 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-07-24T22:06:53.270 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-07-24T22:06:53.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x 2024-07-24T22:06:53.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:53 smithi060 systemd[1]: Stopping Ceph mgr.x for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:53.944 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:53 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x[33226]: 2024-07-24T22:06:53.616+0000 7f8990188640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:06:53.945 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:53 smithi060 podman[143109]: 2024-07-24 22:06:53.647883665 +0000 UTC m=+0.183741271 container died 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=main-86385b1, io.buildah.version=1.36.0) 2024-07-24T22:06:54.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:54 smithi060 podman[143109]: 2024-07-24 22:06:54.293090989 +0000 UTC m=+0.828948590 container cleanup 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, RELEASE=main-86385b1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:06:54.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:54 smithi060 bash[143109]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x 2024-07-24T22:06:54.572 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Jul 24 22:06:54 smithi060 podman[143122]: 2024-07-24 22:06:54.461606105 +0000 UTC m=+0.806652762 container remove 8c6b9dd4242ed9935a816a80e31c6d3f12cbe345a6ffd3e4b19a75282453bb09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86385b1daf5235400bbcd34a02515d168bde88e2, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-07-24T22:06:54.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.x.service' 2024-07-24T22:06:54.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:54.731 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-07-24T22:06:54.731 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-07-24T22:06:54.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.y 2024-07-24T22:06:55.045 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jul 24 22:06:54 smithi086 systemd[1]: Stopping Ceph mgr.y for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:55.370 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jul 24 22:06:55 smithi086 podman[76425]: 2024-07-24 22:06:55.080396013 +0000 UTC m=+0.156734954 container died 54c370e8ba497012b2c49532fc9a62121d8a0e846fbe09f19693d021c2ce16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:06:55.688 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jul 24 22:06:55 smithi086 podman[76425]: 2024-07-24 22:06:55.468813457 +0000 UTC m=+0.545152395 container cleanup 54c370e8ba497012b2c49532fc9a62121d8a0e846fbe09f19693d021c2ce16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-07-24T22:06:55.688 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jul 24 22:06:55 smithi086 bash[76425]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-y 2024-07-24T22:06:55.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.y.service' 2024-07-24T22:06:55.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:55.845 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-07-24T22:06:55.845 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-07-24T22:06:55.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.z 2024-07-24T22:06:56.190 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 22:06:55 smithi123 systemd[1]: Stopping Ceph mgr.z for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:56.493 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 22:06:56 smithi123 podman[76755]: 2024-07-24 22:06:56.189466501 +0000 UTC m=+0.155001305 container died 9c73f2b52023b825d1fff57dcfdce60275e75979b7af647e49d73dfc04fd6786 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, ceph=True) 2024-07-24T22:06:56.993 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 22:06:56 smithi123 podman[76755]: 2024-07-24 22:06:56.792313763 +0000 UTC m=+0.757848564 container cleanup 9c73f2b52023b825d1fff57dcfdce60275e75979b7af647e49d73dfc04fd6786 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-86385b1, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=) 2024-07-24T22:06:56.993 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Jul 24 22:06:56 smithi123 bash[76755]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mgr-z 2024-07-24T22:06:57.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mgr.z.service' 2024-07-24T22:06:57.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:06:57.169 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-07-24T22:06:57.169 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-24T22:06:57.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.0 2024-07-24T22:06:57.466 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:06:57 smithi060 systemd[1]: Stopping Ceph osd.0 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:06:57.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:06:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0[52988]: 2024-07-24T22:06:57.464+0000 7f3b3552d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:06:57.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:06:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0[52988]: 2024-07-24T22:06:57.464+0000 7f3b3552d640 -1 osd.0 119 *** Got signal Terminated *** 2024-07-24T22:06:57.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:06:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0[52988]: 2024-07-24T22:06:57.464+0000 7f3b3552d640 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:03.383 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:03 smithi060 podman[143228]: 2024-07-24 22:07:03.078204938 +0000 UTC m=+5.712995930 container died 2e4143d28ecc18d0980df96baeebfad726c39d8c4fec1d5d9b7e1a03ea4e0b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1) 2024-07-24T22:07:03.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:03 smithi060 podman[143228]: 2024-07-24 22:07:03.481479907 +0000 UTC m=+6.116270900 container cleanup 2e4143d28ecc18d0980df96baeebfad726c39d8c4fec1d5d9b7e1a03ea4e0b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0) 2024-07-24T22:07:03.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:03 smithi060 bash[143228]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0 2024-07-24T22:07:04.072 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:03 smithi060 podman[143243]: 2024-07-24 22:07:03.732743887 +0000 UTC m=+0.652264792 container remove 2e4143d28ecc18d0980df96baeebfad726c39d8c4fec1d5d9b7e1a03ea4e0b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-24T22:07:04.322 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:03.972713636 +0000 UTC m=+0.017853605 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:04.322 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:04.09199218 +0000 UTC m=+0.137132132 container create 684790644a38199c3607d6c8051257f169156c7d5963e08dd8a76332a6fb9fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=main-86385b1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-24T22:07:04.322 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:04.311811143 +0000 UTC m=+0.356951092 container init 684790644a38199c3607d6c8051257f169156c7d5963e08dd8a76332a6fb9fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-07-24T22:07:04.322 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:04.315380107 +0000 UTC m=+0.360520055 container start 684790644a38199c3607d6c8051257f169156c7d5963e08dd8a76332a6fb9fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:07:04.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:04.420221354 +0000 UTC m=+0.465361305 container attach 684790644a38199c3607d6c8051257f169156c7d5963e08dd8a76332a6fb9fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-24T22:07:04.822 INFO:journalctl@ceph.osd.0.smithi060.stdout:Jul 24 22:07:04 smithi060 podman[143331]: 2024-07-24 22:07:04.514708851 +0000 UTC m=+0.559848818 container died 684790644a38199c3607d6c8051257f169156c7d5963e08dd8a76332a6fb9fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-24T22:07:05.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.0.service' 2024-07-24T22:07:05.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:05.183 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-24T22:07:05.183 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-24T22:07:05.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.1 2024-07-24T22:07:05.500 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:05 smithi086 systemd[1]: Stopping Ceph osd.1 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:05.795 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T22:07:05.497+0000 7f9fd443b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:05.795 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T22:07:05.497+0000 7f9fd443b640 -1 osd.1 119 *** Got signal Terminated *** 2024-07-24T22:07:05.795 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1[42901]: 2024-07-24T22:07:05.497+0000 7f9fd443b640 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:11.296 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:11 smithi086 podman[76548]: 2024-07-24 22:07:11.038608147 +0000 UTC m=+5.668894993 container died 34fd2c55bf9332deb49e8a0dab0e38d538e549c48f79153885755630993f000d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1) 2024-07-24T22:07:11.701 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:11 smithi086 podman[76548]: 2024-07-24 22:07:11.439997402 +0000 UTC m=+6.070284243 container cleanup 34fd2c55bf9332deb49e8a0dab0e38d538e549c48f79153885755630993f000d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716) 2024-07-24T22:07:11.701 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:11 smithi086 bash[76548]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1 2024-07-24T22:07:12.025 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:11 smithi086 podman[76565]: 2024-07-24 22:07:11.700020867 +0000 UTC m=+0.659087923 container remove 34fd2c55bf9332deb49e8a0dab0e38d538e549c48f79153885755630993f000d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:07:12.295 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:11.926613796 +0000 UTC m=+0.018266908 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:12.295 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:12.034387713 +0000 UTC m=+0.126040820 container create bed0adb8a87993329746d8d76527688c93f355ad18290ff859544b92ad22ca5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-24T22:07:12.651 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:12.295558276 +0000 UTC m=+0.387211404 container init bed0adb8a87993329746d8d76527688c93f355ad18290ff859544b92ad22ca5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:07:12.651 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:12.298641925 +0000 UTC m=+0.390295033 container start bed0adb8a87993329746d8d76527688c93f355ad18290ff859544b92ad22ca5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716) 2024-07-24T22:07:12.651 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:12.378492263 +0000 UTC m=+0.470145382 container attach bed0adb8a87993329746d8d76527688c93f355ad18290ff859544b92ad22ca5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:07:13.008 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jul 24 22:07:12 smithi086 podman[76659]: 2024-07-24 22:07:12.650322002 +0000 UTC m=+0.741975109 container died bed0adb8a87993329746d8d76527688c93f355ad18290ff859544b92ad22ca5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-1-deactivate, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-24T22:07:13.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.1.service' 2024-07-24T22:07:13.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:13.333 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-24T22:07:13.333 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-24T22:07:13.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.2 2024-07-24T22:07:13.647 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:13 smithi123 systemd[1]: Stopping Ceph osd.2 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:13.917 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T22:07:13.645+0000 7f38c4b64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:13.917 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T22:07:13.645+0000 7f38c4b64640 -1 osd.2 119 *** Got signal Terminated *** 2024-07-24T22:07:13.917 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2[43465]: 2024-07-24T22:07:13.645+0000 7f38c4b64640 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:19.481 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:19 smithi123 podman[76882]: 2024-07-24 22:07:19.211532799 +0000 UTC m=+5.678711846 container died 3d7ca62017c77d7efb541799718b6da911649b65010e017e16428c32aa8f77e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1) 2024-07-24T22:07:19.743 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:19 smithi123 podman[76882]: 2024-07-24 22:07:19.579015825 +0000 UTC m=+6.046194873 container cleanup 3d7ca62017c77d7efb541799718b6da911649b65010e017e16428c32aa8f77e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1) 2024-07-24T22:07:19.743 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:19 smithi123 bash[76882]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2 2024-07-24T22:07:20.203 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:19 smithi123 podman[76909]: 2024-07-24 22:07:19.880423286 +0000 UTC m=+0.669490151 container remove 3d7ca62017c77d7efb541799718b6da911649b65010e017e16428c32aa8f77e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0) 2024-07-24T22:07:20.460 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.10465012 +0000 UTC m=+0.017045006 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:20.460 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.214435567 +0000 UTC m=+0.126830446 container create ee61bdf30bb5cb7ce804f114eb481b765d41c063428409d0529ed4578a2b3e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:07:20.460 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.458864014 +0000 UTC m=+0.371258890 container init ee61bdf30bb5cb7ce804f114eb481b765d41c063428409d0529ed4578a2b3e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-24T22:07:20.743 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.462113145 +0000 UTC m=+0.374508021 container start ee61bdf30bb5cb7ce804f114eb481b765d41c063428409d0529ed4578a2b3e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-07-24T22:07:20.743 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.592149768 +0000 UTC m=+0.504544662 container attach ee61bdf30bb5cb7ce804f114eb481b765d41c063428409d0529ed4578a2b3e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, RELEASE=main-86385b1, CEPH_POINT_RELEASE=) 2024-07-24T22:07:21.243 INFO:journalctl@ceph.osd.2.smithi123.stdout:Jul 24 22:07:20 smithi123 podman[76996]: 2024-07-24 22:07:20.851284657 +0000 UTC m=+0.763679543 container died ee61bdf30bb5cb7ce804f114eb481b765d41c063428409d0529ed4578a2b3e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-86385b1, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:07:21.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.2.service' 2024-07-24T22:07:21.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:21.635 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-24T22:07:21.635 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-24T22:07:21.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.3 2024-07-24T22:07:21.944 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:21 smithi060 systemd[1]: Stopping Ceph osd.3 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:22.322 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:21 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T22:07:21.942+0000 7f53c7ae4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:22.322 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:21 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T22:07:21.942+0000 7f53c7ae4640 -1 osd.3 119 *** Got signal Terminated *** 2024-07-24T22:07:22.322 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:21 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T22:07:21.942+0000 7f53c7ae4640 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:24.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:23.880+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:24.572 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:24 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:24.070+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:25.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:24.912+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:25.572 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:25.102+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:26.068 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:25 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T22:07:25.658+0000 7f53c00f1640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:57.750145+0000 front 2024-07-24T22:06:57.750121+0000 (oldest deadline 2024-07-24T22:07:25.350959+0000) 2024-07-24T22:07:26.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:25.863+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:26.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:26 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:26.067+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:26.639 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:26 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:26.386+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:26.912 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:26.593+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:27.072 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:26 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3[59875]: 2024-07-24T22:07:26.637+0000 7f53c00f1640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:57.750145+0000 front 2024-07-24T22:06:57.750121+0000 (oldest deadline 2024-07-24T22:07:25.350959+0000) 2024-07-24T22:07:27.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:26.911+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:27.427 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:27 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:27.082+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:27.819 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:27 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:27.425+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:27.820 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:27 smithi060 podman[143433]: 2024-07-24 22:07:27.544657627 +0000 UTC m=+5.705676736 container died dee4812d1905ed0797791e58b2b009dc1643e27756a7de5950b979b11b6f1db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-86385b1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716) 2024-07-24T22:07:27.955 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:27.640+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:28.072 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143433]: 2024-07-24 22:07:28.001585638 +0000 UTC m=+6.162604754 container cleanup dee4812d1905ed0797791e58b2b009dc1643e27756a7de5950b979b11b6f1db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-24T22:07:28.072 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 bash[143433]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3 2024-07-24T22:07:28.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:27.953+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:28.394 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143460]: 2024-07-24 22:07:28.303110878 +0000 UTC m=+0.757856180 container remove dee4812d1905ed0797791e58b2b009dc1643e27756a7de5950b979b11b6f1db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-24T22:07:28.395 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:28 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:28.077+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:28.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:28 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:28.103+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:28.680 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143549]: 2024-07-24 22:07:28.544630532 +0000 UTC m=+0.017892590 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:28.680 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:28 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:28.392+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:28.934 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:28.682+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:29.024 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143549]: 2024-07-24 22:07:28.679222429 +0000 UTC m=+0.152484472 container create 0916952061a7c30973888bc98284e72e6bfb3d245b703402202ed8b191910303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-07-24T22:07:29.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143549]: 2024-07-24 22:07:28.92366377 +0000 UTC m=+0.396925836 container init 0916952061a7c30973888bc98284e72e6bfb3d245b703402202ed8b191910303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240716, RELEASE=main-86385b1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:07:29.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:28 smithi060 podman[143549]: 2024-07-24 22:07:28.927583984 +0000 UTC m=+0.400846029 container start 0916952061a7c30973888bc98284e72e6bfb3d245b703402202ed8b191910303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, ceph=True) 2024-07-24T22:07:29.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:29 smithi060 podman[143549]: 2024-07-24 22:07:29.023282767 +0000 UTC m=+0.496544822 container attach 0916952061a7c30973888bc98284e72e6bfb3d245b703402202ed8b191910303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, RELEASE=main-86385b1, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-07-24T22:07:29.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:28.933+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:29.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:29 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:29.110+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:29.322 INFO:journalctl@ceph.osd.3.smithi060.stdout:Jul 24 22:07:29 smithi060 podman[143549]: 2024-07-24 22:07:29.126470439 +0000 UTC m=+0.599732491 container died 0916952061a7c30973888bc98284e72e6bfb3d245b703402202ed8b191910303 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=main-86385b1, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716) 2024-07-24T22:07:29.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:29 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:29.114+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:29.729 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:29 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:29.429+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:29.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.3.service' 2024-07-24T22:07:29.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:29.795 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-24T22:07:29.796 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-24T22:07:29.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.4 2024-07-24T22:07:29.956 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:29.684+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:30.104 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:29 smithi086 systemd[1]: Stopping Ceph osd.4 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:30.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:29.953+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:30.435 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:30 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:30.109+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:30.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:30 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:30.125+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:30.546 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:30 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T22:07:30.102+0000 7fd4af7f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:30.546 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:30 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T22:07:30.102+0000 7fd4af7f0640 -1 osd.4 119 *** Got signal Terminated *** 2024-07-24T22:07:30.547 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:30 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T22:07:30.102+0000 7fd4af7f0640 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:30.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:30 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:30.433+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:30.977 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:30.690+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:31.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:30 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:30.974+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:31.467 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:31 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:31.081+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:31.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:31 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:31.167+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:31.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:31 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:31.465+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:31.935 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:31 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:31.679+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:31.935 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:31 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:31.793+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:31.936 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:31 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:31.933+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:32.511 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:32 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:32.107+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:32.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:32 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:32.166+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:32.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:32 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:32.509+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:32.976 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:32 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:32.719+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:32.976 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:32 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:32.743+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:33.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:32 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:32.973+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:33.497 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:33 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:33.083+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:33.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:33 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:33.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:33.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:33 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:33.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:33.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:33 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:33.495+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:33.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:33 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:33.495+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:33.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:33 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:33.691+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:33.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:33 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:33.732+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:33.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:33 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:33.939+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:34.323 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:34 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:34.056+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:34.536 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:34 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:34.153+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:34.537 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:34 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:34.153+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:34.795 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:34 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4[49781]: 2024-07-24T22:07:34.535+0000 7fd4abe05640 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.391881+0000 front 2024-07-24T22:07:10.391941+0000 (oldest deadline 2024-07-24T22:07:34.491993+0000) 2024-07-24T22:07:34.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:34 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:34.546+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:34.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:34 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:34.498+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:34.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:34 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:34.498+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:34.909 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:34 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:34.763+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:34.910 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:34 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:34.653+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:35.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:34 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:34.907+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:35.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:35.066+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:35.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:35 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:35.106+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:35.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:35 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:35.106+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:35.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:35.474+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:35.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:35 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:35.474+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:35.845 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:35 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:35.579+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:35.845 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:35 smithi086 podman[76765]: 2024-07-24 22:07:35.559800184 +0000 UTC m=+5.576308889 container died 8df6f89a86a8dbdbb84e1d3c756e7b4e55577a3145d9c0d0c4b4bb14368d7244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-24T22:07:35.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:35 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:35.758+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:35.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:35 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:35.685+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:35.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:35 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:35.685+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:35.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:35 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:35.931+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:35.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:35 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:35.931+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:36.157 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:35 smithi086 podman[76765]: 2024-07-24 22:07:35.95153935 +0000 UTC m=+5.968048063 container cleanup 8df6f89a86a8dbdbb84e1d3c756e7b4e55577a3145d9c0d0c4b4bb14368d7244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-24T22:07:36.157 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:35 smithi086 bash[76765]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4 2024-07-24T22:07:36.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:36 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:36.016+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:36.518 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76782]: 2024-07-24 22:07:36.202821256 +0000 UTC m=+0.643993236 container remove 8df6f89a86a8dbdbb84e1d3c756e7b4e55577a3145d9c0d0c4b4bb14368d7244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-24T22:07:36.519 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:36 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:36.155+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:36.519 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:36 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:36.155+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:36.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:36 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:36.605+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:36.796 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.420067264 +0000 UTC m=+0.017350265 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:36.796 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.537065847 +0000 UTC m=+0.134348913 container create 29e9efedf22c1791f14fae11e2d58a2db2c4af91c7f32f81679dc5e4bd9f2f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-24T22:07:36.796 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.747818591 +0000 UTC m=+0.345101592 container init 29e9efedf22c1791f14fae11e2d58a2db2c4af91c7f32f81679dc5e4bd9f2f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-07-24T22:07:36.796 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.750774493 +0000 UTC m=+0.348057502 container start 29e9efedf22c1791f14fae11e2d58a2db2c4af91c7f32f81679dc5e4bd9f2f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-24T22:07:36.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:36 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:36.478+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:36.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:36 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:36.478+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:36.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:36 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:36.733+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:36.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:36 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:36.724+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:36.994 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:36 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:36.724+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:36.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:36 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:36.883+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:36.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:36 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:36.883+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:37.200 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.839631065 +0000 UTC m=+0.436914075 container attach 29e9efedf22c1791f14fae11e2d58a2db2c4af91c7f32f81679dc5e4bd9f2f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:07:37.200 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jul 24 22:07:36 smithi086 podman[76871]: 2024-07-24 22:07:36.935770771 +0000 UTC m=+0.533053778 container died 29e9efedf22c1791f14fae11e2d58a2db2c4af91c7f32f81679dc5e4bd9f2f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-24T22:07:37.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:36 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:36.983+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:37.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:36 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:36.983+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:37.519 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:37 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:37.199+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:37.519 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:37 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:37.199+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:37.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.4.service' 2024-07-24T22:07:37.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:37.579 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-24T22:07:37.580 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-24T22:07:37.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.5 2024-07-24T22:07:37.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:37 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:37.651+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:37.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:37 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:37.502+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:37.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:37 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:37.502+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:37.861 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:37.748+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:37.861 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 systemd[1]: Stopping Ceph osd.5 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:37.861 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:37.735+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:37.861 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:37.736+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:38.243 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:37.890+0000 7f186b583640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:38.243 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:37.890+0000 7f186b583640 -1 osd.5 119 *** Got signal Terminated *** 2024-07-24T22:07:38.243 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:37.890+0000 7f186b583640 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:38.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:37.859+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:38.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:37 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:37.859+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:38.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:37.999+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:38.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:37.999+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:38.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:38.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:38.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:38.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:38.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:38.523+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:38.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:38.523+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:38.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:38.779+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:38.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:38.698+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:38.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:38.698+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:38.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:38.845+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:38.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:38 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:38.845+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:39.045 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:38 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:38.628+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:39.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:39.018+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:39.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:39.018+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:39.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:39 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:39.227+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:39.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:39 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:39.227+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:39.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:39.549+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:39.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:39.549+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:39.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:39.668+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:39.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:39.668+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:39.994 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:39.753+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:39.994 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:39.753+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:39.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:39.882+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:39.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:39.882+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:39.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:39 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:39.882+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:40.046 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:39 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:39.582+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:40.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:39.991+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:40.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:39 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:39.991+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:40.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:40.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:40.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:40.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:40.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:40.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:40.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:40.790+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:40.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:40.790+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:40.994 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:40.679+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:40.994 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:40.679+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:40.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:40.880+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:40.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:40.880+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:40.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:40 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:40.880+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:40.996 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:40.592+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:40.996 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:40.592+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:40.997 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:40.592+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:41.045 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:40.567+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:41.045 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:40 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:40.567+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:41.319 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:41 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:41.148+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:41.319 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:41 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:41.148+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:41.319 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:41 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:41.148+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:41.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:40.995+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:41.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:40.995+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:41.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:40 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:40.995+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:41.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:41 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:41.535+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:41.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:41 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:41.535+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:41.930 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:41.673+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:41.930 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:41.673+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:41.930 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:41.790+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:41.930 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:41.790+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:41.971 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:41.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:41.971 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:41.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:41.971 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:41.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:42.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:41.927+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:42.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:41.927+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:42.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:41 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:41.927+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:42.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:41.970+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:42.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:41.970+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:42.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:41 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:41.970+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:42.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:42.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:42.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:42.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:42.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:42.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:42.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:42 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:42.557+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:42.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:42 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:42.557+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:42.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:42 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:42.557+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:42.889 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:42.627+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.755631+0000 front 2024-07-24T22:07:00.755534+0000 (oldest deadline 2024-07-24T22:07:26.055452+0000) 2024-07-24T22:07:42.890 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5[50355]: 2024-07-24T22:07:42.627+0000 7f1863b90640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.857236+0000 front 2024-07-24T22:07:08.857198+0000 (oldest deadline 2024-07-24T22:07:34.756996+0000) 2024-07-24T22:07:42.890 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:42.796+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:42.890 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:42.796+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:43.045 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:42.557+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:43.046 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:42 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:42.557+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:43.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:42.888+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:43.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:42.888+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:43.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:42 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:42.888+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:43.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:43.007+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:43.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:43.007+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:43.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:43.007+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:43.528 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:43 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:43.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:43.528 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:43 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:43.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:43.528 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:43 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:43.152+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:43.683 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:43 smithi123 podman[77099]: 2024-07-24 22:07:43.394708923 +0000 UTC m=+5.626573891 container died 60d6aae85ffbee6774c801b1932f012df77b04ade7f894083074494bb2b33b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=main-86385b1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-24T22:07:43.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:43 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:43.525+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:43.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:43 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:43.525+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:43.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:43 smithi123 podman[77099]: 2024-07-24 22:07:43.780660837 +0000 UTC m=+6.012525807 container cleanup 60d6aae85ffbee6774c801b1932f012df77b04ade7f894083074494bb2b33b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0) 2024-07-24T22:07:43.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:43 smithi123 bash[77099]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5 2024-07-24T22:07:43.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:43.827+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:43.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:43.827+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:43.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:43.854+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:43.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:43.854+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:43.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:43 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:43.854+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:44.019 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:43.600+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:44.019 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:43.600+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:44.019 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:43 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:43.600+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:44.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:44.018+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:44.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:44.018+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:44.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:44.018+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:44.392 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77117]: 2024-07-24 22:07:44.031726499 +0000 UTC m=+0.633764251 container remove 60d6aae85ffbee6774c801b1932f012df77b04ade7f894083074494bb2b33b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-24T22:07:44.509 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:44 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:44.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:44.509 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:44 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:44.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:44.509 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:44 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:44.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:44.702 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.390944343 +0000 UTC m=+0.108524084 container create f8547d51416124694bc7526aaf71fb60f197f6a2274f2b193ce11b89db7d3c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=main-86385b1) 2024-07-24T22:07:44.703 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.299721053 +0000 UTC m=+0.017300794 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:44.703 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.626990831 +0000 UTC m=+0.344570582 container init f8547d51416124694bc7526aaf71fb60f197f6a2274f2b193ce11b89db7d3c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-24T22:07:44.703 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.630434805 +0000 UTC m=+0.348014548 container start f8547d51416124694bc7526aaf71fb60f197f6a2274f2b193ce11b89db7d3c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716) 2024-07-24T22:07:44.703 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.701514541 +0000 UTC m=+0.419094293 container attach f8547d51416124694bc7526aaf71fb60f197f6a2274f2b193ce11b89db7d3c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:07:44.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:44 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:44.507+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:44.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:44 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:44.507+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:44.993 INFO:journalctl@ceph.osd.5.smithi123.stdout:Jul 24 22:07:44 smithi123 podman[77205]: 2024-07-24 22:07:44.817872465 +0000 UTC m=+0.535452206 container died f8547d51416124694bc7526aaf71fb60f197f6a2274f2b193ce11b89db7d3c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0) 2024-07-24T22:07:44.994 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:44.873+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:44.994 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:44.873+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:44.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:44.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:44.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:44.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:44.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:44 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:44.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:45.063 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:44.610+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:45.063 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:44.610+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:45.063 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:44 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:44.610+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:45.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.062+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:45.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.062+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:45.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.062+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:45.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.5.service' 2024-07-24T22:07:45.463 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:45 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:45.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:45.463 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:45 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:45.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:45.464 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:45 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:45.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:45.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:45.478 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-24T22:07:45.479 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-07-24T22:07:45.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.6 2024-07-24T22:07:45.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:45 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:45.461+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:45.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:45 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:45.461+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:45.798 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 systemd[1]: Stopping Ceph osd.6 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:45.799 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:45.611+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:45.799 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:45.611+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:45.799 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:45.611+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:46.054 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.796+0000 7fb98f8fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:46.055 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.796+0000 7fb98f8fc640 -1 osd.6 119 *** Got signal Terminated *** 2024-07-24T22:07:46.055 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:45.796+0000 7fb98f8fc640 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:46.055 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:46.052+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:46.055 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:46.052+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:46.055 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:46.052+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:46.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:45 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:45.907+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:46.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:45 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:45.907+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:46.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:45 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:45.815+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:46.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:45 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:45.815+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:46.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:45 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:45.815+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:46.489 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:46.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:46.489 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:46.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:46.489 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:46.115+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:46.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:46.487+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:46.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:46 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:46.487+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:47.006 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:46.653+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:47.006 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:46.653+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:47.006 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:46 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:46.653+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:47.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:46 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:46.860+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:47.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:46 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:46.860+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:47.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:46 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:46.775+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:47.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:46 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:46.775+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:47.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:46 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:46.775+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:47.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:47.004+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:47.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:47.004+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:47.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:47.004+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:47.517 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:47 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:47.117+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:47.518 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:47 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:47.117+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:47.518 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:47 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:47.117+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:47.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:47 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:47.516+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:47.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:47 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:47.516+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:47.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:47 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:47.893+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:47.993 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:47 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:47.893+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:47.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:47 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:47.726+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:47.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:47 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:47.726+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:47.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:47 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:47.726+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:48.046 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:47.672+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:48.047 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:47.672+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:48.047 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:47 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:47.672+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:48.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:48.044+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:48.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:48.044+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:48.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:48.044+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:48.487 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:48 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:48.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:48.487 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:48 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:48.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:48.487 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:48 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:48.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:48.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:48 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:48.485+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:48.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:48 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:48.485+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:49.059 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:48.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:49.059 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:48.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:49.059 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:48 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:48.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:49.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:48 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:48.881+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:49.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:48 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:48.881+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:49.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:48 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:48.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:49.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:48 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:48.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:49.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:48 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:48.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:49.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:49.056+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:49.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:49.056+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:49.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:49.056+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:49.480 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:49.116+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:49.481 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:49.116+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:49.481 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:49.116+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:49.481 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:49.116+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:49.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:49.478+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:49.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:49 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:49.478+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:50.041 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:49.697+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:50.041 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:49.697+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:50.041 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:49.697+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:50.041 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:49 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:49.697+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:50.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:49.889+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:50.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:49.889+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:50.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:49.773+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:50.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:49.773+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:50.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:49 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:49.773+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:50.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:50.038+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.566823+0000 front 2024-07-24T22:07:00.566857+0000 (oldest deadline 2024-07-24T22:07:24.066933+0000) 2024-07-24T22:07:50.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:50.038+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.468859+0000 front 2024-07-24T22:07:10.468873+0000 (oldest deadline 2024-07-24T22:07:36.368870+0000) 2024-07-24T22:07:50.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:50.038+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:16.869954+0000 front 2024-07-24T22:07:16.869871+0000 (oldest deadline 2024-07-24T22:07:40.970027+0000) 2024-07-24T22:07:50.322 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6[66808]: 2024-07-24T22:07:50.038+0000 7fb98bf11640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.870872+0000 front 2024-07-24T22:07:26.870950+0000 (oldest deadline 2024-07-24T22:07:49.170995+0000) 2024-07-24T22:07:50.503 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:50.077+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:50.503 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:50.077+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:50.503 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:50.077+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:50.503 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:50.077+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:50.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:50.500+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:50.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:50.500+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:50.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:50 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:50.500+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:51.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:50.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:51.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:50.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:51.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:50.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:51.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:50 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:50.683+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:51.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:50.864+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:51.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:50.864+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:51.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:50.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:51.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:50.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:51.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:50 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:50.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:51.295 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:51.042+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:51.295 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:51.042+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:51.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:51.042+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:51.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:51.042+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:51.694 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:51 smithi060 podman[143656]: 2024-07-24 22:07:51.422208221 +0000 UTC m=+5.733848604 container died 7e0b132b80944ed69684ff4ca2cc6631533bff98a0b2b9682351859c2471cc11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=main-86385b1) 2024-07-24T22:07:51.759 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:51.454+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:51.759 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:51.454+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:51.759 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:51.454+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:52.072 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:51 smithi060 podman[143656]: 2024-07-24 22:07:51.8973155 +0000 UTC m=+6.208955912 container cleanup 7e0b132b80944ed69684ff4ca2cc6631533bff98a0b2b9682351859c2471cc11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, io.buildah.version=1.36.0) 2024-07-24T22:07:52.072 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:51 smithi060 bash[143656]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6 2024-07-24T22:07:52.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:51 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:51.692+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:52.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:51 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:51.692+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:52.073 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:51 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:51.692+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:52.073 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:51 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:51.692+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:52.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:51.850+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:52.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:51.850+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:52.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:51.850+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:52.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:51.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:52.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:51.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:52.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:51 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:51.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:52.323 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:52.060+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:52.323 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:52.060+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:52.323 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:52.060+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:52.323 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:52.060+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:52.536 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143670]: 2024-07-24 22:07:52.182718968 +0000 UTC m=+0.756162586 container remove 7e0b132b80944ed69684ff4ca2cc6631533bff98a0b2b9682351859c2471cc11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716) 2024-07-24T22:07:52.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:52.448+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:52.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:52.448+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:52.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:52 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:52.448+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:52.811 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:52 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:52.660+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:52 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:52.660+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:52 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:52.660+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:52 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:52.660+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143761]: 2024-07-24 22:07:52.437553037 +0000 UTC m=+0.017216476 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143761]: 2024-07-24 22:07:52.549374341 +0000 UTC m=+0.129037765 container create 9693490152cfc616abd5a5843cc5229d873ff0fcb16a29ca84e1992b4d7b8af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:07:52.812 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143761]: 2024-07-24 22:07:52.810684162 +0000 UTC m=+0.390347586 container init 9693490152cfc616abd5a5843cc5229d873ff0fcb16a29ca84e1992b4d7b8af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-24T22:07:53.072 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143761]: 2024-07-24 22:07:52.814149361 +0000 UTC m=+0.393812791 container start 9693490152cfc616abd5a5843cc5229d873ff0fcb16a29ca84e1992b4d7b8af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:07:53.072 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:52 smithi060 podman[143761]: 2024-07-24 22:07:52.918871847 +0000 UTC m=+0.498535279 container attach 9693490152cfc616abd5a5843cc5229d873ff0fcb16a29ca84e1992b4d7b8af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-07-24T22:07:53.072 INFO:journalctl@ceph.osd.6.smithi060.stdout:Jul 24 22:07:53 smithi060 podman[143761]: 2024-07-24 22:07:53.010903106 +0000 UTC m=+0.590566552 container died 9693490152cfc616abd5a5843cc5229d873ff0fcb16a29ca84e1992b4d7b8af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:07:53.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:52.833+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:53.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:52.833+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:53.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:52.833+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:53.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:52.766+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:53.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:52.766+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:53.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:52 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:52.766+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:53.427 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:53.088+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:53.427 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:53.088+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:53.427 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:53.088+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:53.427 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:53.088+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:53.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.6.service' 2024-07-24T22:07:53.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:07:53.634 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-07-24T22:07:53.634 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-07-24T22:07:53.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.7 2024-07-24T22:07:53.684 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.425+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:53.684 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.425+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:53.684 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.425+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:53.938 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 systemd[1]: Stopping Ceph osd.7 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:07:54.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:53 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:53.620+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:54.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:53 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:53.620+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:54.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:53 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:53.620+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:54.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:53 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:53.620+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:54.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:53.847+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:54.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:53.847+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:54.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:53.847+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:54.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:53.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:54.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:53.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:54.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:53 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:53.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:54.295 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.936+0000 7f4f053b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:07:54.295 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.936+0000 7f4f053b4640 -1 osd.7 119 *** Got signal Terminated *** 2024-07-24T22:07:54.295 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:53 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:53.936+0000 7f4f053b4640 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:07:54.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:54.082+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:54.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:54.082+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:54.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:54.082+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:54.296 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:54.082+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:54.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:54.444+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:54.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:54.444+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:54.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:54 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:54.444+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:55.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:54.616+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:55.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:54.616+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:55.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:54.616+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:55.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:54 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:54.616+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:55.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:54.859+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:55.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:54.859+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:55.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:54.860+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:55.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:54.844+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:55.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:54.844+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:55.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:54 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:54.844+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:55.406 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:55.090+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:55.406 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:55.090+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:55.406 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:55.090+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:55.406 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:55.090+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:55.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:55.403+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:55.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:55.403+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:55.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:55.403+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:56.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:55 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:55.659+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:56.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:55 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:55.659+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:56.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:55 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:55.659+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:56.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:55 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:55.659+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:56.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:55.814+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:56.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:55.814+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:56.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:55.814+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:56.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:55.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:56.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:55.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:56.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:55 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:55.832+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:56.398 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:56.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:56.398 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:56.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:56.398 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:56.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:56.399 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:56.109+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:56.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:56.396+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:56.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:56.396+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:56.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:56 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:56.396+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:57.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:56.670+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:57.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:56.670+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:57.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:56.670+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:57.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:56.670+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:57.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:56 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:56.670+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:07:57.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:56.783+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:56.783+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:56.783+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:56.783+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:56.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:56.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:56.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:57.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:56 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:56.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:07:57.412 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:57.157+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:57.412 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:57.157+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:57.412 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:57.157+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:57.412 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:57.157+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:57.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:57.410+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:57.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:57.410+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:57.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:57.410+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:57.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:57 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:57.410+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:30.526429+0000 front 2024-07-24T22:07:30.526442+0000 (oldest deadline 2024-07-24T22:07:56.426607+0000) 2024-07-24T22:07:58.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:57.661+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:58.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:57.661+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:58.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:57.661+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:58.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:57.661+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:58.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:57 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:57.661+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:07:58.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:57.775+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:58.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:57.775+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:57.775+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:57.775+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:57.846+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:57.846+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:57.846+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:58.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:57 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:57.846+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:07:58.449 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:58.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:58.449 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:58.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:58.450 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:58.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:58.450 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:58.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:58.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:58.447+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:08.522234+0000 front 2024-07-24T22:07:08.522200+0000 (oldest deadline 2024-07-24T22:07:34.422336+0000) 2024-07-24T22:07:58.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:58.447+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.124645+0000 front 2024-07-24T22:07:17.124452+0000 (oldest deadline 2024-07-24T22:07:40.024467+0000) 2024-07-24T22:07:58.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:58.447+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.325621+0000 front 2024-07-24T22:07:25.325544+0000 (oldest deadline 2024-07-24T22:07:50.025593+0000) 2024-07-24T22:07:58.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:58 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7[56597]: 2024-07-24T22:07:58.447+0000 7f4efd9c1640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:30.526429+0000 front 2024-07-24T22:07:30.526442+0000 (oldest deadline 2024-07-24T22:07:56.426607+0000) 2024-07-24T22:07:59.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:58 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:58.624+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:07:59.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:58 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:58.624+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:07:59.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:58 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:58.624+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:07:59.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:58 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:58.624+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:07:59.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:58 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:58.624+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:07:59.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:58.785+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:07:59.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:58.785+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:07:59.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:58.785+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:07:59.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:58.785+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:07:59.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:58.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:07:59.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:58.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:07:59.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:58.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:07:59.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:58 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:58.813+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:07:59.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:59.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:07:59.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:59.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:07:59.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:59.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:07:59.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:07:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:07:59.183+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:07:59.872 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:59 smithi086 podman[76985]: 2024-07-24 22:07:59.579725044 +0000 UTC m=+5.746014335 container died 46158284376ba77f769c87a244665a8280c9b27a16d674d59324331dd46f745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:08:00.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:59 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:59.586+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:00.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:59 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:59.586+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:00.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:59 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:59.586+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:00.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:59 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:59.586+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:00.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:07:59 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:07:59.586+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:00.160 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:59 smithi086 podman[76985]: 2024-07-24 22:07:59.870784669 +0000 UTC m=+6.037073966 container cleanup 46158284376ba77f769c87a244665a8280c9b27a16d674d59324331dd46f745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-24T22:08:00.161 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:07:59 smithi086 bash[76985]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7 2024-07-24T22:08:00.161 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77000]: 2024-07-24 22:08:00.096680036 +0000 UTC m=+0.515448436 container remove 46158284376ba77f769c87a244665a8280c9b27a16d674d59324331dd46f745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, ceph=True) 2024-07-24T22:08:00.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:59.794+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:00.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:59.794+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:59.794+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:07:59.795+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:59.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:59.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:59.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:00.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:07:59 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:07:59.764+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:00.415 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.311386978 +0000 UTC m=+0.016958060 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:08:00.415 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:00 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:00.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:00.415 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:00 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:00.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:00.416 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:00 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:00.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:00.416 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:00 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:00.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:00.416 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:00 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:00.159+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:00.667 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.414295766 +0000 UTC m=+0.119866835 container create cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-24T22:08:00.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:00.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:00.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:00.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:00.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:00.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:00.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:00.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:00.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:00 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:00.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:01.045 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.666693926 +0000 UTC m=+0.372265004 container init cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240716, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-24T22:08:01.046 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.670352399 +0000 UTC m=+0.375923472 container start cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-24T22:08:01.046 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.783496759 +0000 UTC m=+0.489067831 container attach cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-86385b1, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716) 2024-07-24T22:08:01.046 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:00 smithi086 podman[77088]: 2024-07-24 22:08:00.855467328 +0000 UTC m=+0.561038401 container died cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=main-86385b1) 2024-07-24T22:08:01.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:00.768+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:01.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:00.768+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:01.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:00.768+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:01.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:00.768+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:01.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:00.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:01.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:00.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:01.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:00.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:01.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:00 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:00.776+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:01.397 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jul 24 22:08:01 smithi086 podman[77111]: 2024-07-24 22:08:01.378543569 +0000 UTC m=+0.517973434 container remove cd5c0a3b12066e6cdbbf5c6773e53de11d9b1ed5e780f4d873d05a14e12dc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, ceph=True, org.label-schema.build-date=20240716) 2024-07-24T22:08:01.397 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:01 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:01.195+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:01.397 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:01 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:01.195+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:01.397 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:01 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:01.195+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:01.398 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:01 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:01.195+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:01.398 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:01 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:01.195+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:01.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.7.service' 2024-07-24T22:08:01.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:01.439 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-07-24T22:08:01.440 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-07-24T22:08:01.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.8 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 systemd[1]: Stopping Ceph osd.8 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.706+0000 7f37c8e83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.706+0000 7f37c8e83640 -1 osd.8 119 *** Got signal Terminated *** 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.706+0000 7f37c8e83640 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.731+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.731+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:01.733 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.731+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:01.734 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:01.731+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:01.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:01.784+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:01.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:01.784+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:01.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:01.784+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:01.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:01 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:01.784+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:02.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:01.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:02.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:01.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:02.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:01.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:02.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:01.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:02.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:01 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:01.579+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:02.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:02 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:02.158+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:02.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:02 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:02.158+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:02.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:02 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:02.158+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:02.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:02 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:02.158+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:02.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:02 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:02.158+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:02.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:02 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:02.561+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:02.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:02 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:02.561+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:02.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:02 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:02.561+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:02.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:02 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:02.561+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:02.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:02 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:02.561+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:03.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:02.777+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:03.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:02.777+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:02.777+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:02.777+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:02.805+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:02.805+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:02.805+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:03.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:02 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:02.805+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:03.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:03.111+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:03.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:03.111+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:03.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:03.111+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:03.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:03.111+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:03.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:03.111+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:03.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:03 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:03.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:03.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:03 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:03.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:03.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:03 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:03.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:03.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:03 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:03.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:03.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:03 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:03.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:04.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:03.752+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:04.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:03.752+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:04.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:03.752+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:04.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:03.752+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:04.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:03.842+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:04.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:03.842+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:04.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:03.842+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:04.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:03 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:03.842+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:04.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:04 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:04.074+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:04.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:04 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:04.074+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:04.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:04 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:04.074+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:04.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:04 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:04.074+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:04.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:04 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:04.074+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:04.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:04.491+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:04.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:04.491+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:04.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:04.491+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:04.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:04.491+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:04.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:04 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:04.491+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:05.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:04.776+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:05.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:04.776+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:05.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:04.776+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:05.243 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:04.776+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:05.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:04.857+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:05.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:04.857+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:05.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:04.857+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:05.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:04 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:04.857+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:05.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:05.097+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:05.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:05.097+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:05.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:05.097+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:05.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:05.097+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:05.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:05 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:05.097+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:05.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:05.496+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:05.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:05.496+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:05.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:05.496+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:05.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:05.496+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:05.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:05 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:05.496+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:06.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:05.812+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:06.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:05.812+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:05.812+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:05.812+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:05.774+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:07.583930+0000 front 2024-07-24T22:07:07.583811+0000 (oldest deadline 2024-07-24T22:07:31.683884+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:05.774+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.784716+0000 front 2024-07-24T22:07:15.784687+0000 (oldest deadline 2024-07-24T22:07:39.285002+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:05.774+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:26.187301+0000 front 2024-07-24T22:07:26.187382+0000 (oldest deadline 2024-07-24T22:07:50.887364+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:05.774+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.289102+0000 front 2024-07-24T22:07:34.289292+0000 (oldest deadline 2024-07-24T22:07:56.589226+0000) 2024-07-24T22:08:06.244 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:05 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8[57310]: 2024-07-24T22:08:05.774+0000 7f37c1490640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.291604+0000 front 2024-07-24T22:07:42.291626+0000 (oldest deadline 2024-07-24T22:08:05.191848+0000) 2024-07-24T22:08:06.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:06 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:06.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:06.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:06 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:06.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:06.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:06 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:06.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:06.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:06 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:06.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:06.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:06 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:06.135+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:06.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:06 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:06.503+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:06.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:06 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:06.503+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:06.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:06 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:06.503+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:06.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:06 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:06.503+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:06.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:06 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:06.503+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:07.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:06 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:06.808+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:07.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:06 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:06.808+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:07.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:06 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:06.808+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:07.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:06 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:06.808+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:07.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:06 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:06.808+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:07.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:07.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:07.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:07.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:07.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:07.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:07 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:07.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:07.648 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:07 smithi123 podman[77310]: 2024-07-24 22:08:07.364837783 +0000 UTC m=+5.743755072 container died f85605850ded0aacf8ddd0b1674bcdced6f8155673bd735493990e82df5872af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-07-24T22:08:07.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:07.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:07.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:07.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:07.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:07.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:07 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:07.529+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:07 smithi123 podman[77310]: 2024-07-24 22:08:07.647936684 +0000 UTC m=+6.026853967 container cleanup f85605850ded0aacf8ddd0b1674bcdced6f8155673bd735493990e82df5872af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:07 smithi123 bash[77310]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:07.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:07.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:07.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:07.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:07.900 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:07.798+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:08.193 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:07 smithi123 podman[77329]: 2024-07-24 22:08:07.899150524 +0000 UTC m=+0.528472561 container remove f85605850ded0aacf8ddd0b1674bcdced6f8155673bd735493990e82df5872af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-24T22:08:08.493 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.192006971 +0000 UTC m=+0.086004013 container create 1809a47acb43b956d66cbd05fe6c3e122c9efbd375e2054dacc3a20911883b61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:08:08.493 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.123303597 +0000 UTC m=+0.017300645 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:08:08.493 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.427491343 +0000 UTC m=+0.321488382 container init 1809a47acb43b956d66cbd05fe6c3e122c9efbd375e2054dacc3a20911883b61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-24T22:08:08.493 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.430637659 +0000 UTC m=+0.324634698 container start 1809a47acb43b956d66cbd05fe6c3e122c9efbd375e2054dacc3a20911883b61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:08:08.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:08.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:08.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:08.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:08.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:08.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:08 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:08.161+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:08.773 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.522903411 +0000 UTC m=+0.416900469 container attach 1809a47acb43b956d66cbd05fe6c3e122c9efbd375e2054dacc3a20911883b61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, ceph=True) 2024-07-24T22:08:08.773 INFO:journalctl@ceph.osd.8.smithi123.stdout:Jul 24 22:08:08 smithi123 podman[77417]: 2024-07-24 22:08:08.61837391 +0000 UTC m=+0.512370962 container died 1809a47acb43b956d66cbd05fe6c3e122c9efbd375e2054dacc3a20911883b61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-24T22:08:08.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:08.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:08.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:08.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:08.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:08.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:08 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:08.565+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:09.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:08.771+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:09.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:08.771+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:09.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:08.771+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:09.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:08.771+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:09.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:08 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:08.771+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:09.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.8.service' 2024-07-24T22:08:09.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:09.248 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-07-24T22:08:09.248 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-07-24T22:08:09.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.9 2024-07-24T22:08:09.540 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 systemd[1]: Stopping Ceph osd.9 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:09.540 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:09.540 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:09.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:09.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:09.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:09.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:09.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:09.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:09 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:09.181+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:09.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:09.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:09.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:09.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.537+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:09.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.558+0000 7fe1fa61a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:09.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.558+0000 7fe1fa61a640 -1 osd.9 119 *** Got signal Terminated *** 2024-07-24T22:08:09.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:09 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:09.558+0000 7fe1fa61a640 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:08:10.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:09 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:09.746+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:10.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:09 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:09.746+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:10.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:09 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:09.746+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:10.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:09 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:09.746+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:10.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:09 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:09.746+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:10.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:10.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:10.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:10.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:10.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:10.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:10 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:10.217+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:11.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:10 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:10.581+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:11.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:10 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:10.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:11.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:10 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:10.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:11.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:10 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:10.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:11.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:10 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:10.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:11.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:10 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:10.752+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:11.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:11.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:11.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:11.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:11.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:11.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:11 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:11.248+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:12.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:11 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:11.570+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:12.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:11.788+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:12.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:11.788+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:12.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:11.788+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:12.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:11.788+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:12.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:11.788+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:12.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:12.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:12.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:12.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:12.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:12.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:12 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:12.261+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:12.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:12.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:12.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:12.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:12.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:12.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:12.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:12 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:12.545+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:49.956231+0000 front 2024-07-24T22:07:49.956263+0000 (oldest deadline 2024-07-24T22:08:12.256315+0000) 2024-07-24T22:08:12.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:12.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:12.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:12.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:12.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:12.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:12 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:12.740+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:13.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:13.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:13.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:13.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:13.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:13.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:13 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:13.283+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:13.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:13.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:13 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:13.559+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:49.956231+0000 front 2024-07-24T22:07:49.956263+0000 (oldest deadline 2024-07-24T22:08:12.256315+0000) 2024-07-24T22:08:13.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:13.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:13.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:13.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:13.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:13.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:13 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:13.704+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:14.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:14.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:14.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:14.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:14.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:14.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:14 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:14.266+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:14.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:00.348229+0000 front 2024-07-24T22:07:00.348355+0000 (oldest deadline 2024-07-24T22:07:25.648415+0000) 2024-07-24T22:08:14.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.149634+0000 front 2024-07-24T22:07:09.149596+0000 (oldest deadline 2024-07-24T22:07:32.649821+0000) 2024-07-24T22:08:14.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:17.351153+0000 front 2024-07-24T22:07:17.351068+0000 (oldest deadline 2024-07-24T22:07:39.651251+0000) 2024-07-24T22:08:14.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:24.852510+0000 front 2024-07-24T22:07:24.852586+0000 (oldest deadline 2024-07-24T22:07:49.552600+0000) 2024-07-24T22:08:14.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:31.853756+0000 front 2024-07-24T22:07:31.853982+0000 (oldest deadline 2024-07-24T22:07:55.953753+0000) 2024-07-24T22:08:14.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:41.755445+0000 front 2024-07-24T22:07:41.755421+0000 (oldest deadline 2024-07-24T22:08:07.055308+0000) 2024-07-24T22:08:14.823 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:14 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9[73601]: 2024-07-24T22:08:14.526+0000 7fe1f2c27640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:49.956231+0000 front 2024-07-24T22:07:49.956263+0000 (oldest deadline 2024-07-24T22:08:12.256315+0000) 2024-07-24T22:08:14.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:14.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:14.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:14.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:14.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:14.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:14 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:14.689+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:15.352 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:15 smithi060 podman[143875]: 2024-07-24 22:08:15.073983751 +0000 UTC m=+5.621033805 container died 6d335ff6ad2b6514dc30672549ec18ff7b840f5b8557c6c6aadbd8cb20190267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9, RELEASE=main-86385b1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-24T22:08:15.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:15.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:15 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:15.219+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:15.819 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:15 smithi060 podman[143875]: 2024-07-24 22:08:15.575308337 +0000 UTC m=+6.122358387 container cleanup 6d335ff6ad2b6514dc30672549ec18ff7b840f5b8557c6c6aadbd8cb20190267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD) 2024-07-24T22:08:15.819 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:15 smithi060 bash[143875]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9 2024-07-24T22:08:15.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.713+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:15.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.713+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:15.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.713+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:15.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.713+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:15.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.713+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:15.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:15 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:15.714+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:16.072 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:15 smithi060 podman[143891]: 2024-07-24 22:08:15.818142965 +0000 UTC m=+0.738141856 container remove 6d335ff6ad2b6514dc30672549ec18ff7b840f5b8557c6c6aadbd8cb20190267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-24T22:08:16.456 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.043641743 +0000 UTC m=+0.017751954 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:08:16.456 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.169642862 +0000 UTC m=+0.143753128 container create 392da15721d165ea19cf5e603027279cbb86b5b05798a9c13656af0752577d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9-deactivate, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-07-24T22:08:16.456 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.372245653 +0000 UTC m=+0.346355884 container init 392da15721d165ea19cf5e603027279cbb86b5b05798a9c13656af0752577d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-24T22:08:16.457 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.375718719 +0000 UTC m=+0.349828923 container start 392da15721d165ea19cf5e603027279cbb86b5b05798a9c13656af0752577d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:08:16.457 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.455070022 +0000 UTC m=+0.429180234 container attach 392da15721d165ea19cf5e603027279cbb86b5b05798a9c13656af0752577d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:16.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:16 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:16.196+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:16.822 INFO:journalctl@ceph.osd.9.smithi060.stdout:Jul 24 22:08:16 smithi060 podman[143981]: 2024-07-24 22:08:16.579703036 +0000 UTC m=+0.553813240 container died 392da15721d165ea19cf5e603027279cbb86b5b05798a9c13656af0752577d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1) 2024-07-24T22:08:16.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:16.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:16.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:16.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:16.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:16.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:16 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:16.729+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:17.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.9.service' 2024-07-24T22:08:17.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:17.227 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-07-24T22:08:17.227 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-07-24T22:08:17.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.10 2024-07-24T22:08:17.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.193+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:17.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 systemd[1]: Stopping Ceph osd.10 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:18.045 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.556+0000 7f3dac0f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:18.045 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.556+0000 7f3dac0f1640 -1 osd.10 119 *** Got signal Terminated *** 2024-07-24T22:08:18.046 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:17 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:17.556+0000 7f3dac0f1640 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:08:18.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:18.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:18.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:18.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:18.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:18.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:17.743+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:18.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:18.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:18 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:18.178+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:19.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:19.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:19.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:19.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:19.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:19.244 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:18 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:18.756+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:19.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:19.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:19 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:19.149+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:19.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:19.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:19.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:19.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:19.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:19.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:19.738+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:20.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:20.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:20 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:20.160+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:20.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:20.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:20.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:20.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:20.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:20.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:20.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:20 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:20.710+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:21.501 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:21 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:21.141+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:21.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:21.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:21.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:21.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:21.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:21.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:21.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:21 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:21.666+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:22.545 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:07:02.006546+0000 front 2024-07-24T22:07:02.006291+0000 (oldest deadline 2024-07-24T22:07:27.306402+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:10.207352+0000 front 2024-07-24T22:07:10.207448+0000 (oldest deadline 2024-07-24T22:07:33.107487+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.408374+0000 front 2024-07-24T22:07:15.408650+0000 (oldest deadline 2024-07-24T22:07:40.108442+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-07-24T22:07:25.409467+0000 front 2024-07-24T22:07:25.409331+0000 (oldest deadline 2024-07-24T22:07:48.309423+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:33.411256+0000 front 2024-07-24T22:07:33.411274+0000 (oldest deadline 2024-07-24T22:07:59.211767+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:40.913063+0000 front 2024-07-24T22:07:40.913014+0000 (oldest deadline 2024-07-24T22:08:06.212967+0000) 2024-07-24T22:08:22.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:22 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10[63600]: 2024-07-24T22:08:22.190+0000 7f3da8706640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:48.513878+0000 front 2024-07-24T22:07:48.513951+0000 (oldest deadline 2024-07-24T22:08:14.314106+0000) 2024-07-24T22:08:22.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:22.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:22.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:22.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:22.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:22.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:22.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:22 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:22.674+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:23.425 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:23 smithi086 podman[77202]: 2024-07-24 22:08:23.140858027 +0000 UTC m=+5.712768954 container died f8bcd136460272180b673254bce8b9edbd27b1b8c45a2b34dc03906a0a8c93b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-24T22:08:23.758 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:23 smithi086 podman[77202]: 2024-07-24 22:08:23.506752023 +0000 UTC m=+6.078662951 container cleanup f8bcd136460272180b673254bce8b9edbd27b1b8c45a2b34dc03906a0a8c93b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-24T22:08:23.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:23 smithi086 bash[77202]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:23.978 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:23 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:23.697+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:24.045 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:23 smithi086 podman[77218]: 2024-07-24 22:08:23.757796312 +0000 UTC m=+0.609541478 container remove f8bcd136460272180b673254bce8b9edbd27b1b8c45a2b34dc03906a0a8c93b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-24T22:08:24.420 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:23.978106884 +0000 UTC m=+0.016860019 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:08:24.421 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:24.084022018 +0000 UTC m=+0.122775192 container create 62c9be2adf0376dfad4884c6c616b6a3304618b6df1ee3f40c1a4638d30536c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:24.421 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:24.311190503 +0000 UTC m=+0.349943632 container init 62c9be2adf0376dfad4884c6c616b6a3304618b6df1ee3f40c1a4638d30536c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-24T22:08:24.421 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:24.314154033 +0000 UTC m=+0.352907161 container start 62c9be2adf0376dfad4884c6c616b6a3304618b6df1ee3f40c1a4638d30536c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-86385b1, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:24.421 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:24.41948194 +0000 UTC m=+0.458235086 container attach 62c9be2adf0376dfad4884c6c616b6a3304618b6df1ee3f40c1a4638d30536c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-07-24T22:08:24.795 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jul 24 22:08:24 smithi086 podman[77312]: 2024-07-24 22:08:24.50191976 +0000 UTC m=+0.540672890 container died 62c9be2adf0376dfad4884c6c616b6a3304618b6df1ee3f40c1a4638d30536c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-10-deactivate, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:24.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:24.995 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:24 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:24.706+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:25.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.10.service' 2024-07-24T22:08:25.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:25.111 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-07-24T22:08:25.111 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-07-24T22:08:25.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.11 2024-07-24T22:08:25.417 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 systemd[1]: Stopping Ceph osd.11 for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:25.686 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.415+0000 7fca40425640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:25.686 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.415+0000 7fca40425640 -1 osd.11 119 *** Got signal Terminated *** 2024-07-24T22:08:25.686 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.415+0000 7fca40425640 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-24T22:08:25.686 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:25.687 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:25 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:25.683+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:26.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:26.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:26.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:26.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:26.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:26.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:26.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:26 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:26.664+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:27.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:27.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:27.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:27.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:27.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:27.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:27.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:27 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:27.616+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:28.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:28.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:28.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:28.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:28.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:28.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:28.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:28.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:28 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:28.608+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2024-07-24T22:08:01.501147+0000 front 2024-07-24T22:08:01.501169+0000 (oldest deadline 2024-07-24T22:08:27.901591+0000) 2024-07-24T22:08:29.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-07-24T22:06:59.592579+0000 front 2024-07-24T22:06:59.592689+0000 (oldest deadline 2024-07-24T22:07:23.593326+0000) 2024-07-24T22:08:29.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-07-24T22:07:09.494652+0000 front 2024-07-24T22:07:09.494457+0000 (oldest deadline 2024-07-24T22:07:35.394553+0000) 2024-07-24T22:08:29.993 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-07-24T22:07:15.394830+0000 front 2024-07-24T22:07:15.394918+0000 (oldest deadline 2024-07-24T22:07:39.495038+0000) 2024-07-24T22:08:29.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-07-24T22:07:34.797000+0000 front 2024-07-24T22:07:34.797014+0000 (oldest deadline 2024-07-24T22:07:56.496858+0000) 2024-07-24T22:08:29.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-07-24T22:07:42.397949+0000 front 2024-07-24T22:07:42.397937+0000 (oldest deadline 2024-07-24T22:08:05.898014+0000) 2024-07-24T22:08:29.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-07-24T22:07:45.898561+0000 front 2024-07-24T22:07:45.898578+0000 (oldest deadline 2024-07-24T22:08:11.798559+0000) 2024-07-24T22:08:29.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-07-24T22:07:55.199885+0000 front 2024-07-24T22:07:55.200064+0000 (oldest deadline 2024-07-24T22:08:20.499812+0000) 2024-07-24T22:08:29.994 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:29 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11[64342]: 2024-07-24T22:08:29.606+0000 7fca3ca3a640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2024-07-24T22:08:01.501147+0000 front 2024-07-24T22:08:01.501169+0000 (oldest deadline 2024-07-24T22:08:27.901591+0000) 2024-07-24T22:08:31.243 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:30 smithi123 podman[77532]: 2024-07-24 22:08:30.985771368 +0000 UTC m=+5.680853858 container died b6f021a26669deebd804869b8f1f10eba50a3608ac7fc2c18f3688968fcab016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:08:31.634 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:31 smithi123 podman[77532]: 2024-07-24 22:08:31.348276329 +0000 UTC m=+6.043358826 container cleanup b6f021a26669deebd804869b8f1f10eba50a3608ac7fc2c18f3688968fcab016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-24T22:08:31.634 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:31 smithi123 bash[77532]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11 2024-07-24T22:08:31.969 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:31 smithi123 podman[77549]: 2024-07-24 22:08:31.63287665 +0000 UTC m=+0.645536374 container remove b6f021a26669deebd804869b8f1f10eba50a3608ac7fc2c18f3688968fcab016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, ceph=True, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-24T22:08:32.239 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:31 smithi123 podman[77637]: 2024-07-24 22:08:31.96878259 +0000 UTC m=+0.113053323 container create 40ff1540f3bcaed0925b716d5a0a7ab079fdef209e2f09f71a9f0156db339323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-07-24T22:08:32.239 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:31 smithi123 podman[77637]: 2024-07-24 22:08:31.872894582 +0000 UTC m=+0.017165271 image pull 2e906353c5e5862501954d038b33a590403821502f8379d78db37da9c924b2a0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb 2024-07-24T22:08:32.239 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:32 smithi123 podman[77637]: 2024-07-24 22:08:32.170961486 +0000 UTC m=+0.315232175 container init 40ff1540f3bcaed0925b716d5a0a7ab079fdef209e2f09f71a9f0156db339323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=main-86385b1) 2024-07-24T22:08:32.239 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:32 smithi123 podman[77637]: 2024-07-24 22:08:32.17393303 +0000 UTC m=+0.318203705 container start 40ff1540f3bcaed0925b716d5a0a7ab079fdef209e2f09f71a9f0156db339323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:08:32.239 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:32 smithi123 podman[77637]: 2024-07-24 22:08:32.237463927 +0000 UTC m=+0.381734616 container attach 40ff1540f3bcaed0925b716d5a0a7ab079fdef209e2f09f71a9f0156db339323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:32.493 INFO:journalctl@ceph.osd.11.smithi123.stdout:Jul 24 22:08:32 smithi123 podman[77637]: 2024-07-24 22:08:32.366881108 +0000 UTC m=+0.511151789 container died 40ff1540f3bcaed0925b716d5a0a7ab079fdef209e2f09f71a9f0156db339323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=main-86385b1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=) 2024-07-24T22:08:32.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@osd.11.service' 2024-07-24T22:08:32.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:32.978 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-07-24T22:08:32.978 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-07-24T22:08:32.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.a 2024-07-24T22:08:33.288 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:33 smithi060 systemd[1]: Stopping Ceph mds.a for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:33.572 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:33 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a[80944]: 2024-07-24T22:08:33.285+0000 7f2dcd6ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:33.572 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:33 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a[80944]: 2024-07-24T22:08:33.285+0000 7f2dcd6ee640 -1 mds.a *** got signal Terminated *** 2024-07-24T22:08:37.195 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:36 smithi060 podman[144099]: 2024-07-24 22:08:36.927833335 +0000 UTC m=+3.749242393 container died 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-07-24T22:08:37.546 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 podman[144099]: 2024-07-24 22:08:37.352142737 +0000 UTC m=+4.173551838 container cleanup 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-24T22:08:37.546 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 bash[144099]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a 2024-07-24T22:08:37.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.a.service' 2024-07-24T22:08:37.803 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 podman[144111]: 2024-07-24 22:08:37.545143166 +0000 UTC m=+0.614556145 container remove 270371cb0ea40a0a22ff65aa634f4eb13e18d33eb03388e10bf162bfb53c0477 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-24T22:08:37.803 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.a.service: Deactivated successfully. 2024-07-24T22:08:37.804 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 systemd[1]: Stopped Ceph mds.a for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:08:37.804 INFO:journalctl@ceph.mds.a.smithi060.stdout:Jul 24 22:08:37 smithi060 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.a.service: Consumed 1.227s CPU time. 2024-07-24T22:08:37.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:37.818 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-07-24T22:08:37.818 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-07-24T22:08:37.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.d 2024-07-24T22:08:38.072 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:37 smithi060 systemd[1]: Stopping Ceph mds.d for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:38.572 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d[79715]: 2024-07-24T22:08:38.172+0000 7fc09e30b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:38.572 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:38 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d[79715]: 2024-07-24T22:08:38.172+0000 7fc09e30b640 -1 mds.d *** got signal Terminated *** 2024-07-24T22:08:41.634 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:41 smithi060 podman[144216]: 2024-07-24 22:08:41.357797022 +0000 UTC m=+3.315779819 container died 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, ceph=True) 2024-07-24T22:08:42.059 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:41 smithi060 podman[144216]: 2024-07-24 22:08:41.80662893 +0000 UTC m=+3.764611716 container cleanup 63a2dbb60cf725bcc8e47b7b5b389febf41125de2975a1243e86ada0cd488035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, ceph=True, CEPH_POINT_RELEASE=) 2024-07-24T22:08:42.060 INFO:journalctl@ceph.mds.d.smithi060.stdout:Jul 24 22:08:41 smithi060 bash[144216]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-d 2024-07-24T22:08:42.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.d.service' 2024-07-24T22:08:42.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:42.270 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-07-24T22:08:42.271 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-07-24T22:08:42.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.g 2024-07-24T22:08:42.588 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:42 smithi060 systemd[1]: Stopping Ceph mds.g for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:43.072 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:42 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g[80517]: 2024-07-24T22:08:42.585+0000 7f992ed10640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:43.072 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:42 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g[80517]: 2024-07-24T22:08:42.585+0000 7f992ed10640 -1 mds.g *** got signal Terminated *** 2024-07-24T22:08:44.978 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:44 smithi060 podman[144338]: 2024-07-24 22:08:44.58032799 +0000 UTC m=+2.087613927 container died c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240716, RELEASE=main-86385b1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:08:44.979 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:44 smithi060 podman[144338]: 2024-07-24 22:08:44.977533427 +0000 UTC m=+2.484819363 container cleanup c4ec78250afc1061c718d9b65f9d03feb172abb49286b87c52c3c5f349e50d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-24T22:08:45.322 INFO:journalctl@ceph.mds.g.smithi060.stdout:Jul 24 22:08:44 smithi060 bash[144338]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-g 2024-07-24T22:08:45.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.g.service' 2024-07-24T22:08:45.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:45.509 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-07-24T22:08:45.510 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-07-24T22:08:45.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.j 2024-07-24T22:08:45.850 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:45 smithi060 systemd[1]: Stopping Ceph mds.j for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:46.322 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j[79379]: 2024-07-24T22:08:45.848+0000 7f34b3501640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:46.322 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:45 smithi060 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j[79379]: 2024-07-24T22:08:45.848+0000 7f34b3501640 -1 mds.j *** got signal Terminated *** 2024-07-24T22:08:50.116 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:49 smithi060 podman[144460]: 2024-07-24 22:08:49.861148065 +0000 UTC m=+4.116290757 container died 7cced47b312f32007c7884991cba658d47e462e5ae062510656ad8bc08e45d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:50.775 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:50 smithi060 podman[144460]: 2024-07-24 22:08:50.40714001 +0000 UTC m=+4.662282695 container cleanup 7cced47b312f32007c7884991cba658d47e462e5ae062510656ad8bc08e45d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-86385b1, ceph=True, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-24T22:08:50.775 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:50 smithi060 bash[144460]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j 2024-07-24T22:08:50.775 INFO:journalctl@ceph.mds.j.smithi060.stdout:Jul 24 22:08:50 smithi060 podman[144474]: 2024-07-24 22:08:50.575090556 +0000 UTC m=+0.712919277 container remove 7cced47b312f32007c7884991cba658d47e462e5ae062510656ad8bc08e45d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-24T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.j.service' 2024-07-24T22:08:50.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:50.819 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-07-24T22:08:50.819 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-07-24T22:08:50.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.b 2024-07-24T22:08:51.182 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:50 smithi086 systemd[1]: Stopping Ceph mds.b for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:51.443 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T22:08:51.180+0000 7f2fdfd7c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:51.444 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:51 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b[69195]: 2024-07-24T22:08:51.180+0000 7f2fdfd7c640 -1 mds.b *** got signal Terminated *** 2024-07-24T22:08:54.973 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:54 smithi086 podman[77414]: 2024-07-24 22:08:54.632353822 +0000 UTC m=+3.613626246 container died a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-24T22:08:55.265 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 podman[77414]: 2024-07-24 22:08:55.063118269 +0000 UTC m=+4.044390682 container cleanup a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:55.265 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 bash[77414]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b 2024-07-24T22:08:55.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.b.service' 2024-07-24T22:08:55.534 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 podman[77440]: 2024-07-24 22:08:55.264559543 +0000 UTC m=+0.634011186 container remove a498ea4c2ad86a1479260bacb458f422297d254c8937bd4c5fd6fbce683bbc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1) 2024-07-24T22:08:55.534 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.b.service: Deactivated successfully. 2024-07-24T22:08:55.534 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 systemd[1]: Stopped Ceph mds.b for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:08:55.534 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jul 24 22:08:55 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.b.service: Consumed 12min 39.812s CPU time. 2024-07-24T22:08:55.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:55.548 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-07-24T22:08:55.548 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-07-24T22:08:55.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.e 2024-07-24T22:08:55.796 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:55 smithi086 systemd[1]: Stopping Ceph mds.e for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:08:56.295 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: 2024-07-24T22:08:55.900+0000 7fd7fc015640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:08:56.295 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:55 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e[68587]: 2024-07-24T22:08:55.900+0000 7fd7fc015640 -1 mds.e *** got signal Terminated *** 2024-07-24T22:08:58.937 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:58 smithi086 podman[77548]: 2024-07-24 22:08:58.625923149 +0000 UTC m=+2.859895156 container died 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.build-date=20240716, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-24T22:08:59.236 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 podman[77548]: 2024-07-24 22:08:59.050488546 +0000 UTC m=+3.284460554 container cleanup 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, ceph=True, org.label-schema.build-date=20240716, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-07-24T22:08:59.236 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 bash[77548]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e 2024-07-24T22:08:59.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.e.service' 2024-07-24T22:08:59.491 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 podman[77560]: 2024-07-24 22:08:59.234982315 +0000 UTC m=+0.606528084 container remove 98e573c1dcd38ad544182a30debc9d9d7736c3895ea3d6d5315a75dc81078362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-e, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-86385b1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-07-24T22:08:59.491 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.e.service: Deactivated successfully. 2024-07-24T22:08:59.491 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 systemd[1]: Stopped Ceph mds.e for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:08:59.491 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jul 24 22:08:59 smithi086 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.e.service: Consumed 39.443s CPU time. 2024-07-24T22:08:59.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:08:59.503 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-07-24T22:08:59.503 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-07-24T22:08:59.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.h 2024-07-24T22:08:59.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:08:59 smithi086 systemd[1]: Stopping Ceph mds.h for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:00.295 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:08:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: 2024-07-24T22:08:59.812+0000 7fb12c249640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:00.295 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:08:59 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h[68889]: 2024-07-24T22:08:59.812+0000 7fb12c249640 -1 mds.h *** got signal Terminated *** 2024-07-24T22:09:02.795 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:09:02 smithi086 podman[77670]: 2024-07-24 22:09:02.623947982 +0000 UTC m=+2.910767113 container died 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-07-24T22:09:03.148 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:09:02 smithi086 podman[77670]: 2024-07-24 22:09:02.93767256 +0000 UTC m=+3.224491682 container cleanup 605db696520f7e807e7a0d7da3c60fb64683e0f3fa6552594c9705f1f32b9604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:09:03.148 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jul 24 22:09:02 smithi086 bash[77670]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-h 2024-07-24T22:09:03.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.h.service' 2024-07-24T22:09:03.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:03.318 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-07-24T22:09:03.318 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-07-24T22:09:03.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.k 2024-07-24T22:09:03.796 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:03 smithi086 systemd[1]: Stopping Ceph mds.k for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:03.796 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: 2024-07-24T22:09:03.624+0000 7fc190e22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:03.796 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:03 smithi086 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k[68272]: 2024-07-24T22:09:03.624+0000 7fc190e22640 -1 mds.k *** got signal Terminated *** 2024-07-24T22:09:07.045 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:06 smithi086 podman[77790]: 2024-07-24 22:09:06.629225175 +0000 UTC m=+3.090086184 container died 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-86385b1, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:09:07.443 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:07 smithi086 podman[77790]: 2024-07-24 22:09:07.166164705 +0000 UTC m=+3.627025717 container cleanup 72d2c88eba95a6ace15d9fe47ff3407cbc4f7d2a73c6862406d06acf98f5102d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:09:07.443 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jul 24 22:09:07 smithi086 bash[77790]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-k 2024-07-24T22:09:07.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.k.service' 2024-07-24T22:09:07.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:07.618 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-07-24T22:09:07.618 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-07-24T22:09:07.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.c 2024-07-24T22:09:07.949 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:07 smithi123 systemd[1]: Stopping Ceph mds.c for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:08.210 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: 2024-07-24T22:09:07.947+0000 7f3080e25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:08.210 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:07 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c[69471]: 2024-07-24T22:09:07.947+0000 7f3080e25640 -1 mds.c *** got signal Terminated *** 2024-07-24T22:09:10.914 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:10 smithi123 podman[77739]: 2024-07-24 22:09:10.599114553 +0000 UTC m=+2.799359967 container died 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240716) 2024-07-24T22:09:11.197 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 podman[77739]: 2024-07-24 22:09:11.003703408 +0000 UTC m=+3.203948809 container cleanup 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=main-86385b1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-24T22:09:11.198 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 bash[77739]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c 2024-07-24T22:09:11.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.c.service' 2024-07-24T22:09:11.465 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 podman[77764]: 2024-07-24 22:09:11.196292559 +0000 UTC m=+0.598415906 container remove 7c6e24d83f589b3d401a20eb18219fb0cd5577aec527e2aff260a016de796f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=main-86385b1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:09:11.465 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.c.service: Deactivated successfully. 2024-07-24T22:09:11.465 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 systemd[1]: Stopped Ceph mds.c for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:09:11.465 INFO:journalctl@ceph.mds.c.smithi123.stdout:Jul 24 22:09:11 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.c.service: Consumed 9min 12.168s CPU time. 2024-07-24T22:09:11.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:11.478 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-07-24T22:09:11.478 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-07-24T22:09:11.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f 2024-07-24T22:09:11.743 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:11 smithi123 systemd[1]: Stopping Ceph mds.f for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:12.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[74056]: 2024-07-24T22:09:11.798+0000 7f9f93bf3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:12.243 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:11 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f[74056]: 2024-07-24T22:09:11.798+0000 7f9f93bf3640 -1 mds.f *** got signal Terminated *** 2024-07-24T22:09:16.159 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:15 smithi123 podman[77872]: 2024-07-24 22:09:15.813239809 +0000 UTC m=+4.121966422 container died 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, RELEASE=main-86385b1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240716, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-24T22:09:16.459 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:16 smithi123 podman[77872]: 2024-07-24 22:09:16.240063646 +0000 UTC m=+4.548790243 container cleanup 76219245eeea8c93b6732e31a7e2985568f57c72c74e3f451d4b70f5d1b37953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240716, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-86385b1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-24T22:09:16.459 INFO:journalctl@ceph.mds.f.smithi123.stdout:Jul 24 22:09:16 smithi123 bash[77872]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-f 2024-07-24T22:09:16.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.f.service' 2024-07-24T22:09:16.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:16.706 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-07-24T22:09:16.706 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-07-24T22:09:16.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.i 2024-07-24T22:09:17.045 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:16 smithi123 systemd[1]: Stopping Ceph mds.i for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:17.493 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T22:09:17.043+0000 7f7b95b81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:17.493 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:17 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i[69770]: 2024-07-24T22:09:17.043+0000 7f7b95b81640 -1 mds.i *** got signal Terminated *** 2024-07-24T22:09:18.870 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:18 smithi123 podman[77994]: 2024-07-24 22:09:18.600326954 +0000 UTC m=+1.672414365 container died 79d23c86058f1bde6d6e47ab08bad4b7cb71d73b55d97440def60297cc344b59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-86385b1, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:09:19.171 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:18 smithi123 podman[77994]: 2024-07-24 22:09:18.952218045 +0000 UTC m=+2.024305454 container cleanup 79d23c86058f1bde6d6e47ab08bad4b7cb71d73b55d97440def60297cc344b59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-86385b1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240716, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-24T22:09:19.171 INFO:journalctl@ceph.mds.i.smithi123.stdout:Jul 24 22:09:18 smithi123 bash[77994]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-i 2024-07-24T22:09:19.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.i.service' 2024-07-24T22:09:19.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:19.351 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-07-24T22:09:19.351 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-07-24T22:09:19.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l 2024-07-24T22:09:19.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:19 smithi123 systemd[1]: Stopping Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968... 2024-07-24T22:09:19.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: 2024-07-24T22:09:19.672+0000 7fc87dd78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-24T22:09:19.743 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:19 smithi123 ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l[72886]: 2024-07-24T22:09:19.672+0000 7fc87dd78640 -1 mds.l *** got signal Terminated *** 2024-07-24T22:09:22.232 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:21 smithi123 podman[78113]: 2024-07-24 22:09:21.977389081 +0000 UTC m=+2.404243571 container died 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-86385b1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-24T22:09:22.493 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 podman[78113]: 2024-07-24 22:09:22.31409848 +0000 UTC m=+2.740952980 container cleanup 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=main-86385b1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240716, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-24T22:09:22.493 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 bash[78113]: ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l 2024-07-24T22:09:22.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service' 2024-07-24T22:09:22.762 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 podman[78127]: 2024-07-24 22:09:22.506808746 +0000 UTC m=+0.523890927 container remove 9c7625390a40675f36cdd4d1bbc544b91a32c0585e4232fd618e87de6f99286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af9a789bf27d9163de7a71d6f7d1087cc5b5534cc45ac7ad0b7b3df8cafcb1eb, name=ceph-646042b4-4a04-11ef-bcad-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240716, RELEASE=main-86385b1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-07-24T22:09:22.762 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Deactivated successfully. 2024-07-24T22:09:22.763 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 systemd[1]: Stopped Ceph mds.l for 646042b4-4a04-11ef-bcad-c7b262605968. 2024-07-24T22:09:22.763 INFO:journalctl@ceph.mds.l.smithi123.stdout:Jul 24 22:09:22 smithi123 systemd[1]: ceph-646042b4-4a04-11ef-bcad-c7b262605968@mds.l.service: Consumed 10.223s CPU time. 2024-07-24T22:09:22.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-24T22:09:22.773 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-07-24T22:09:22.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force --keep-logs 2024-07-24T22:09:22.989 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:09:25.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force --keep-logs 2024-07-24T22:09:25.799 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:09:28.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force --keep-logs 2024-07-24T22:09:28.526 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:09:30.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:09:30.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:09:30.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-24T22:09:31.008 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-24T22:09:31.011 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi060/crash 2024-07-24T22:09:31.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash -- . 2024-07-24T22:09:31.041 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash: Cannot open: No such file or directory 2024-07-24T22:09:31.041 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-07-24T22:09:31.043 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi086/crash 2024-07-24T22:09:31.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash -- . 2024-07-24T22:09:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash: Cannot open: No such file or directory 2024-07-24T22:09:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-07-24T22:09:31.079 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi123/crash 2024-07-24T22:09:31.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash -- . 2024-07-24T22:09:31.117 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/646042b4-4a04-11ef-bcad-c7b262605968/crash: Cannot open: No such file or directory 2024-07-24T22:09:31.118 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-07-24T22:09:31.119 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-24T22:09:31.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-24T22:09:31.159 INFO:teuthology.orchestra.run.smithi060.stdout:2024-07-24T21:50:42.381353+0000 mds.i (mds.0) 220 : cluster [WRN] Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details 2024-07-24T22:09:31.160 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-07-24T22:09:31.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[SEC\]' /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-24T22:09:31.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]' /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-24T22:09:31.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[WRN\]' /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-24T22:09:31.394 INFO:teuthology.orchestra.run.smithi060.stdout:2024-07-24T21:50:42.381353+0000 mds.i (mds.0) 220 : cluster [WRN] Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details 2024-07-24T22:09:31.394 INFO:tasks.cephadm:Compressing logs... 2024-07-24T22:09:31.394 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-24T22:09:31.437 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-24T22:09:31.439 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-24T22:09:31.461 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log 2024-07-24T22:09:31.461 INFO:teuthology.orchestra.run.smithi060.stderr:: No such file or directory 2024-07-24T22:09:31.461 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log 2024-07-24T22:09:31.461 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.g.log 2024-07-24T22:09:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.j.log 2024-07-24T22:09:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.3.log 2024-07-24T22:09:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.0.log 2024-07-24T22:09:31.465 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.x.log 2024-07-24T22:09:31.465 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.a.log 2024-07-24T22:09:31.465 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.6.log 2024-07-24T22:09:31.465 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log.gz 2024-07-24T22:09:31.466 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log 2024-07-24T22:09:31.468 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.g.log.gz 2024-07-24T22:09:31.470 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 94.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.j.log.gz 2024-07-24T22:09:31.470 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.a.log 2024-07-24T22:09:31.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log 2024-07-24T22:09:31.470 INFO:teuthology.orchestra.run.smithi060.stderr: 84.4% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log.gz 2024-07-24T22:09:31.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.a.log: 90.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log.gz 2024-07-24T22:09:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.d.log 2024-07-24T22:09:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.9.log 2024-07-24T22:09:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-24T22:09:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log 2024-07-24T22:09:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log 2024-07-24T22:09:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.7.log 2024-07-24T22:09:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log: /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.b.log 2024-07-24T22:09:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.4.log 2024-07-24T22:09:31.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.a.log.gz 2024-07-24T22:09:31.480 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-24T22:09:31.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.d.log.gz 2024-07-24T22:09:31.481 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.1.log 2024-07-24T22:09:31.481 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.y.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.h.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.10.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.e.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.b.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.k.log 2024-07-24T22:09:31.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-24T22:09:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-24T22:09:31.485 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.2.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.f.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log: /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.5.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.c.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.c.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.z.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.i.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log 2024-07-24T22:09:31.486 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.11.log 2024-07-24T22:09:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.8.log 2024-07-24T22:09:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log 2024-07-24T22:09:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.l.log 2024-07-24T22:09:31.487 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log: -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-24T22:09:31.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log.gz 2024-07-24T22:09:31.488 INFO:teuthology.orchestra.run.smithi060.stderr: 95.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-24T22:09:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 82.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log.gz 2024-07-24T22:09:31.592 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.z.log.gz 2024-07-24T22:09:31.631 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.y.log.gz 2024-07-24T22:09:31.698 INFO:teuthology.orchestra.run.smithi123.stderr: 82.1% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.cephadm.log.gz 2024-07-24T22:09:31.729 INFO:teuthology.orchestra.run.smithi123.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-24T22:09:31.804 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log.gz 2024-07-24T22:09:31.830 INFO:teuthology.orchestra.run.smithi060.stderr: 90.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mgr.x.log.gz 2024-07-24T22:09:31.921 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-volume.log.gz 2024-07-24T22:09:31.946 INFO:teuthology.orchestra.run.smithi123.stderr: 90.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log.gz 2024-07-24T22:09:31.994 INFO:teuthology.orchestra.run.smithi123.stderr: 90.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log.gz 2024-07-24T22:09:32.011 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.audit.log.gz 2024-07-24T22:09:32.016 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-24T22:09:32.152 INFO:teuthology.orchestra.run.smithi086.stderr: 90.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph.log.gz 2024-07-24T22:09:33.089 INFO:teuthology.orchestra.run.smithi086.stderr: 94.9% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.k.log.gz 2024-07-24T22:09:33.365 INFO:teuthology.orchestra.run.smithi060.stderr: 90.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.a.log.gz 2024-07-24T22:09:49.251 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.b.log.gz 2024-07-24T22:09:50.600 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.9.log.gz 2024-07-24T22:09:50.891 INFO:teuthology.orchestra.run.smithi123.stderr: 91.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mon.c.log.gz 2024-07-24T22:10:05.403 INFO:teuthology.orchestra.run.smithi060.stderr: 93.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.6.log.gz 2024-07-24T22:10:09.685 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.0.log.gz 2024-07-24T22:10:11.208 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.3.log.gz 2024-07-24T22:10:11.210 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-07-24T22:10:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m39.760s 2024-07-24T22:10:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:user 2m8.744s 2024-07-24T22:10:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m4.321s 2024-07-24T22:10:46.696 INFO:teuthology.orchestra.run.smithi123.stderr: 95.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.l.log.gz 2024-07-24T22:11:56.827 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.8.log.gz 2024-07-24T22:11:58.011 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.2.log.gz 2024-07-24T22:11:59.298 INFO:teuthology.orchestra.run.smithi086.stderr: 95.1% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.e.log.gz 2024-07-24T22:12:08.700 INFO:teuthology.orchestra.run.smithi123.stderr: 95.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.f.log.gz 2024-07-24T22:13:47.895 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.11.log.gz 2024-07-24T22:14:18.381 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.4.log.gz 2024-07-24T22:14:18.396 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.1.log.gz 2024-07-24T22:15:06.957 INFO:teuthology.orchestra.run.smithi123.stderr: 93.6% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.5.log.gz 2024-07-24T22:15:24.364 INFO:teuthology.orchestra.run.smithi086.stderr: 94.9% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.h.log.gz 2024-07-24T22:16:48.951 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.10.log.gz 2024-07-24T22:17:59.042 INFO:teuthology.orchestra.run.smithi086.stderr: 93.8% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-osd.7.log.gz 2024-07-24T22:22:19.681 INFO:teuthology.orchestra.run.smithi086.stderr: 94.0% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.b.log.gz 2024-07-24T22:22:19.694 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-07-24T22:22:19.694 INFO:teuthology.orchestra.run.smithi086.stderr:real 12m48.240s 2024-07-24T22:22:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:user 6m56.407s 2024-07-24T22:22:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m19.651s 2024-07-24T22:32:01.352 INFO:teuthology.orchestra.run.smithi123.stderr: 93.9% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.c.log.gz 2024-07-24T22:33:05.634 INFO:teuthology.orchestra.run.smithi123.stderr: 94.3% -- replaced with /var/log/ceph/646042b4-4a04-11ef-bcad-c7b262605968/ceph-mds.i.log.gz 2024-07-24T22:33:05.664 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-07-24T22:33:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:real 23m34.210s 2024-07-24T22:33:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:user 9m21.148s 2024-07-24T22:33:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m27.618s 2024-07-24T22:33:05.665 INFO:tasks.cephadm:Archiving logs... 2024-07-24T22:33:05.665 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi060/log 2024-07-24T22:33:05.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-24T22:33:23.783 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi086/log 2024-07-24T22:33:23.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-24T22:34:23.860 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi123/log 2024-07-24T22:34:23.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-24T22:35:42.230 INFO:tasks.cephadm:Removing cluster... 2024-07-24T22:35:42.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force 2024-07-24T22:35:42.449 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:35:42.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force 2024-07-24T22:35:43.262 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:35:43.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 646042b4-4a04-11ef-bcad-c7b262605968 --force 2024-07-24T22:35:45.446 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 646042b4-4a04-11ef-bcad-c7b262605968 2024-07-24T22:35:46.453 INFO:tasks.cephadm:Removing cephadm ... 2024-07-24T22:35:46.454 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-24T22:35:46.475 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-24T22:35:46.495 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-24T22:35:46.510 INFO:tasks.cephadm:Teardown complete 2024-07-24T22:35:46.511 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-24T22:35:46.522 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-24T22:35:46.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-24T22:35:46.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-24T22:35:46.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-24T22:35:46.652 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-24T22:35:46.662 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-24T22:35:46.663 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T22:35:46.665 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T22:35:46.668 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-24T22:35:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-07-24T22:35:46.682 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-07-24T22:35:46.682 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -581us[ -579us] +/- 38ms 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -621us[ -620us] +/- 72ms 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +1542us[+1543us] +/- 67ms 2024-07-24T22:35:46.686 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T22:35:46.707 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T22:35:46.707 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-07-24T22:35:46.707 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -584us[ -588us] +/- 38ms 2024-07-24T22:35:46.707 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -615us[ -615us] +/- 72ms 2024-07-24T22:35:46.708 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +1558us[+1558us] +/- 67ms 2024-07-24T22:35:46.708 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T22:35:46.757 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-24T22:35:46.758 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-07-24T22:35:46.758 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -560us[ -584us] +/- 38ms 2024-07-24T22:35:46.758 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 68 -575us[ -600us] +/- 71ms 2024-07-24T22:35:46.758 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +1584us[+1560us] +/- 67ms 2024-07-24T22:35:46.758 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-24T22:35:46.758 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-24T22:35:46.769 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-24T22:35:46.770 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-24T22:35:46.846 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-24T22:35:46.947 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-24T22:35:47.075 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-24T22:35:47.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T22:35:47.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:47.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T22:35:47.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:47.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-24T22:35:47.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:47.325 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-24T22:35:47.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-24T22:35:47.389 INFO:teuthology.task.internal:Duration was 4108.599866 seconds 2024-07-24T22:35:47.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-24T22:35:47.439 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-24T22:35:47.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-24T22:35:47.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-24T22:35:47.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-24T22:35:47.487 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T22:35:47.496 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T22:35:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-24T22:35:47.823 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-24T22:35:47.823 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-07-24T22:35:47.823 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-24T22:35:47.849 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-07-24T22:35:47.849 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-24T22:35:47.878 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-07-24T22:35:47.878 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-24T22:35:47.906 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-24T22:35:47.906 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-24T22:35:47.909 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-24T22:35:47.921 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-24T22:35:47.986 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-24T22:35:47.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-24T22:35:47.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-24T22:35:47.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-24T22:35:49.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-24T22:35:49.520 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-24T22:35:49.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-24T22:35:49.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-24T22:35:49.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-24T22:35:49.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-24T22:35:49.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.704 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-07-24T22:35:49.719 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-07-24T22:35:49.722 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-07-24T22:35:49.752 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:49.774 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:49.810 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-24T22:35:49.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-24T22:35:49.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-24T22:35:49.837 INFO:teuthology.task.internal:Transferring archived files... 2024-07-24T22:35:49.839 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi060 2024-07-24T22:35:49.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-24T22:35:49.916 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi086 2024-07-24T22:35:49.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-24T22:35:49.957 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-07-24_14:14:53-fs-main-testing-default-smithi/7816828/remote/smithi123 2024-07-24T22:35:49.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-24T22:35:49.997 INFO:teuthology.task.internal:Removing archive directory... 2024-07-24T22:35:49.997 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-24T22:35:50.000 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-24T22:35:50.002 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-24T22:35:50.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-24T22:35:50.059 INFO:teuthology.task.internal:Not uploading archives. 2024-07-24T22:35:50.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-24T22:35:50.128 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-24T22:35:50.128 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-24T22:35:50.130 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-24T22:35:50.133 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-24T22:35:50.143 INFO:teuthology.orchestra.run.smithi060.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 24 22:35 /home/ubuntu/cephtest 2024-07-24T22:35:50.148 INFO:teuthology.orchestra.run.smithi086.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 24 22:35 /home/ubuntu/cephtest 2024-07-24T22:35:50.148 INFO:teuthology.orchestra.run.smithi123.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 24 22:35 /home/ubuntu/cephtest 2024-07-24T22:35:50.151 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-24T22:35:50.212 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-24T22:35:50.318 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 4108.5998656749725 failure_reason: '"2024-07-24T21:50:42.381353+0000 mds.i (mds.0) 220 : cluster [WRN] Scrub error on inode 0x10000000881 (/volumes/qa/sv_1/5bf008ce-3167-4364-b97f-6c159317768b/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.i log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-07-24T22:35:50.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-24T22:35:50.406 INFO:teuthology.run:FAIL